America's Job Portal
Description
& Summary: We are seeking an experienced Data Engineer β Databricks to design, build, and operate scalable, high-performance data pipelines on the Databricks Lakehouse Platform. The role involves hands-on development using Apache Spark, Databricks notebooks, Delta Lake, and cloud-native services, along with close collaboration with analytics, AI/ML, and business teamsResponsibilities:
Β· Design, build, and maintain end-to-end data pipelines using Databricks (PySpark / Spark SQL) β’ Implement batch and incremental data processing using Delta Lake and multi-hop architecture β’ Develop and optimize Databricks notebooks, jobs, and workflows β’ Ingest, transform, and curate large-scale structured and semi-structured datasets β’ Support analytics, reporting, and downstream data consumption use cases β’ Ensure data quality, reliability, lineage, and governance β’ Collaborate with data scientists, analysts, and architects on AI/ML workloads β’ Optimize Spark j...