America's Job Portal
We are looking for a Data Pipeline Engineer with strong expertise in Databricks, AWS Glue, Python, SQL, and MongoDB to design, develop, and optimize scalable data pipelines. The ideal candidate should have experience building cloud-native ETL solutions, transforming large datasets, and delivering high-quality data for analytics and business applications.
Key Responsibilities:
β Develop and maintain scalable ETL/data pipelines using Databricks and AWS Glue.
β Build data ingestion and transformation workflows for structured and semi-structured data.
β Write efficient PySpark, Python, and SQL code for large-scale data processing.
β Integrate data from multiple sources, including MongoDB and relational databases.
β Implement data quality, validation, and reconciliation checks.
β Optimize pipeline performance, reliability, and scalability.
β Collaborate with architects, analysts, and application teams to deliver data solutions.
β Support prod...