Role Overview
As a Data Engineer specializing in Azure Databricks, you will design and maintain scalable data architecture within the Microsoft cloud ecosystem. You will act as a link between raw data sources and actionable insights, ensuring that ingestion pipelines and storage systems are optimized for high-performance analytics and reporting.
Responsibilities
- Design, develop, and maintain automated data pipelines for large‑scale ingestion and transformation using Azure Databricks.
- Implement and optimize ETL/ELT processes to convert raw data into structured formats suitable for downstream consumption.
- Build and manage data lakehouse architectures, ensuring data consistency and reliability across different storage layers.
- Collaborate with analysts and data scientists to define data requirements and develop models that support complex business queries.
- Monitor pipeline performance and proactively troublesho...