Position Overview As a Software Engineer specializing in Apache Airflow, you will build and manage complex data pipeline orchestration to ensure reliable data delivery. You will work at the intersection of data engineering and automation, designing scalable workflows that power the organization’s data‑driven decision‑making.
Responsibilities - Design, develop, and maintain complex Directed Acyclic Graphs (DAGs) to orchestrate data movement and transformation.
- Manage task dependencies and scheduling logic to ensure timely and accurate data processing across various systems.
- Monitor workflow execution and implement proactive alerting to identify and resolve pipeline failures or performance bottlenecks.
- Optimize Airflow infrastructure and operator configurations to handle increasing data volumes and concurrency requirements.
- Collaborate with data engineers and analysts to translate business requirements into efficient ...