Lead Data Engineer
Outside IR35
£500–£525 per day
3 Month Initial Contract
 
We're looking for an experienced Lead Data Engineer to join a high-profile project. This is a hybrid role, requiring 2 days per week onsite in London, with the remainder worked remotely.

What we're looking for:

Proven experience leading data engineering teams, with a strong track record of mentoring and developing junior engineers.
Expert knowledge of Python and Airbyte CDK, with experience building custom data connectors (a key focus of the role).
Strong experience developing ELT pipelines using PySpark in Databricks within a Medallion Architecture.
Experience with Terraform and deploying Airbyte connections and Databricks assets using GitHub Actions.
Confident working with JIRA, GitHub, and VS Code. 
Desirable:

Experience with Databricks Delta Live Tables (DLT).
Knowledge of Databricks Asset Bundles.