Position Overview
Description We are looking for an experienced Data Engineer to join a team delivering modern data solutions in Poughkeepsie, New York. This Long-term Contract position focuses on building reliable, scalable data platforms in Databricks while supporting analytics needs across the business. The role calls for a hands-on engineer who can improve data performance, uphold governance standards, and partner effectively with cross-functional stakeholders in an agile environment.
Responsibilities:
β’ Create and support scalable data pipelines in Databricks using Spark technologies such as PySpark or Scala to process and deliver high-quality data.
β’ Develop lakehouse architectures on Azure Data Lake Storage Gen2 and ensure strong integration with Databricks for efficient data management.
β’ Establish and monitor data quality controls and governance practices within the platform using validation methods and Delta Lake capabilities.
β’ Investigate pipeline and application ineffic...