Zachary Piper Solutions is seeking a Software Engineer HYBRID in Elkridge, MD
Responsibilities of the Software Engineer include:
Serve as a Databricks subject matter expert, leveraging its full suite of tools to build, deploy, and manage scalable data and AI solutions.Design, develop, and maintain ETL pipelines using Databricks, Spark, Delta Lake, Delta Live Tables, and Databricks Workflows.Transform and ingest data from external systems using protocols such as HTML and SFTP.Collaborate with developers and external teams to validate data ingestion and integration processes.Enhance the ETL platform with reusable components and automation to reduce integration timelines.Ensure high data quality, performance, and reliability across large-scale datasets.Provide detailed documentation of system architecture, development processes, and enhancements....