Data Engineer TASKS AND RESPONSIBILITIES Build and optimize ETL/ELT pipelines using AWS Glue, Spark, Athena, Lake Formation, S3, Step Functions, Lambda.
Contributes to design and implementation of AWS Data Lake, including orchestration and security guardrails.
Collaborate with Dev Sec Ops engineers on infrastructure automation and CI/CD pipelines using tools such as Git Hub Actions and Terraform.
Automate deployment and management of data infrastructure and workflows to ensure reliability and scalability.
Define and enforce data standards, including metadata, lineage and governance.
Ensure platform scalability, performance and cost efficiency (e.g., partitioning, caching strategies).
Coordinate integration needs, including feature engineering and data validation.
Participate in validation and documentation efforts aligned with Gx P.
Engage with stakeholders to align technical design with business needs.
Manage and reduce technical debt in data pipelines and i...