We are seeking a highly skilled and experienced BigData/PySpark Engineer to join our dynamic Big Data Analytics team. This role is pivotal in designing, developing, and optimizing robust, scalable data pipelines for large-scale data processing and analytics.
Key Responsibilities:
Design & Development: Create and optimize scalable ETL (Extraction, Transformation, Loading) pipelines using PySpark for massive datasets. Coding & Engineering: Write clean, efficient, well-documented code primarily in Python (PySpark) often leveraging frameworks/tools. Collaboration: Work with cross-functional teams (senior developers, data engineers, analysts, business partners) to understand data requirements and ensure seamless solution integration. Troubleshooting & Optimization: Debug and resolve data processing issues and performance bottlenecks in Spark applications and other big data technologies. Full SDLC Involvement: Participate in the entire so...