As a Software Engineer specializing in Apache Spark, you will develop and optimize large-scale data processing solutions to handle complex batch and micro-batch workloads. You will play a key role in ensuring the performance and cost-efficiency of our data pipelines while enabling data-driven insights at scale.

What You'll Do

• Design and develop high-performance data processing jobs using Apache Spark, PySpark, and Scala.
• Build and maintain scalable data pipelines on platforms like Databricks to support enterprise analytics.
• Optimize Spark applications for memory management, CPU utilization, and overall execution cost.
• Implement robust data transformations and ensure data quality across large-scale distributed datasets.
• Collaborate with data architects to refine data models and improve the reliability of the processing ecosystem.

What We Are Looking For

• 4+ years of experience in Softwar...