Required Skills

• Strong hands-on experience with Apache Spark 3.5.x or later.


• Expertise in Spark DataFrames, Datasets, RDDs, and Spark SQL.


• Strong understanding of Spark architecture, execution model, and distributed computing principles.


• Experience designing and building scalable ETL pipelines for batch and real-time data processing.


• Proven experience in Spark job tuning and performance optimization.


• Hands-on experience with cloud platforms such as AWS and/or Azure.


• Experience with cloud storage solutions including Amazon S3 and Azure Data Lake Storage (ADLS).


• Strong SQL and data modeling skills.


• Experience working in Linux/Unix environments.

Preferred Skills

• Experience with Snowflake for data warehousing and analytics.


• Exposure to streaming/real-ti...