We are seeking a Senior Big Data Engineer with 8+ years of IT experience and strong expertise in Hadoop, Spark, Python, and Snowflake. The ideal candidate will have hands-on experience building scalable data pipelines, migrating data from Hadoop to Snowflake using PySpark, and developing enterprise ETL solutions.
Roles and Responsibilities
Design, develop, and maintain scalable data pipelines using Hadoop, Spark, Python, and Snowflake.
Perform data migration from Hadoop to Snowflake using PySpark frameworks and libraries.
Build and optimize ETL workflows for large-scale data ingestion, transformation, and validation.
Develop and optimize complex SQL queries for data processing and analytics.
Ensure data quality, performance, and reliability across enterprise data platforms.
Collaborate with cross-functional teams to gather requirements, troubleshoot issues, and deliver data solutions.
Required Qualifications
8+ years of overall IT experience in data engineering or big data development.
Strong hands-on experience with Hadoop, Apache Spark, Python, and Snowflake.
Experience migrating enterprise data from Hadoop to Snowflake using PySpark.
Solid understanding of ETL processes, data integration, and data transformation techniques.
Strong SQL skills with experience writing and optimizing complex queries.
Excellent analytical, problem-solving, and communication skills.
Experience working in Unix/Linux environments. (Preferred)