We are looking for a Senior Hadoop Developer with strong experience in Big Data, Hadoop/Cloudera, Spark, Python, SQL, and ETL to design, develop, and support scalable data solutions in a production environment.
Roles and Requirements
- 4+ years of experience in Big Data and data warehousing.
- Strong hands-on experience with Hadoop/Cloudera, HDFS, Hive, and Spark/PySpark.
- Proficiency in Python, Scala, SQL, and Unix/Shell scripting.
- Strong experience in ETL, data ingestion, transformation, and integration.
- Experience with complex SQL query optimization and performance tuning.
- Experience with AutoSys or similar enterprise scheduling tools.
- Strong understanding of distributed systems and large-scale data processing.
- Experience supporting production Big Data applications and troubleshooting critical issues.
- Understanding of Agile methodologies and enterprise development/operational processes.
- Strong analytical, problem-solving, communication, and collaboration skills.
- Experience designing analytical/data warehouse environments and BI solutions.
- Financial services/banking experience preferred.
Primary Responsibilities
- Develop and maintain scalable Hadoop/Spark-based data pipelines.
- Support data ingestion from multiple source systems.
- Develop ETL and data transformation solutions using Hive, Spark, Python, and SQL.
- Optimize SQL queries and Big Data jobs for performance.
- Troubleshoot production issues and perform root-cause analysis.
- Support scheduling, monitoring, and operational processes.
- Collaborate with architects, developers, analysts, and business teams.
- Evaluate and adopt emerging Big Data technologies where appropriate.