We are seeking a Hadoop Site Reliability Engineer with hands-on experience in Big Data ecosystems and cloud-based Hadoop platforms. The ideal candidate will provide administration, monitoring, and performance tuning for Hadoop, Spark, Hive, HBase, and Kafka workloads while supporting enterprise customers in production environments. This role requires expertise in Hadoop administration, cloud platforms (AWS, CloudEra, HDInsight), troubleshooting production issues, and building tools to improve system reliability and supportability.
Required Technical / Functional Skills
- Expertise in Big Data and Hadoop Ecosystems (Hortonworks, CloudEra)
- Hands-on experience with Kafka for streaming data
- Experience with HBase administration
- Knowledge of Spark, Hive, and Hadoop performance tuning
- Familiarity with cloud platforms like AWS HDInsight
- Strong problem-solving and root-cause analysis skills
- Ability to monitor and maintain cluster health for enterprise workloads
- Experience building tools to improve system supportability and debuggability
Roles & Responsibilities
- Perform Hadoop administration for enterprise workloads (CloudEra, Hortonworks, HDInsight, AWS)
- Support HDInsight product team and resolve end-customer issues via ICMs
- Run prototypes for migrating Big Data workloads to new platforms
- Troubleshoot production issues, identify root causes, and provide timely mitigation
- Tune Spark, Hive, and Hadoop jobs for optimal performance
- Build tools and services to improve system debuggability and supportability
- Monitor cluster health, ensure reliability, and provide proactive support for top customers
Technologies / Tools
- Hadoop Ecosystem (HDFS, YARN, MapReduce)
- Spark, Hive
- HBase
- Kafka
- Cloud Platforms: AWS, CloudEra, HDInsight
- Performance Tuning & Monitoring Tools