We are seeking an experienced and skilled Hadoop Platform Administrator to manage and support our Hadoop Big Data platform. The ideal candidate will have in-depth knowledge of Hadoop ecosystem components, experience with system administration, and strong problem-solving skills. You will be responsible for the setup, monitoring, troubleshooting, and optimization of Hadoop clusters, ensuring high availability, performance, and security of the platform. Additionally, you'll collaborate with cross-functional teams and external vendors like Cloudera to resolve issues and ensure smooth operations.

Responsibilities

• Administer and maintain Hadoop clusters, ensuring optimal performance, scalability, and reliability.
• Install, configure, and upgrade Hadoop components (e.g., HDFS, YARN, Hive, HBase, Spark, etc.) as needed.
• Perform capacity planning, cluster scaling, and tuning to meet business requirements.
• Handle cluster backups, restore, and disaster recovery procedures
• Manage and resolve incidents and problems related to Hadoop, escalating as necessary to vendor support.
• Set up and manage monitoring tools to ensure visibility into the health of the Hadoop cluster.
• Perform ongoing performance tuning and optimization of Hadoop clusters.
• Develop and automate regular cluster health checks and maintenance tasks.
• Develop and maintain automation scripts for routine cluster management tasks using Python and Linux shell scripting.
• Create and manage automated workflows to improve operational efficiency and reduce manual intervention.
• Perform regular system administration tasks on Linux/Unix servers supporting the Hadoop cluster.
• Manage user access control, data encryption, and audit logging within the Hadoop ecosystem.

Required Skills/Qualifications

• Proven experience as a Hadoop Administrator with hands-on experience in managing large-scale Hadoop clusters in production environments.
• Strong experience in supporting and maintaining Hadoop-based Big Data platforms.
• Expertise in Cloudera Hadoop distribution or other Hadoop distributions (e.g., Hortonworks, MapR).
• Proficiency in Linux/Unix system administration and shell scripting.
• Strong experience with monitoring tools like Dynatrace, Monpo, Logscale, and Grafana.
• Hands-on experience with Python for automation and scripting.
• Solid understanding of Hadoop components such as HDFS, YARN, Hive, HBase, Spark, and ZooKeeper.

Posted By