We are seeking an experienced Hadoop Administrator with 5+ years of relevant experience to manage, secure, and maintain large-scale Big Data clusters. The ideal candidate will have strong expertise in Hadoop administration, cluster management, performance optimization, and high availability of core Hadoop components, including HDFS and YARN.
Key Responsibilities
- Install, configure, provision, and administer Hadoop clusters in large-scale enterprise environments.
- Manage and maintain core Hadoop components, including HDFS and YARN.
- Monitor cluster health, troubleshoot infrastructure issues, and ensure high availability and reliability.
- Perform capacity planning, resource allocation, and cluster scaling to support growing data requirements.
- Optimize Hadoop cluster performance through tuning, monitoring, and proactive maintenance.
- Manage HDFS storage, permissions, replication, and data availability.
- Administer YARN resource management and troubleshoot job execution and resource allocation issues.
- Implement and maintain security policies, access controls, authentication, and authorization mechanisms.
- Perform routine cluster maintenance, upgrades, patching, and configuration management.
- Identify and resolve performance bottlenecks, system failures, and operational issues.
- Support backup, recovery, disaster recovery, and business continuity procedures.
- Collaborate with development, data engineering, and infrastructure teams to ensure reliable Big Data operations.
- Participate in incident management and support 24/7 availability of critical Hadoop services as required.
- Maintain technical documentation, operational procedures, and cluster administration standards.
Required Skills and Qualifications
- Minimum 5+ years of hands-on experience in Hadoop Administration.
- Strong knowledge of Hadoop architecture and distributed computing environments.
- Extensive experience administering and maintaining HDFS and YARN.
- Experience with Hadoop cluster installation, configuration, provisioning, and monitoring.
- Expertise in cluster performance tuning, capacity planning, and troubleshooting.
- Knowledge of Hadoop security, access management, and cluster reliability best practices.
- Experience with cluster upgrades, patching, maintenance, and incident resolution.
- Strong analytical, problem-solving, and communication skills.
- Ability to work effectively with cross-functional technical teams.