Roles and Responsibilities

• Design, deploy, and manage Hadoop ecosystem components (YARN, HMS, SPARK, HIVE, IMPALA) for optimal performance and scalability.
• Automate deployment pipelines and configurations using DevOps tools like Ansible, Chef, and implement CI/CD workflows.
• Troubleshoot, debug, and optimize critical Hadoop components (YARN, HMS, SPARK) to resolve performance bottlenecks and system issues.
• Collaborate with Hadoop admins, application stakeholders, and Linux system admins to coordinate upgrades, patches, and automation.
• Conduct regular system health checks, monitor performance metrics, and suggest system improvements for efficiency and stability.
• Leverage data observability tools (Unravel, Pepperdata) to monitor and analyze application performance, identifying and resolving issues.
• Document configurations, scripts, troubleshooting steps, and procedures for knowledge sharing and transparency.
• Provide regular reports on system performance, issues, and improvements to key stakeholders.

Required Skills & Qualifications

• 10+ years in IT, with 4+ years of experience in DevOps and Hadoop Administration.
• Strong understanding of Hadoop Architecture, including components like YARN, HMS, SPARK, HIVE, and IMPALA.
• Hands-on experience with Ansible, Chef, or similar configuration management and automation tools.
• Ability to debug and troubleshoot critical Hadoop components and fine-tune systems for optimal performance.
• Proven ability to work with cross-functional teams (Hadoop admins, application developers, system administrators) to coordinate upgrades, patches, and automation efforts.
• Familiarity with Data Observability tools like Unravel and Pepperdata to monitor and improve big data application performance.
• Solid understanding of Linux/Unix systems, with experience in managing system parameters and performing health checks.
• Strong analytical skills, with the ability to quickly diagnose and resolve complex issues in a distributed environment.
• Ability to communicate effectively with both technical and non-technical stakeholders.

Posted By