Spark Admin

  • Salary

    $100,000 - $110,000 Per Year


7 to 10 Years

Posted On

May 26, 2023


2T Consulting

0 View

1 Applicants

Job Overview

Required Skills

  • Spark cluster
  • Container
  • Linux / AIX / Unix
  • Mesos / Kubernetes / Docker Datacenter
  • Puppet / Chef / Ansible
  • Hadoop

Job Type


Work Authorization



Charlotte, NC/ Dallas, TX/ Jersey City, NJ

Job Description

We are seeking an experienced Spark Admin with Hadoop expertise to join our team. As a Spark Admin, you will be responsible for managing and maintaining our Spark and Hadoop clusters, ensuring their optimal performance, availability, and security. You will work closely with cross-functional teams to support data processing and analysis requirements.


  • Manage and administer Spark and Hadoop clusters, ensuring their smooth operation and optimal performance.
  • Install, configure, and deploy Spark and Hadoop components, including HDFS, YARN, Hive, Spark SQL, and other related tools.
  • Monitor cluster health, resource utilization, and job performance using cluster management and monitoring tools.
  • Troubleshoot and resolve issues related to cluster performance, data processing, and job failures.
  • Implement security measures and access controls to protect data and ensure compliance with data governance policies.
  • Collaborate with data engineers and data scientists to optimize data processing workflows and job scheduling.
  • Stay up to date with the latest Spark and Hadoop releases, patches, and best practices.
  • Automate administrative tasks and develop scripts to improve efficiency and productivity.
  • Provide technical support and guidance to development teams and end users.
  • Conduct performance tuning and optimization of Spark and Hadoop configurations.


  • Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.
  • Minimum of 5 years of experience as a Spark Admin, Hadoop Administrator, or a similar role.
  • Strong proficiency in Spark and Hadoop administration, including installation, configuration, and maintenance.
  • Experience with cluster management and monitoring tools such as Ambari, Cloudera Manager, or Hortonworks.
  • Knowledge of Hadoop ecosystem components, such as HDFS, YARN, Hive, Spark SQL, and MapReduce.
  • Familiarity with Linux/Unix environments and shell scripting.
  • Understanding of networking, storage, and security concepts related to Spark and Hadoop clusters.
  • Strong problem-solving and troubleshooting skills.
  • Excellent communication and collaboration abilities.

Job ID: 2C230099

  • Posted By

    Sheri F Weaver


Sr. Recruiter


2T Consulting

Last Login

Sep 24, 2023

Posted On

May 26, 2023

Related Jobs