We are seeking an experienced Scala Developer with strong expertise in big data technologies, particularly Hadoop and Spark, to join our dynamic team. The ideal candidate will have a solid background in developing, optimizing, and managing large-scale data processing solutions. The role requires hands-on experience with Hadoop ecosystem components, Scala programming, and Spark performance tuning, alongside strong scripting skills.

Key Responsibilities

• Design, develop, and optimize applications using Scala for high-performance data processing.
• Work extensively with Hadoop, HDFS, and Spark to handle large-scale data processing tasks.
• Develop and optimize Spark applications, ensuring proper configurations and performance tuning for optimal efficiency.
• Utilize Hadoop components such as OOzie, Sqoop, and other utilities to integrate, schedule, and process bulk data.
• Work with various data sources and perform ETL tasks using Hadoop ecosystem tools, ensuring efficient data transfer and transformation.
• Apply best practices for Spark configuration, performance optimization, and memory management.
• Write and maintain shell scripts to automate and streamline data processing workflows.
• Collaborate with cross-functional teams to design and implement scalable, efficient big data solutions.

Required Qualifications

• 8 to 10 years of overall IT experience, with a focus on big data and distributed systems.
• At least 3 years of hands-on experience in Scala programming.
• Minimum of 3 years working experience with Hadoop (HDFS, MapReduce) and Spark (including configuration and performance tuning).
• Extensive experience with bulk data processing and knowledge of Hadoop utilities (OOzie, Sqoop).
• Strong hands-on experience with shell scripting for data processing automation.
• Solid understanding of big data processing frameworks, including the Hadoop ecosystem, Spark, and related tools.

Posted By