Responsibilities

• Design and develop big data solutions using Hadoop, ensuring optimal performance and scalability of data pipelines.
• Utilize Scala for building data processing applications and frameworks within the big data ecosystem.
• Implement and manage data solutions using Databricks, optimizing data workflows and collaborating with data science teams.
• Work with large datasets to extract, transform, and load (ETL) data efficiently, ensuring high data quality and accessibility.
• Collaborate with cross-functional teams to gather requirements and translate them into technical specifications for data solutions.
• Define and enforce best practices for data engineering, including coding standards, testing strategies, and documentation.
• Monitor and optimize data processing jobs for performance and cost-efficiency, addressing bottlenecks and enhancing throughput.

Qualifications

• Proven experience as a Data Engineer or Big Data Developer, with a strong focus on Hadoop and Scala.
• Significant experience in designing and implementing big data solutions.
• Proficiency in Scala programming, with a solid understanding of functional programming concepts.
• Hands-on experience with Databricks and its ecosystem.
• Familiarity with Hadoop ecosystem components (e.g., HDFS, MapReduce, Hive, Spark).
• Strong understanding of data engineering principles, including ETL processes and data modeling.
• Excellent analytical and problem-solving skills, with the ability to troubleshoot and optimize complex data workflows.
• Strong communication skills to effectively collaborate with technical and non-technical stakeholders.

Posted By