Responsibilities:

• Design, develop, and maintain large-scale data processing systems using Big Data technologies such as Hadoop, Spark, and Apache Kafka.
• Collaborate with data engineers and data scientists to build and optimize data pipelines for collecting, processing, and storing large volumes of structured and unstructured data.
• Implement data transformations, data cleansing, and ETL processes to ensure data quality and consistency.
• Optimize data storage and retrieval for performance and cost efficiency.
• Develop and maintain real-time data streaming and batch processing solutions to support analytical and reporting requirements.
• Troubleshoot and resolve data-related issues, ensuring data availability and reliability.
• Stay current with emerging Big Data technologies and industry best practices to continuously improve data processing capabilities.
• Provide technical guidance to junior developers and work with cross-functional teams to achieve data-related project goals.

Requirements:

• Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.
• Proven experience in Big Data development, with 5 to 12 years of relevant work experience.
• Proficiency in Big Data technologies, including Hadoop, Spark, Kafka, and associated ecosystems (e.g., HDFS, YARN, Hive, Pig).
• Strong programming skills in languages such as Java, Scala, Python, or similar.
• Experience with distributed data storage and processing platforms like HBase, Cassandra, or similar.
• Knowledge of database systems, both SQL and NoSQL.
• Familiarity with data warehousing, data lakes, and data integration concepts.
• Strong problem-solving and debugging skills.
• Excellent communication skills and the ability to work collaboratively with cross-functional teams.
• Experience with cloud-based Big Data solutions (e.g., AWS, Azure, GCP) is a plus.

Posted By