Roles and Responsibilities

• Lead the end-to-end migration of data systems from HDFS to Cloud, ensuring scalability, performance, and reliability.
• Architect and implement cloud-native solutions for both batch and real-time streaming data workloads.
• Develop and maintain high-performance data pipelines using Apache Spark, Apache Flink, Scala, and SQL.
• Collaborate with Cloud infrastructure teams to align architecture with enterprise standards.
• Create reusable components and frameworks to accelerate cloud adoption across data teams.
• Define and enforce data governance, security, and compliance standards (e.g., GDPR, CCPA).
• Troubleshoot and resolve performance bottlenecks in distributed data processing environments.
• Partner with cross-functional teams including data engineering, DevOps, analytics, and product stakeholders.

Required Skills & Qualifications

• Bachelor’s or Master’s degree in Computer Science, Information Systems, or a related field.
• 7+ years of experience in data engineering or big data architecture roles.
• Proven experience migrating data systems from HDFS to cloud platforms (Apple Cloud preferred).
• Strong expertise in:
  • Apache Spark (batch & structured streaming)
  • Apache Flink (real-time stream processing)
  • Scala programming
  • Advanced SQL for large-scale data transformation.
  • Deep understanding of distributed storage systems, cloud object storage, and metadata management.
• Knowledge of data security, encryption, and compliance frameworks (e.g., GDPR, CCPA).
• Experience with CI/CD, automation, and infrastructure-as-code tools (e.g., Terraform, Jenkins).

Posted By