Primary Responsibilities

• Lead the end-to-end migration of legacy data systems from HDFS to Cloud, ensuring minimal disruption and high availability.
• Architect and implement cloud-native data solutions optimized for both batch and real-time processing.
• Design, develop, and optimize data pipelines using Apache Spark, Apache Flink, Scala, and SQL.
• Collaborate closely with Cloud infrastructure teams to ensure alignment with enterprise cloud strategies and technical standards.
• Develop reusable frameworks and tools to streamline future data migrations and transformations.
• Define and enforce data governance, security, and compliance policies, including GDPR and CCPA.
• Monitor, debug, and optimize performance of cloud-based data applications and services.
• Partner with cross-functional teams including data engineers, DevOps, analysts, and product managers.

Required Qualifications

• Bachelor’s or Master’s degree in Computer Science, Information Systems, or a related technical field.
• 8+ years of experience in data engineering or big data architecture, including large-scale systems.
• Proven experience migrating data platforms from HDFS to cloud-based infrastructure; experience with Cloud is highly desirable.
• Strong expertise in: Apache Spark (including Structured Streaming), Apache Flink, Scala, Advanced SQL for data modeling and transformation.
• Solid understanding of distributed storage, cloud object storage, and metadata management.
• In-depth knowledge of data security, encryption, and compliance frameworks.
• Experience with CI/CD pipelines and Infrastructure-as-Code tools such as Terraform, Jenkins, or equivalents.

Posted By