Roles and Responsibilities

• Design and implement scalable data pipelines using PySpark/Python.
• Develop and optimize SQL queries for data extraction, transformation, and loading (ETL) processes.
• Utilize Databricks on AWS to build and manage data workflows.
• Able to read Legacy code and understand it
• Excellent know how on preparing script for data validation
• Monitor and troubleshoot data pipelines to ensure smooth operation.
• Document data processes and workflows for future reference and maintenance.
• Work closely with existing team to understand the requirement and deliver accordingly

Technical Skills

• Strong proficiency in PySpark and Python programming.
• Extensive experience with SQL and database management.
• Hands-on experience with Databricks on AWS.
• Familiarity with AWS services such as S3, Lambda, Glue, and Redshift.
• Knowledge of data warehousing concepts and ETL processes.
• Excellent problem-solving skills and attention to detail.
• Strong communication and collaboration skills.

Posted By