Responsibilities

• Assist in the design, implementation, and maintenance of the Data Lake architecture on AWS, ensuring efficient storage and retrieval of large-scale datasets.
• Develop and maintain data processing scripts using Python and PySpark for ETL jobs, data cleansing, and transformation processes.
• Leverage AWS services (e.g., S3, EMR, Lambda) to manage and optimize data storage and processing.
• Use GIT for version control, managing codebases, and collaborating with development teams.
• Manage and optimize Snowflake data warehouse environments, ensuring seamless integration with the Data Lake.
• Work with Data 360 (or similar platforms) for data governance, integration, and quality control.

Required Skills

• Solid experience with AWS cloud services, particularly for data storage, processing, and analytics.
• Proficiency in Python for scripting and automation, and PySpark for distributed data processing.
• Knowledge and hands-on experience with Data Lake architecture and best practices.
• Working knowledge of GIT for version control and collaborative development.
• Experience with Snowflake data warehouse and its integration with cloud-based data solutions.

Posted By