Roles and Responsibilities

• Develop and maintain data pipelines to ingest, transform, and load data from various sources.
• Design and manage scalable data lakes to support comprehensive data storage and analysis needs.
• Design retention, lifecycle, and archival policies for datasets.
• Work with Azure Data Lake Storage to efficiently store and manage large datasets.
• Create and manage data models and data warehouses to support business intelligence and analytics needs.
• Set up and manage external tables and integrate with various data sources.
• Implement CI/CD pipelines using Azure DevOps to streamline development and deployment processes.
• Ensure data security and compliance with relevant regulations and best practices (eg: NIST).

Qualifications/Skills

• Experience with data integration and migration from on-premises to cloud environments.
• Proficiency in Python and experience with Python Notebooks.
• Proficiency in scripting and automation (PowerShell, ARM templates, Terraform).
• Proven ability to lead cloud data migration projects
• Hands-on experience in performance tuning and cost optimization
• Proven experience in setting up data governance using Purview    
• Experience in solutioning that meets NIST guidelines
• Azure certifications (e.g., Azure Solutions Architect, Azure Data Engineer) are a plus

Posted By