We are seeking a highly skilled and motivated Senior Data Engineer to join our growing data engineering team. As a Senior Data Engineer, you will play a pivotal role in designing, building, and managing complex data solutions in the cloud, leveraging Microsoft Azure technologies. You will be responsible for developing and maintaining scalable data architectures, automating workflows, and ensuring high-quality data pipelines to support data-driven decision-making across the organization.

Roles and Responsibilities

• Develop, implement, and optimize data pipelines using Azure services such as Azure Data Lake, Azure Data Factory, Azure SQL Data Warehouse, Azure Synapse, and Cosmos DB.
• Provision and configure Azure cloud resources, and create automation scripts to streamline workflows and deployments using cloud orchestration tools (e.g., Azure DevOps, CI/CD pipelines).
• Work on the design and implementation of distributed data processing solutions using both batch and streaming data pipelines, ensuring they scale efficiently.
• Work closely with Data Scientists, Data Analysts, and other engineers to gather requirements, ensure the right data architecture, and implement data-driven solutions.
• Develop data solutions and automate tasks using Python, JavaScript, or PySpark, and implement data transformations and integrations.
• Experience with provisioning, configuring, and managing PaaS, IaaS, and SaaS solutions within Azure to optimize data workflows.
• Design and implement Continuous Integration and Continuous Deployment (CI/CD) pipelines to ensure efficient and reliable data solution delivery.
• Monitor and optimize data processing pipelines to ensure they meet performance and scalability requirements.
• Address issues related to data integrity, performance bottlenecks, and system failures, ensuring uptime and reliability of data pipelines.

Required Skills & Qualifications

• Minimum of 5 years of relevant experience in Data Engineering, with at least 3 years of hands-on experience working in cloud environments 
• Proven experience working with Azure services, including but not limited to Azure Data Lake, Azure Data Factory, Azure SQL Data Warehouse, Azure Synapse, and Cosmos DB.
• At least 5 years of experience with SQL, including data modeling, querying, and optimization in large-scale environments.
• Minimum of 3 years of experience with cloud orchestration and automation tools, including designing CI/CD pipelines and automating data workflows.
• Strong understanding of big data architecture and distributed data processing, including experience with batch and streaming pipelines.
• Proficient in Python, JavaScript, or PySpark for data processing, scripting, and automation tasks.
• Hands-on experience with PaaS/IaaS/SaaS products and solutions within Azure ecosystem.
• Strong analytical and problem-solving skills with the ability to troubleshoot and resolve complex data pipeline issues.

Posted By