Roles and Responsibilities

• Lead the design and development of scalable and efficient data pipelines, ETL/ELT workflows, and data models within Azure cloud environment. Collaborate with stakeholders to define data requirements and design solutions that align with business objectives.
• Design and implement dimensional data models (star/snowflake schema) using advanced SQL.
• Develop robust, high-performance ETL/ELT pipelines in Azure Data Factory (ADF), ensuring reliable data movement and transformation between systems.
• Utilize Databricks, Apache Spark, and PySpark to process large datasets and implement big data solutions. Apply Spark SQL and other big data tools for distributed processing.
• Build and manage data solutions on Azure, using technologies such as Azure Synapse Analytics, ADLS (Azure Data Lake Storage), and Databricks.
• Ensure that data pipelines and models are optimized for performance and scalability. Monitor and troubleshoot performance bottlenecks across systems.
• Conduct hands-on coding assessments and tests for data processing and transformation jobs.
• Review and ensure code quality, and promote best practices for data engineering.
• Support cloud migration initiatives, moving on-premise data and applications to the Azure cloud platform.
• Follow migration methodologies and ensure a seamless transition to cloud-based data platforms.
• Provide clear and concise documentation for designs, architectures, and processes.
• Create and deliver status reports, recommendations, and insights to stakeholders.

Required Skills & Experience

• Expertise with Azure technologies like Azure Synapse, Azure Data Factory, ADLS, and Databricks is a must.
• 10+ years of experience in SQL development, schema design, and dimensional data modeling (star, snowflake). Expertise in writing complex SQL queries and optimizing performance for large datasets.
• Hands-on experience with ETL/ELT tools, such as Azure Data Factory (ADF), Informatica, and Talend.
• Familiarity with data warehousing platforms like Azure Synapse, Azure SQL, Snowflake, and Amazon Redshift.
• Strong experience with big data tools such as Databricks, Apache Spark, PySpark, and Spark SQL.
• Ability to design and implement distributed data processing jobs using Spark frameworks.

Preferred Skills

• Knowledge of Event Hub, IoT Hub, Azure Stream Analytics, Azure Analysis Services, and Cosmos DB.
• Experience with SAP ECC / S/4HANA and integration with data platforms.
• Intermediate knowledge of Power BI for visualizing and reporting on data.
• Familiarity with Azure DevOps, CI/CD pipelines, and cloud deployment methodologies.

Qualifications

• Bachelor’s or Master’s degree in Computer Science, Engineering, Information Systems, or a related field.
• 10+ years of hands-on experience in data engineering, data warehousing, and cloud technologies.
• Strong technical background with expertise in data engineering best practices, cloud-based architecture, and modern data processing techniques.

Posted By