Duties and Responsibilities

• Assembling large, complex sets of data that meet non-functional and functional business requirements.
• Identifying, designing, and implementing internal process improvements including re-designing infrastructure for greater scalability, optimizing data delivery, and automating manual processes.
• Building required infrastructure for optimal extraction, transformation, and loading of data from various data sources using AWS and SQL technologies.
• Building analytical tools to utilize the data pipeline, providing actionable insight into key business performance metrics including operational efficiency and customer acquisition.
• Working with stakeholders including data, design, product, and executive teams, and assisting them with data-related technical issues.

Requirements and skills

• 8+ plus years of experience in solution data pipeline for large enterprise data Warehouse applications using Azure Data Factory, Data bricks, Data Ingestion, Data Transformation /Computation, Orchestration, Reporting & Data Analytics.
• Knowledge of Azure Services such as ADF, Databricks, Event Hub, Delta Lake, and Data Lake.
• Strong command of Python, Pyspark, SQL & Power BI.
• Experience in GitHub, Git Action, and Jira end-to-end data pipeline.
• Experience in Terraform for Infrastructure Provisioning.
• Strong knowledge of the CICD Pipeline for automatic deployments.
• Proficient in technical artifacts e.g., Application Architecture, Solution Design Documents, etc.

Posted By