Responsibilities

• Design and develop data pipelines using Azure Data Factory, Data Lake, Synapse, and Databricks.
• Implement ETL processes with PySpark, T-SQL, and Scala for data ingestion and transformation.
• Manage and monitor Spark clusters and optimize performance.
• Integrate data from sources like Azure SQL, Blob Storage, Oracle, and SharePoint.
• Use Function Apps, Event Hubs, and Service Bus for event-driven architecture.
• Set up API integrations and secure access with Azure API Management.
• Build scalable data models and support data lakehouse architecture on Synapse.
• Troubleshoot issues and support operations teams.

Required Skills

• Azure Data Factory, Data Lake, Synapse, Stream Analytics, Event Hub
• PySpark, Scala, T-SQL
• Azure Databricks, Function Apps, API Management
• Service Bus, Event Triggers, Azure SQL, Blob Storage
• Strong understanding of big data pipelines and cloud architecture

Posted By