We are seeking an experienced Technology Lead / Azure Data Engineer with strong hands-on expertise in Azure Synapse Analytics and PySpark. The ideal candidate will have a deep understanding of Apache Spark, distributed data processing, Data Lake architecture, and modern data engineering practices.
Must-Have Skills
- Azure Synapse Analytics
- PySpark
- Apache Spark architecture
- Spark SQL
- Data Lake architecture
- Medallion Architecture
- Data transformation and processing
- Strong troubleshooting and problem-solving skills
Nice-to-Have
- Microsoft Fabric
- Experience with modern cloud data platforms
Responsibilities
- Design, develop, and optimize data solutions using Azure Synapse Analytics.
- Develop scalable data pipelines and transformations using PySpark and Spark SQL.
- Apply Apache Spark architecture and distributed processing concepts to large-scale data workloads.
- Design and implement Data Lake and Medallion Architecture solutions.
- Troubleshoot performance, data quality, and pipeline-related issues.
- Collaborate with cross-functional teams to understand requirements and deliver scalable solutions.
- Evaluate and leverage Microsoft Fabric capabilities where appropriate.
- Provide technical leadership and contribute to solution architecture and development standards.
Required Qualifications
- 8+ years of experience in data engineering, cloud platforms, or related technology roles.
- Strong hands-on experience with Azure Synapse Analytics.
- Strong programming experience with PySpark and Spark SQL.
- Deep understanding of Apache Spark and distributed data processing.
- Experience designing and implementing Data Lake architectures.
- Excellent communication and collaboration skills.
- Strong analytical, troubleshooting, and problem-solving abilities.