Azure Databricks Architect

Summary:

We are seeking a Azure Databricks Architect to define and lead the architecture of scalable, secure, and high-performance data platforms on Microsoft Azure. This role is hands-on and requires both architectural leadership and direct contribution to design, development, optimization, and delivery. The ideal candidate will shape platform strategy while actively building proof-of-concepts, reference implementations, and production-grade solutions.

Must-have qualifications

·       12+ years of experience in data engineering, data architecture, or platform engineering.

·       Strong hands-on experience with Azure Databricks, Spark, Python/Scala, SQL, and Delta Lake.

·       Proven experience designing and implementing cloud data hubs, data lakes, and analytical data stores.

·       Experience with Azure DevOps, CI/CD, and automated deployment practices.

·       Ability to work directly on technical implementation, not only provide oversight.

·       Strong communication and stakeholder collaboration skills.

Preferred qualifications

·       Experience in financial services or insurance.

·       Knowledge of Event Hub, Cosmos DB, Azure Synapse, or other Azure analytics services.

·       Databricks or Microsoft Azure certifications.

·       Experience with reusable frameworks, shell scripting, and production support in large-scale environments.

Key responsibilities

·       Own the end-to-end architecture for Azure Databricks-based data platforms, including ingestion, transformation, storage, serving, and governance layers.

·       Design reusable frameworks, ELT patterns, and data models that improve delivery speed, consistency, and maintainability.

·       Build and optimize pipelines and Spark jobs directly, with a strong focus on performance, reliability, and cost efficiency.

·       Architect solutions using Azure Databricks, Delta Lake, ADLS, Azure Data Factory, and related Azure platform services.

·       Establish governance, security and code-quality standards for the platform.

·       Partner closely with business analysts, functional teams, and engineering teams to translate requirements into working solutions.

·       Guide troubleshooting, code reviews, and production support for complex data workloads.

·       Drive modernization and cloud-native adoption across data and analytics capabilities.

Posted By