DFS Tower Lead

We are seeking an experienced DFS Tower Lead / L3 Production Support Engineer to provide technical leadership and advanced production support for enterprise data platforms and business-critical data pipelines. The role requires strong expertise in AWS, Databricks, Airflow, Dremio, Python, SQL, and data quality, with a focus on incident management, RCA, platform stability, upgrades, and continuous service improvement.

Required Technical / Functional Skills

*Strong experience with AWS, S3, Athena, and CloudWatch.

*Hands-on experience with Databricks, Dremio, and Apache Airflow.

*Strong programming and troubleshooting skills in Python and SQL.

*Experience with GitHub for source control and deployment support.

*Strong knowledge of metadata, data profiling, schema validation, and data quality checks.

*Experience with regression testing, runtime upgrades, patching, and environment maintenance.

*Strong expertise in production support, incident management, RCA, and permanent corrective actions.

*Good understanding of enterprise data platforms, ETL/ELT pipelines, and downstream data dependencies.

*Strong communication, stakeholder management, and technical leadership skills.

Roles & Responsibilities

*Lead L3 production support for enterprise data platforms and business-critical data pipelines.

*Monitor and support workloads across AWS, Databricks, Dremio, and Airflow environments.

*Troubleshoot complex production incidents and ensure resolution within defined SLAs.

*Perform detailed Root Cause Analysis (RCA) and implement permanent fixes.

*Analyze, debug, and optimize Airflow ETL/ELT workflows.

*Support Databricks clusters, jobs, notebooks, workflows, and runtime configurations.

*Use Python and SQL for troubleshooting, data validation, analysis, and operational activities.

*Monitor platform health and performance using AWS CloudWatch.

*Support data stored in S3 and queried through Athena.

*Perform data profiling, metadata validation, schema validation, and data quality checks.

*Support schema changes, column additions, and data model enhancements while ensuring downstream compatibility.

*Execute regression testing for deployments, platform changes, and infrastructure upgrades.

*Lead/support Databricks runtime upgrades, patching, and environment maintenance.

*Use GitHub for version control, release tracking, and deployment support.

*Collaborate with Data Engineering, Architecture, Business, and Infrastructure teams to resolve issues and improve platform stability.

*Participate in on-call support and critical incident management as required.

*Create and maintain operational runbooks, knowledge articles, and support documentation.

*Drive continuous improvement, automation, operational efficiency, and service stability across the data platform.

Posted By