We are seeking an experienced Data Engineer Lead to design, develop, and manage scalable data solutions within a cloud-based data platform. The ideal candidate will have strong expertise in ETL development, Azure data services, SQL programming, and large-scale data management. This role requires both hands-on technical expertise and leadership capabilities to guide data engineering initiatives and deliver reliable data solutions for business stakeholders.
Key Responsibilities
- Lead the design, development, and implementation of data engineering solutions and ETL pipelines.
- Develop and manage data integration processes using Azure technologies such as Azure Data Factory, Synapse, and Databricks.
- Establish and maintain cloud connectivity between systems including ADLS, ADF, Synapse, and Databricks.
- Write and optimize T-SQL queries, stored procedures, and scripts for large-scale data processing.
- Develop and maintain Python scripts to support data engineering workflows.
- Design, build, and maintain data marts and large-scale relational databases.
- Manage and troubleshoot large database environments, ensuring system stability and performance.
- Implement database security, user access control, and data governance practices.
- Design and manage database backup, archiving, storage, and capacity planning strategies.
- Perform performance tuning and optimization for large data environments.
- Support data extraction, transformation, and loading (ETL) from multiple data sources.
- Collaborate with business teams to deliver data extraction, reporting, and analytics solutions.
- Lead technical troubleshooting, testing activities, and issue resolution for data platforms.
- Mentor team members and provide technical leadership to data engineering teams.
Required Qualifications
- 10+ years of experience working with large SQL-based data marts and relational databases.
- 3+ years of experience in a Technical Lead or Team Lead role.
- Strong experience with ETL frameworks such as Azure Data Factory, Synapse, or Databricks.
- Advanced expertise in T-SQL programming.
- Experience with Python scripting for data processing and automation.
- Extensive experience with database troubleshooting, performance tuning, and support.
- Ability to work with large volumes of data and deliver efficient data solutions.
- Strong analytical and problem-solving skills.
- Excellent verbal and written communication skills.
Preferred Qualifications
- Experience working with financial institution data marts.
- Knowledge or experience with .NET applications.
- Experience designing Tableau dashboards and data visualizations.