Job Summary
We are seeking an experienced Teradata Data Engineer with strong expertise in advanced SQL, Teradata development, PySpark, and big data technologies. The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines and data integration solutions to support enterprise analytics and reporting requirements.
The role involves working closely with business stakeholders, product owners, and technology teams to deliver high-quality data engineering solutions while ensuring data accuracy, performance, scalability, and adherence to enterprise data management standards.
Key Responsibilities
- Design, develop, and maintain data integration, transformation, and processing solutions for enterprise data platforms and analytics systems.
- Develop and optimize complex SQL queries, Teradata scripts, and data processing workflows to support business reporting and analytical requirements.
- Build and maintain scalable data pipelines using Teradata, PySpark, Hive, and Impala.
- Work closely with business stakeholders, product owners, and data engineering teams to understand requirements and translate them into technical solutions.
- Design and implement data transformation processes, data structures, metadata management, data quality controls, and workload dependencies.
- Develop and optimize large-scale data processing solutions that meet functional and non-functional requirements.
- Identify and resolve data quality issues, performance bottlenecks, and technical challenges across data platforms.
- Participate in requirement analysis, story refinement, technical design, and Agile development activities.
- Develop and maintain automated test cases, including integration, regression, and performance testing.
- Analyze test results, troubleshoot defects, and implement appropriate corrective actions.
- Follow established CI/CD practices, version control standards, and automated deployment processes.
- Collaborate with cross-functional teams to ensure successful code deployment, production support, and release management.
- Ensure compliance with enterprise data management, data architecture, and technology standards.
- Identify gaps in data management practices and recommend improvements.
- Provide technical guidance and mentorship to junior data engineers and promote engineering best practices.
Required Qualifications & Experience
- 7+ years of experience in advanced SQL development and relational database technologies.
- 5+ years of hands-on experience with Teradata and related utilities, including BTEQ and TPT.
- 4+ years of experience working with PySpark and big data technologies such as Apache Hive and Impala.
- 3+ years of experience in the financial services industry, preferably in data development and reporting.
- Strong experience in ETL development, data warehousing, data integration, and data pipeline optimization.
- Hands-on experience with job scheduling tools such as Autosys.
- Experience with Shell scripting and automation.
- Familiarity with GitHub, Bitbucket, and Ansible.
- Working knowledge of Agile methodologies and Scrum-based software delivery.
- Experience with CI/CD practices and automated deployment processes.
- Strong analytical, problem-solving, and troubleshooting skills.
- Excellent communication and collaboration skills, with the ability to work effectively with business and technical stakeholders.
- Strong organizational skills, attention to detail, and ability to manage multiple priorities and timelines.
- Ability to work independently while contributing effectively as part of a collaborative team.
Preferred Qualifications
- Exposure to Oracle, Tableau, and .NET technologies.
- Experience with JIRA and enterprise data management tools.
- Knowledge of enterprise data architecture, data governance, and data management standards.
- Experience implementing automated testing and deployment solutions.
- Previous experience mentoring junior developers and providing technical leadership.