The Data Engineer will develop and manage data pipelines, data warehouses, and data integration processes to support data analytics and business intelligence initiatives. This role requires strong technical skills, experience with various data technologies, and the ability to collaborate with data scientists, analysts, and other stakeholders to deliver high-quality data solutions.

Key Responsibilities

• Design, build, and maintain scalable data pipelines that extract, transform, and load (ETL) data from various sources into data warehouses or data lakes.
• Integrate data from multiple sources, including databases, APIs, and external services. Ensure data consistency, quality, and accuracy across systems.
• Develop and maintain data storage solutions, including relational databases, NoSQL databases, and data lakes.
• Monitor and optimize the performance of data pipelines, databases, and queries. Address issues related to data processing, storage, and retrieval.
• Design and implement data models and schemas that support efficient data storage and retrieval. Ensure data models meet the requirements of business intelligence and analytics.
• Implement data quality checks and validation processes to ensure data integrity and accuracy. Troubleshoot and resolve data issues as they arise.
• Create and maintain documentation for data pipelines, processes, and data models. 
• Implement data security measures and ensure compliance with data protection regulations and organizational policies.

Qualifications

• Bachelor’s degree in Computer Science, Data Engineering, Information Technology, or a related field. 
• Minimum of [7] years of experience in data engineering or a related role, with a proven track record of developing and managing data pipelines and databases.
• Proficiency in programming languages such as Python, Java, or Scala.
• Experience with ETL tools and frameworks.
• Knowledge of big data technologies.
• Familiarity with cloud data platforms (e.g., AWS Redshift, Google BigQuery, Azure Synapse).
• Experience with database management and data warehousing solutions.

Posted By