As a Lead Data Engineer, you will oversee the design, development, and management of data infrastructure and pipelines. You will work closely with data scientists, analysts, and software engineers to ensure the efficient collection, transformation, and storage of data. You will guide a team of data engineers, ensuring best practices in data architecture, quality, and scalability.
Key Responsibilities:
- Lead the development and maintenance of scalable data pipelines and ETL processes.
- Design and optimize data architecture for high performance and reliability.
- Collaborate with cross-functional teams to define data requirements and deliver solutions.
- Mentor and guide junior data engineers.
- Ensure data quality, integrity, and security standards are met.
- Evaluate and implement new technologies to improve data processes.
Required Skills:
- Strong experience with SQL, Python, and big data technologies (e.g., Hadoop, Spark).
- Expertise in data warehousing and cloud platforms (e.g., AWS, GCP, Azure).
- Proven leadership experience in managing data engineering teams.
- Knowledge of data modeling, ETL processes, and real-time data streaming.
- Strong problem-solving and communication skill.