Responsibilities

• Create and implement a framework for ML and LLM operations, ensuring efficient integration and operationalization.
• Develop and maintain processes related to AI compliance and governance, ensuring adherence to regulations and best practices.
• Continuously monitor the Data Lake and engage appropriate support teams to address issues promptly and effectively.
• Generate reports to track usage data for billing purposes and service level agreement (SLA) compliance.
• Collaborate with business and cross-functional teams to document requirements and align data solutions with business needs.
• Provide support for ETL/ELT job availability and performance, troubleshooting issues as necessary.
• Offer technical guidance and cross-training to both business and internal team members, enhancing overall team capabilities.
• Work with business partners to identify and implement opportunities for continuous improvement in data processes.

Qualifications

• At least 6 years of experience in Data Engineering, focusing on Data Warehousing and Data Analytics.
• Solid experience with Google Cloud Platform (GCP), particularly with Vertex AI, Dataflow, DataProc, Cloud Composer, BigQuery, Cloud Storage, GKE, and Airflow.
• Strong hands-on expertise in Python, with experience in Scala and working knowledge of notebooks.
• Extensive experience in designing and building scalable data pipelines for extraction, transformation, and loading (ETL/ELT).
• At least 5 years of experience in data governance and metadata management, ensuring data quality and compliance.
• Ability to work independently, solve complex problems, and provide updates to stakeholders effectively.
• Strong understanding of relational and dimensional data modeling concepts.
• Familiarity with DevOps and CI/CD practices to support efficient deployment and operation of data solutions.
• Excellent written and verbal communication skills, with experience in technical documentation and the ability to communicate effectively with senior business managers and executives.

Posted By