Roles and Responsibilities

• Manage and optimize the Databricks environment, ensuring high availability and performance.
• Monitor and troubleshoot issues related to Databricks clusters, jobs, and workflows.
• Support the deployment and management of machine learning models in production environments.
• Collaborate with data scientists and engineers to streamline MLOps processes, ensuring efficient model training, validation, and monitoring.
• Design, develop, and maintain ETL processes and data pipelines using Databricks and related technologies.
• Implement data transformation and integration strategies to support analytical workloads.
• Identify and implement optimizations to improve performance and reduce costs within the Databricks environment.

Required Qualifications

• Proven experience as a Databricks Engineer or Administrator, with a strong understanding of MLOps.
• Proficiency in Spark and Databricks functionalities, including data processing and analytics.
• Familiarity with ETL processes and tools, data warehousing concepts, and data pipeline development.
• Strong knowledge of cloud platforms (e.g., AWS, Azure) and their integration with Databricks.
• Familiarity with containerization and orchestration technologies (e.g., Docker, Kubernetes).
• Excellent troubleshooting and problem-solving skills.

Posted By