Experience Required

• Strong knowledge of Data Management policies.
• Experience in building ETL / data warehouse transformation processes.
• Must have Hands-on experience with PySpark and Python development.
• Must have knowledge of Hadoop.
• Good understanding of Spark Architecture with Databricks.
• Must have knowledge of SQL.
• Strong problem Management, troubleshooting and analytical skills

 

Roles & Responsibilities

• Develop spark applications using Spark-SQL in databricks for data extraction, aggregation and transformation from multiple file formats for analysing and transforming the data.
• Ability to apply the spark DataFrame API.
• Responsible for estimating the cluster size,monitoring and troubleshooting of the spark databricks cluster.
• Take ownership of project and see it through completion.
• Strong analytical and problem-solving skill.
• Ability to work independently as well as part-of-team.