ETL PySpark Lead Developer
PB Consulting
Philadelphia, PA
Posted On: Mar 22, 2024
Posted On: Mar 22, 2024
Job Overview
Salary
Depends on Experience
Required Skills
- PySpark
- ETL
- Scala
- Python
- Big Data
- Kafka
- AWS
- Jenkins
- Databricks
Job Description
- Speak to business and intake team to define the requirements.
- Coordinate with Solution and Business Analysts and create the tech designs
- Hands-on architecture/development of ETL pipelines using our internal framework written in Java
- Interpret data, analyze results using statistical techniques and provide ongoing reports
- Develop and implement databases, data collection systems, data analytics and other strategies that optimize statistical efficiency and quality
- Acquire data from primary or secondary data sources and maintain databases/data systems
- Identify, analyze, and interpret trends or patterns in complex data sets
- Filter and clean data by reviewing reports and performance indicators to locate and correct problems
- Work with management to prioritize business and information needs.
Preferred Skills
- Scala Spark
- Python
- Big Data
- Kafka
- AWS
- Oracle SQL
- Hive SQL
- Databricks
- Jenkins
Qualifications
- Bachelor's in computer science, IT or related fields.
- Experience Required 10 - 12 Years.
- Generic Managerial Skills Motivated team player with excellent communication.
- Interpersonal, analytical and problem solving skills.
Job ID: PC240088