Sr Data Engineer | Full Time

PB Consulting

New York, NY

Posted On: Jun 04, 2024

Job Overview

Job Type

Full-time

Experience

8 - 10 Years

Salary

Depends on Experience

Work Arrangement

On-Site

Travel Requirements

0%

Required Skills

  • Scala
  • Python
  • Apache Spark
  • PySpark
  • AWS
  • SQL
Job Description
Roles and Responsibilities
  • Develop products and services on the latest technologies through contributions in development, enhancements, testing, and implementation. 
  • Work on migrating applications from an on-premises location to the cloud service providers. 
  • Develop, modify, and extend code for building cloud infrastructure, and automate using CI/CD pipeline. 
  • Partners with business and peers in the pursuit of solutions that achieve business goals through an agile software development methodology.
  • Work with peers across the system to define and implement best practices and standards. 
  • Assess applications and help determine the appropriate application infrastructure patterns. 

 

Required Experience/Skills
  • Hands-on experience in building/implementing cloud platforms/applications on the AWS platform. 
  • Experience in developing data pipeline solutions to ingest and exploit new and existing data sources. 
  • Expertise in leveraging SQL, programming languages like Python, and ETL tools like Databricks.
  • Expertise in AWS Compute (EC2, EMR), AWS Storage (S3, EBS), AWS Databases (RDS, DynamoDB), AWS Data Integration (Glue). 
  • Good understanding of AWS Identify and Access management, AWS Networking, and AWS Monitoring tools. 
  • Proficiency in CI/CD and deployment automation using the GITLAB pipeline. 
  • Proficiency in one or more programming languages e.g., Python, Scala. 

Job ID: PC240203


Posted By

Naincy Chauhan

Sr. Manager