We are looking for a skilled Data Warehouse Engineer to join our team. The ideal candidate will have extensive experience in data warehousing, ETL pipeline development, and machine learning model implementation using AWS services. You will play a key role in transforming large data sets into actionable insights and solutions for complex problems.
Responsibilities
- Build and maintain ETL pipelines to process large data sets using AWS services such as AWS Glue, EMR, Kinesis, Kafka, and CloudWatch.
- Design and develop machine learning models and algorithms to address complex business problems, leveraging AWS services effectively.
- Ensure data quality, integrity, and accessibility throughout the data pipeline, facilitating efficient data processing and analysis.
- Work with cross-functional teams to understand data requirements and provide actionable insights derived from data analysis.
- Document architecture, processes, and workflows to ensure knowledge sharing and compliance with best practices.
Required Skills/Experience
- 7+ years of hands-on experience in data warehousing, data engineering, and building machine learning models using AWS services.
- Proven ability to work independently with minimal guidance.
- Strong experience in building and maintaining ETL pipelines using AWS Glue, EMR, Kinesis, Kafka, and CloudWatch.
- Proficient in Python development, with strong knowledge of Spark or PySpark, and experience using APIs.
- Strong in writing SQL queries and performance tuning in AWS Redshift and other leading RDBMS like MS SQL Server and Postgres.
- Proficient in AWS services such as AWS Lambda, EventBridge, Step Functions, SNS, SQS, S3, and machine learning models.
- Familiarity with machine learning, deep learning, and natural language processing (NLP) techniques.
- Knowledge of how IAM roles and policies function within AWS environments.