Senior Data Engineer

Neshent Technologies

Atlanta, GA

Posted On: Nov 02, 2026

Job Overview

Job Type

Full-time

Experience

8 - 12 Years

Salary

$110,000 - $130,000 Per Year

Work Arrangement

On-Site

Travel Requirement

Required Skills

Databricks
Delta Lake
Delta Live Tables ( DLT )
Unity Catalog
Python
Apache Spark
Apache Spark SQL
ETL Development
Data Pipeline Development
SQL
Data Modeling
Apache Kafka
Apache Spark Streaming
Hadoop
Hive
HDFS
AWS
Google Cloud
Data Warehousing
Data Ingestion
Data Transformation
Data Quality
Big Data Processing

Job Description

We are looking for an experienced Senior Data Engineer with strong expertise in Databricks, Python, PySpark, Apache Spark, and large-scale data engineering. The ideal candidate will be responsible for designing and developing scalable ETL pipelines, implementing data processing solutions, managing data ingestion frameworks, and optimizing cloud-based data platforms. The role also involves technical leadership, data governance, and collaboration with analytics and business intelligence teams.

Required Technical Skills

Strong hands-on experience with Databricks, Delta Lake, Delta Live Tables, and Unity Catalog.
Proficiency in Python, PySpark, Spark SQL, and Apache Spark.
Strong experience in ETL development, data pipeline design, and data transformation.
Knowledge of Hadoop, Kafka, Hive, HDFS, and Sqoop.
Expertise in SQL, data modeling, and data warehousing.
Experience with batch and real-time data processing using Kafka and Spark Streaming.
Cloud experience with AWS and/or GCP.
Strong understanding of data quality, validation, cleansing, and governance.
Experience working in Agile environments (Scrum, Kanban, SAFe).

Preferred Skills

Scala / Scala-Spark, IBM DataStage, TIBCO.
AWS S3, Glue, Lambda, Kinesis/MSK.
Tableau, Power BI, Spotfire, OAC.
Teradata, DB2, BigQuery, Mainframe.
Git, data governance, and retention modeling.

Roles & Responsibilities

Design, develop, and maintain scalable data pipelines and ETL solutions.
Build data ingestion and transformation frameworks for structured and unstructured data.
Develop and optimize batch and streaming data processing solutions.
Implement data quality, validation, and governance standards.
Integrate heterogeneous data sources into enterprise data platforms.
Support analytics, reporting, BI, and machine learning initiatives.
Collaborate with Data Scientists, BI teams, and business stakeholders.
Lead technical discussions, mentor junior engineers, and drive data engineering best practices.