Data Engineer With ML, Java, Spark, Scala

2T Consulting

Sunnyvale, CA

Posted On: Mar 18, 2024

Posted On: Mar 18, 2024

Job Overview

Job Type

Full-time

Experience

10 - 12 Years

Salary

$130,000 - $140,000 Per Year

Work Arrangement

On-Site

Travel Requirement

0%

Required Skills

  • Data engineering
  • Big Data
  • Java
  • Scala
  • Spark
  • PySpark
  • Golang
  • Hadoop
  • Cloud-native
  • LLM
  • ETL
Job Description

Job Description

  • Build data pipelines to enable training of LLMs
  • Work with technical and non-technical stakeholders to collect requirements, integrate with data systems and deliver large-scale datasets 
  • Operate in a complex multi-cloud data lakehouse, building end-to-end integrated workflows
  • Work with privacy, security, and other policy stakeholders to design, review and implement solutions to automate compliant ETL for downstream teams

 

Qualifications

  • Strong understanding of machine learning principles, especially in the context of LLMs. 
  • 5+ years of experience in big data engineering
  • Proficiency in Java, Scala, Spark
  • Proficiency in PySpark and Golang (preferred)
  • Experience building and leveraging large scale data infrastructure, map-reduce/hadoop, cloud-native deployments
  • Experience with alerting, monitoring and remediation automation in a large scale distributed environment 
  • BS/BA or equivalent degree in computer science or similar (preferred)

Job ID: 2C240082


Posted By

Shayne

Sr. Recruiter


Related Jobs
  • Contract - W2
  • Contract - Independent
  • Contract - Corp-to-Corp

  • Company
  • COMPANY

    Techvilla Solutions

  • Company
  • experience

    2 - 5 Years

  • Travel Requirements
  • Work Arrangement

    Hybrid

  • Wallet
  • SALARY

    $60 - $65 Per Hour

  • Skills
  • SKILLS

    • Automation
    • Selenium
    • Eclipse
    • JavaScript
    • +1 more

Posted On: Nov 01, 2024

  • Contract - W2
  • Contract to Hire - W2

  • Company
  • COMPANY

    2T Consulting

  • Company
  • experience

    8 - 15 Years

  • Travel Requirements
  • Work Arrangement

    On-Site

  • Wallet
  • SALARY

    $58 - $64 Per Hour

  • Skills
  • SKILLS

    • Azure
    • RESTful APIs
    • SQL
    • Kafka
    • +1 more

Posted On: Nov 01, 2024

  • Contract - W2
  • Contract - Independent
  • Contract - Corp-to-Corp

  • Company
  • COMPANY

    2T Consulting

  • Company
  • experience

    6 - 15 Years

  • Travel Requirements
  • Work Arrangement

    Hybrid

  • Wallet
  • SALARY

    Depends on Experience

  • Skills
  • SKILLS

    • Python
    • SQL
    • GCP
    • Pyspark
    • +2 more

Posted On: Oct 29, 2024

  • Contract - Corp-to-Corp
  • Contract - Independent
  • Contract - W2

  • Company
  • COMPANY

    Neshent Tech

  • Company
  • experience

    10 - 25 Years

  • Travel Requirements
  • Work Arrangement

    On-Site

  • Wallet
  • SALARY

    $75 - $80 Per Hour

  • Skills
  • SKILLS

    • ServiceNow
    • ITSM
    • ITOM
    • JavaScript
    • +1 more

Posted On: Oct 28, 2024

  • Contract - W2

  • Company
  • COMPANY

    Long Finch Technologies

  • Company
  • experience

    7 - 12 Years

  • Travel Requirements
  • Work Arrangement

    On-Site

  • Wallet
  • SALARY

    $50 - $60 Per Hour

  • Skills
  • SKILLS

    • Azure
    • Docker
    • DevOps
    • Kubernetes
    • +3 more

Posted On: Oct 28, 2024