Data Engineer With ML, Java, Spark, Scala

  • Salary

    $130,000 - $140,000 Per Year

Experience

10 to 12 Years

Posted On

Mar 18, 2024

Company

2T Consulting

Work Arrangement

On-Site

Job Overview

Required Skills

  • Data engineering
  • Big Data
  • Java
  • Scala
  • Spark
  • PySpark
  • Golang
  • Hadoop
  • Cloud-native
  • LLM
  • ETL

Job Type

Full-time

Location

Sunnyvale, CA

Travel Requirements

0%

Job Description

Job Description

  • Build data pipelines to enable training of LLMs
  • Work with technical and non-technical stakeholders to collect requirements, integrate with data systems and deliver large-scale datasets 
  • Operate in a complex multi-cloud data lakehouse, building end-to-end integrated workflows
  • Work with privacy, security, and other policy stakeholders to design, review and implement solutions to automate compliant ETL for downstream teams

 

Qualifications

  • Strong understanding of machine learning principles, especially in the context of LLMs. 
  • 5+ years of experience in big data engineering
  • Proficiency in Java, Scala, Spark
  • Proficiency in PySpark and Golang (preferred)
  • Experience building and leveraging large scale data infrastructure, map-reduce/hadoop, cloud-native deployments
  • Experience with alerting, monitoring and remediation automation in a large scale distributed environment 
  • BS/BA or equivalent degree in computer science or similar (preferred)

Job ID: 2C240082

  • Posted By

    Sheri F Weaver

Designation

Sr. Recruiter

Company

2T Consulting

Last Login

Jun 14, 2024

Posted On

Mar 18, 2024


Related Jobs
  • Full-time

  • Company
  • COMPANY

    TechVilla Solutions

  • Company
  • experience

    3 to 5 Years

  • Travel Requirements
  • Work Arrangement

    On-Site

  • Wallet
  • SALARY

    $90,000 - $130,000 Per Year

  • Skills
  • SKILLS

    • Contract - Corp-to-Corp
    • Contract - Independent
    • Contract - W2

    • Company
    • COMPANY

      TechVilla Solutions

    • Company
    • experience

      8 to 15 Years

    • Travel Requirements
    • Work Arrangement

      Remote

    • Wallet
    • SALARY

      $70 - $80 Per Hour

    • Skills
    • SKILLS

      • Contract - Corp-to-Corp
      • Contract - Independent
      • Contract - W2

      • Company
      • COMPANY

        PB Consulting

      • Company
      • experience

        12 to 17 Years

      • Travel Requirements
      • Work Arrangement

        Hybrid

      • Wallet
      • SALARY

        $75 - $82 Per Hour

      • Skills
      • SKILLS

        • Contract - Corp-to-Corp
        • Contract - W2
        • Contract - Independent

        • Company
        • COMPANY

          PB Consulting

        • Company
        • experience

          10 to 15 Years

        • Travel Requirements
        • Work Arrangement

          Hybrid

        • Wallet
        • SALARY

          $70 - $78 Per Hour

        • Skills
        • SKILLS

          • Contract - Corp-to-Corp
          • Contract - Independent
          • Contract - W2

          • Company
          • COMPANY

            PB Consulting

          • Company
          • experience

            8 to 12 Years

          • Travel Requirements
          • Work Arrangement

            Hybrid

          • Wallet
          • SALARY

            $65 - $70 Per Hour

          • Skills
          • SKILLS