Job Description

Must Have Technical/Functional Skills

•  Bachelors or Masters degree in Data Science, Computer Science, MIS ,related field, or equivalanet experience
•  10+ yrs of experience designing and developing AI solution architecture to scale
•  Proven Hands-on experience in GenAI Ops-Operationalizing LLM and RAG applications in production
•  Strong hands-on experience with the Langchain framework
•  Experience specifically with the Open AI API, chat completions, embeddings,etc
•  Have a solid awareness on TensorRT and VLLM implementation.
•  Strong proficiency in python and data Science libraries(NumPy,Pandas,scikit-learn,PyTorch/TensorFlow)
•  Proven experience applying guardrails and observability to LLM or RAG-powered applications.
•  Experience with LLMs as Judges and SMLs for evaluation (attribution,adherence, bias, PII, etc)
•  Hands-on experience with OpenShift(or Kubernetes) for containerized AL workloads.
•  Experience measuring and optimizing inference latency.

Roles & Responsibilities

•  Implement Guardrails and observability across RAG and LLM applications
•  Setup GenAI Ops workflows to continuously monitor inference latency, throughput, quality and safety metrics.
•  Define,track,and analyze RAG guardrail metrics using LLMs as Judges and SMLs(e.g. attribution, grounding,prompt injection,tone,PII leekage)
•  Implement annotation,structured feedback loops,fine-tuning, and alignment methods to calibrate judge models
•  Use langchain to orchestrate guardrail checks, manage prompt versioning and integrate judge model scoring workflows.
•  Work with Openshift to deploy,scale and monitor containerized genAI services.
•  Build observability dashboards and alerts(Grafana or equivalent) for AI reliability.
•  Contribute to emerging evaluation and guardrails as autonomous AI workflows expand.

Posted By