On-Site
Full-Time
Gurugram, Haryana
India
Skills
Artificial Intelligence (AI)
MLOps
Large Language Models (LLM)
Prompt Engineering
About the Role
Job Title: Data Scientist and LLM Operations Specialist
Position Overview: We are seeking a highly skilled Data Scientist and LLMOps Resource to play a pivotal role in the development, expansion, operation, and maintenance of generative AI solutions. The primary responsibilities include running an experimental framework to determine the optimal prompt engineering approaches, tuning prompts, and collaborating with subject matter experts (SMEs) for evaluations and results. This role requires a deep understanding of evaluating models output in production, particularly when ground metrics are absent, monitoring for issues such as model drift and hallucinations, and optimizing for offline and online metrics.
Total Experience required: 4-6 years
Location: PAN India
Key Responsibilities:
Scope, develop, expand, operate, and maintain scalable, reliable and safe generative AI solutions.
Design and execute prompt engineering experiments to optimize Large Language Models (LLMs) for various use cases.
Collaborate with SMEs to evaluate prompt effectiveness and align AI solutions with business needs.
Understand and apply offline and online evaluation metrics for LLMs, ensuring continuous model improvements.
Evaluate production models using live data in the absence of ground metrics, implementing robust monitoring systems.
Monitor LLM applications for model drift, hallucinations, and performance degradation.
Ensure smooth integration of LLMs into existing workflows, providing real-time insights and predictive analytics.
Qualifications For AI/ML Engineer:
Solid experience in business analytics, data science, software development, and data engineering.
Expertise in Python and frameworks such as PyTorch, TensorFlow, or ONNX.
Hands-on experience working with LLMs and Generative AI, including model development and inference patterns.
Proven ability to design scalable systems leveraging LLMs, particularly in distributed computing environments.
Preferred Skills:
Experience in prompt engineering and prompt optimization.
Expertise in running experiments to evaluate generative AI performance.
Knowledge of production-level monitoring tools for ML models, including drift detection and mitigation strategies.
Excellent problem-solving skills and ability to work cross-functionally with data scientists, engineers, and SMEs.
Experience with safety, security and responsible use of AI.
Experience with red-teaming (adversarial testing) of generative AI.
Experience with developing AI applications with sensitive data such as PHI, PII and highly confidential data.
https://reqpedia.com/
Best reqpedia Tech Talent Recruitment
Welcome to Reqpedia, where opportunities meet talent! Find your dream job or discover the perfect candidate for your company Our platform connects employers.
Position Overview: We are seeking a highly skilled Data Scientist and LLMOps Resource to play a pivotal role in the development, expansion, operation, and maintenance of generative AI solutions. The primary responsibilities include running an experimental framework to determine the optimal prompt engineering approaches, tuning prompts, and collaborating with subject matter experts (SMEs) for evaluations and results. This role requires a deep understanding of evaluating models output in production, particularly when ground metrics are absent, monitoring for issues such as model drift and hallucinations, and optimizing for offline and online metrics.
Total Experience required: 4-6 years
Location: PAN India
Key Responsibilities:
Scope, develop, expand, operate, and maintain scalable, reliable and safe generative AI solutions.
Design and execute prompt engineering experiments to optimize Large Language Models (LLMs) for various use cases.
Collaborate with SMEs to evaluate prompt effectiveness and align AI solutions with business needs.
Understand and apply offline and online evaluation metrics for LLMs, ensuring continuous model improvements.
Evaluate production models using live data in the absence of ground metrics, implementing robust monitoring systems.
Monitor LLM applications for model drift, hallucinations, and performance degradation.
Ensure smooth integration of LLMs into existing workflows, providing real-time insights and predictive analytics.
Qualifications For AI/ML Engineer:
Solid experience in business analytics, data science, software development, and data engineering.
Expertise in Python and frameworks such as PyTorch, TensorFlow, or ONNX.
Hands-on experience working with LLMs and Generative AI, including model development and inference patterns.
Proven ability to design scalable systems leveraging LLMs, particularly in distributed computing environments.
Preferred Skills:
Experience in prompt engineering and prompt optimization.
Expertise in running experiments to evaluate generative AI performance.
Knowledge of production-level monitoring tools for ML models, including drift detection and mitigation strategies.
Excellent problem-solving skills and ability to work cross-functionally with data scientists, engineers, and SMEs.
Experience with safety, security and responsible use of AI.
Experience with red-teaming (adversarial testing) of generative AI.
Experience with developing AI applications with sensitive data such as PHI, PII and highly confidential data.
https://reqpedia.com/
Best reqpedia Tech Talent Recruitment
Welcome to Reqpedia, where opportunities meet talent! Find your dream job or discover the perfect candidate for your company Our platform connects employers.
Apply for this position
Application Status
Application Draft
In Progress
Submit Application
Pending
Review Process
Expected within 5-7 days
Similar Jobs
