Data Scientist, AI and LLM Applications

A leading organization is seeking a mid-level Data Scientist to support a high-visibility initiative focused on applying advanced artificial intelligence to complex healthcare data. This role centers on building an AI-driven utilization management criteria analyzer that helps business stakeholders understand the clinical and financial impact of changing medication utilization strategies.

The Data Scientist will work closely with technical and non-technical partners to design, develop, and evaluate large language model solutions, agent-based AI workflows, and data science models. The position combines hands-on development with direct stakeholder engagement and requires comfort operating in a fast-moving, evolving environment.

---

Responsibilities

Data science and modeling

• Perform exploratory data analysis, statistical analysis, and feature engineering on large, complex healthcare datasets.
• Develop and deploy scalable machine learning models to support forecasting, simulation, and decision intelligence use cases.
• Work with both structured and unstructured data, including clinical notes and claims data.
• Support scenario modeling and “what-if” analysis to quantify the impact of different utilization management strategies.

LLM and AI development

• Build large language model pipelines to extract metrics, values, units, and contextual information from unstructured clinical data.
• Design and refine prompt engineering strategies to improve consistency, accuracy, and reliability of LLM outputs.
• Develop explainable AI solutions that include supporting evidence and traceable reasoning.
• Evaluate model outputs for accuracy, hallucination risk, and overall performance, and drive iterative improvements.

Agentic AI systems

• Design and implement agent-based AI workflows that support multi-step reasoning and decision logic.
• Implement guardrails such as confidence scoring, validation checks, and fallback mechanisms.
• Orchestrate AI components to support complex analytical workflows and simulations.

Collaboration and platform development

• Partner with product, underwriting, and clinical strategy stakeholders to translate business problems into data science solutions.
• Contribute to the development of a self-service analytics and decision intelligence platform.
• Communicate insights clearly to business audiences and support stakeholder decision-making.

---

Required experience and skills

Core data science

• Strong proficiency in Python and SQL.
• Experience working with large datasets.
• Hands-on experience with exploratory data analysis, statistical analysis, and machine learning model development.
• Ability to analyze data and present findings to non-technical stakeholders.

LLM and AI

• Hands-on experience working with large language models.
• Demonstrated experience with prompt engineering and LLM evaluation frameworks.
• Experience building production-level LLM solutions.

Agentic AI

• Experience designing multi-step AI workflows.
• Familiarity with AI agents or orchestration logic.
• Experience implementing guardrails such as confidence scoring, validation, or fallback logic.

Data handling

• Experience working with unstructured data such as text, documents, or clinical notes.
• Strong skills in data extraction, transformation, and preparation.

Nice-to-have experience

• Experience in the healthcare or payer domain.
• Exposure to clinical data or chart data review workflows.
• Experience with forecasting, financial modeling, or scenario simulation systems.
• Familiarity with explainable AI approaches.
• Experience building recommendation systems.

This role requires strong communication skills, the ability to work directly with stakeholders, and the flexibility to adapt priorities as project needs evolve.

FAQ

1. What are the core responsibilities of a Data Scientist focused on AI and LLM applications?
This role develops and applies AI and large language model (LLM) solutions to solve business problems and improve user experiences. Responsibilities include building prototypes, analyzing datasets, fine-tuning models, and evaluating performance. The data scientist also collaborates with engineering and product teams to operationalize AI solutions.

2. What types of AI and LLM projects are typically handled in this role?
Projects may include conversational AI, document summarization, recommendation systems, semantic search, and content generation tools. The role often involves both experimentation and production-oriented model development. The focus is on creating practical and scalable AI applications.

3. What tools and technologies are commonly used?
Common tools include Python, SQL, and machine learning libraries such as PyTorch or TensorFlow. LLM frameworks, vector databases, and cloud AI platforms are also frequently used. Data visualization and experimentation tools support analysis and evaluation.

4. How are LLMs integrated into business applications?
LLMs are integrated through APIs, orchestration frameworks, and custom workflows that connect AI capabilities with business systems. The data scientist helps design prompts, retrieval pipelines, and evaluation methods. Scalability and reliability are important considerations.

5. How is model performance evaluated in this role?
Performance is measured using metrics such as accuracy, relevance, latency, and user satisfaction. Testing and validation help ensure outputs are reliable and aligned with business goals. Continuous monitoring is often required after deployment.

6. What challenges are common in AI and LLM application development?
Challenges include managing hallucinations, ensuring responsible AI usage, and handling large-scale data processing. Balancing experimentation with production stability can also be difficult. Strong analytical and problem-solving skills are essential.