This role is for one of our clients
Compensation: $70 - $110 per hour
Location: Bay Area, California, United States
Work Arrangement: Hybrid (On-site multiple days per week, as required)
Employment Type: Full-Time, W-2
Work Commitment: 40 hours per week
Initial Engagement: 6 months
Job Overview
We are seeking an experienced, board-certified physician to work directly with the research and program management teams at a leading AI lab, helping advance how next-generation AI models understand, reason about, and respond to real-world clinical challenges.
As a Clinical AI Expert, you will bring your medical expertise and independent clinical judgment to evaluate AI-generated medical content, develop high-quality clinical benchmarks, and establish reference solutions that define what accurate, safe, and evidence-based medical reasoning looks like.
This role is designed for practicing physicians with substantial post-residency experience, strong specialization in a clinical discipline, and the ability to assess medical reasoning beyond surface-level correctness. You will collaborate closely with AI researchers and engineering teams to improve the quality, reliability, and clinical relevance of advanced AI systems.
This is a full-time W-2 employment opportunity with a hybrid work arrangement in the San Francisco Bay Area. You will receive client-issued accounts and equipment and work directly within the client's internal tools and research environment.
Requirements
Key Responsibilities
- Clinical Quality Assurance: Review and evaluate clinical knowledge tasks and AI-generated medical responses, identifying gaps in clinical reasoning, missing considerations, unsafe recommendations, and deviations from established medical guidelines.
- Instruction Specifications and Reference Solutions: Develop detailed task specifications, create accurate reference answers, and design clinical problems that reflect real-world medical practice and decision-making.
- Clinical Benchmark Development: Design challenging, specialty-specific clinical tasks and evaluation datasets to measure the accuracy, depth, safety, and reliability of AI-generated medical reasoning.
- Medical Content Evaluation: Assess AI responses across diagnosis, treatment planning, patient management, clinical guidelines, and specialty-specific medical scenarios.
- Technical Feedback: Provide clear, structured, and actionable written feedback to help research teams identify weaknesses and improve model performance.
- Research Collaboration: Work with AI researchers, engineers, and specialists from adjacent domains to establish consistent evaluation standards and translate clinical expertise into measurable assessment criteria.
- Clinical Standards: Help develop medicine-specific evaluation frameworks, tools, and workflows that ensure AI outputs meet professional clinical standards.
Required Qualifications
- MD or DO degree from an accredited medical school.
- Completed residency training in a recognized medical specialty.
- Minimum of 4 years of independent clinical practice following residency. Residency and fellowship training do not count toward this requirement.
- Active, unrestricted medical license in at least one US state.
- Current board certification in the candidate's respective specialty.
- Demonstrated expertise in a defined clinical specialty, including but not limited to:
- Internal Medicine
- Oncology
- Radiology
- Emergency Medicine
- General or Specialty Surgery
- Psychiatry
- Cardiology
- Neurology
- Pediatrics
- Other recognized medical specialties and subspecialties
- Clear professional progression into senior clinical responsibilities, such as Attending Physician, Medical Director, Division Chief, Associate Professor, Professor, or Chief Medical Officer.
- Proven ownership of clinical decisions and experience applying independent medical judgment in patient care.
- Practical experience using large language models (LLMs) in a professional setting, with the ability to distinguish clinically sound reasoning from plausible but medically incorrect responses.
- Excellent written communication skills and the ability to explain complex clinical concepts through precise, well-structured feedback.
- Ability to commit reliably to 40 hours per week for the initial six-month engagement.
Preferred Qualifications
- Experience in utilization management, clinical informatics, medical affairs, or clinical quality assurance.
- Experience developing clinical guidelines, medical protocols, or evidence-based evaluation frameworks.
- Previous involvement in medical research, clinical education, or multidisciplinary healthcare initiatives.
- Familiarity with AI model evaluation, clinical benchmark development, or medical dataset quality assessment.
Employment Details
- Work Location: San Francisco Bay Area, California, United States.
- Work Arrangement: Hybrid, requiring on-site collaboration with the client's team multiple days per week, as needed.
- Employment Type: Full-time, W-2 employment.
- Work Schedule: 40 hours per week.
- Initial Duration: 6 months.
- Equipment and Access: Client-issued accounts and equipment will be provided.
- Work Environment: Direct collaboration with the client's research and program management teams, using internal tools and established workflows.
- Engagement Structure: This is a structured, role-based employment opportunity rather than a freelance or project-based engagement.
Location and Relocation Requirements
- Candidates must currently reside in the San Francisco Bay Area or be willing to relocate before the engagement begins.
- Relocation expenses must be covered by the candidate.
- Relocation assistance is not provided.
- Candidates must be available for regular on-site collaboration with the client's team multiple days each week, as required.
- This is not a fully remote position.