About the Role
Own complex, time-sensitive data and evaluation programs for AI research teams and internal partners. This role brings together technical program management, data operations, and vendor coordination to turn evolving requirements into reliable programs that improve AI and agent evaluation.
What You'll Do
Manage data and evaluation programs from scoping and requirements gathering through production, quality assurance, delivery, and retrospectives.
Translate ambiguous technical requests into specifications with clear milestones, owners, dependencies, and acceptance criteria.
Improve data quality processes through quantitative metrics and qualitative review, including analysis of task coverage, difficulty, and reward distributions.
Manage external vendors across the delivery lifecycle and scale delivery pipelines.
Improve production workflows, documentation, automation, and cross-functional handoffs.
Partner with research, platform, and marketplace teams to turn program learnings into product improvements.
What We're Looking For
At least 3 years of experience owning complex technical programs in AI/ML, data operations, support engineering, applied research, forward-deployed engineering, or a related cross-functional field.
Experience turning ambiguous technical requirements into executable plans and maintaining data quality through metrics and direct inspection.
Strong judgment assessing dataset quality, moving between aggregate measures and individual examples.
Experience managing vendors, improving delivery pipelines, and documenting or automating workflows and handoffs.
Background supporting AI research teams, technical enterprise customers, or other teams with urgent, changing requirements.
Familiarity with RLHF, reinforcement-learning environments, agent evaluation, human-in-the-loop data pipelines, annotation, or expert data collection is valuable.
Comfort working independently in a fast-paced startup environment.
Compensation & Benefits
Base salary is $130,000 to $225,000 USD annually. Visa sponsorship and relocation support are available for strong full-time candidates.
Location
On-site in San Francisco, California. The role is also open to remote candidates who can maintain 70% to 80% time zone overlap with San Francisco or Singapore.