About the Role
Lead the data quality team at an AI and machine learning company focused on reinforcement learning and agent evaluation. You will shape the strategy and systems used to measure and improve training data, helping ensure it is useful for training agents on real-world tasks.
What You'll Do
Lead development of systems that evaluate tasks across reinforcement learning environments, synthetic data, benchmarks, and specialized workflows.
Define data quality strategy, establish standards, and design experiments to assess agent outputs.
Develop scalable methods to validate synthetic data, including failure analysis, task mutation checks, and trajectory audits.
Work with research engineers, domain experts, and data vendors to diagnose quality issues and improve data generation workflows.
Turn research insights into production tools, dashboards, validation pipelines, and feedback loops.
Develop shared standards for identifying training data that is useful beyond superficial correctness.
Mentor research engineers and promote technical rigor, clear communication, and effective execution.
What We're Looking For
At least 5 years of research or engineering experience, including building AI or machine learning data evaluation systems.
Experience leading technical projects or teams through ambiguous problems, from definition to implementation and iteration.
Advanced proficiency with Python, Docker, and Linux.
Background in AI research, evaluation, post-training, synthetic data, benchmarks, or data quality infrastructure.
Strong understanding of what makes agent training data realistic, learnable, diverse, reliable, and useful.
Experience designing metrics, experiments, and quality assurance processes, and translating research into production systems.
Experience working with subject-matter experts and converting domain judgment into scalable review or data generation workflows.
Clear written communication, strong attention to detail, and ability to work independently in an early-stage startup environment.
Compensation & Benefits
Base salary: $130,000 to $225,000 annually. Visa sponsorship is available.
Location
On-site in the United States.