About the company
Our client is a fast-growing technology company.
The role
Raydar is recruiting for this role on behalf of our client. Lead applied research on how machine learning models are measured and improved, with a focus on the quality of the data behind them. You will join early as a founding member of a small research function and set direction with a high degree of independence, partnering closely with subject-matter specialists.
What you'll do
- Create and maintain benchmarks, test suites and dataset classification schemes that establish quality standards.
- Run independent investigations and publish findings to build the company's visibility in the research community.
- Pinpoint weaknesses in current language models and assemble datasets that address them.
- Collaborate with domain specialists and delivery colleagues to describe what excellent training and testing data looks like in different fields.
- Shape the research roadmap, proposing ideas, designing experiments and rallying colleagues around them.
Requirements
What we're looking for
- About 5 or more years of research experience across academic and industry settings.
- A PhD in machine learning, computer science, statistics or a quantitative discipline from a leading program.
- A strong publication record at top machine learning conferences, in a lead or near-lead author role.
- Direct, full-cycle ownership of post-training work for large language models.
- Deep knowledge of model assessment, benchmark creation and data quality as they feed into model training.
- Prior research work in a corporate AI environment, such as a model developer or AI tooling provider.
- A research specialty in language model training or post-training, rather than speech, audio, vision or robotics.
- Skill in designing experiments, examining data and delivering quantitative analysis from start to finish.
- Curiosity about how models behave and the ability to suggest data-driven remedies for their shortcomings.
- Willingness to work on-site 5 days per week, with some flexibility to 4.
Bonus points
- A master's degree combined with an outstanding body of published work may be considered in place of a PhD.
- Comfort with Python and PyTorch for working alongside engineers on evaluation tooling.
Benefits
Compensation and benefits
- Base salary: USD 250,000 to 350,000 per year
- Equity
Location and work model
- New York, NY or San Francisco, CA, United States
- Hybrid: 5 days in-office, flexible to 4 days
- Full-time