This role is for one of our clients
Compensation: $130 per task
We are seeking experienced C engineers to build the dataset that teaches AI agents to reason about real codebases.
Codebase Q&A is a reinforcement-learning environment for training AI agents to explore and reason about software repositories they have never seen before. You will build difficult codebase-exploration tasks: high-level engineering questions about foundational C systems that cannot be answered by reading a single file, and that require runtime evidence to resolve — the kind of question a senior engineer answers by actually investigating the system.
This track works on some of the most consequential C codebases in production anywhere: PostgreSQL, Redis, git, OpenSSH, OpenSSL, curl, libuv, libgit2, libarchive, Open vSwitch, tcpdump, tmux and zstd.
Contract, task-based work paid at $130 per approved task, performed remotely in our Studio.
Requirements
Key Responsibilities
- Select tasks from a pre-validated pool of C repositories, each pinned to a specific git commit and arriving with an engineering question, positive and negative rubrics, foils, and a golden solution
- Explore the repository until you genuinely understand the subsystem in question, including the relevant PR or issue history
- Rewrite the task's question so it defeats frontier coding agents while remaining well-defined and fairly answerable
- Adjust rubrics and foils so they reward correct reasoning and reject plausible-but-wrong answers
- Run the full validation loop in Studio — Check, Golden, Run, Analyze — and iterate until every stage passes
- Document your work in the task README and submit for expert review, addressing reviewer feedback on returned tasks
Ideal Qualifications
- 3+ years of professional software engineering experience, with substantial production C
- Demonstrated ability to navigate a large, unfamiliar C codebase and explain how it behaves at runtime — not just what the source says
- Comfort with the things that make C systems hard to read: preprocessor-heavy builds, platform conditionals, manual memory management, and autotools/CMake/Meson build graphs
- Experience with databases, cryptography, networking protocols, or systems programming is highly relevant — the task mix is weighted toward architecture and system design (39%) and code onboarding (34%), with a meaningful security slice
- Ability to use debuggers, sanitizers, tracing, and profilers to establish what the code actually does
- Precise written English: the questions and rubrics you write are the product
Working Details
- Fully remote and asynchronous, with expert team leads across US, India, and Nigeria time zones
- Work is performed in our Studio; time is tracked with Insightful, and platform access is provisioned through Okta
- Onboarding requires identity verification, a background check, signed Terms of Work and CIIAA, a tax form, and passing a short calibration quiz before task access is granted
- Payment is per approved task; every task is reviewed by a second engineer and a super reviewer before acceptance
Application Process
- Submit your resume or relevant technical background to get started
- Qualified applicants may be asked to complete a brief technical assessment or provide additional information
We consider all qualified applicants without regard to legally protected characteristics and provide reasonable accommodations upon request.