This role is for one of our clients
Pay: 30–100/hour
Job Type: Contractor (~15 hrs a week)
Location: Remote
Schedule: Flexible, you pick the hours and days (including weekends if desired)
Required Skills: Python3, Java, Rust, Algorithms, Basics of C++, TypeScript, Bug fixing, Feature implementation, Codebase refactoring, Performance optimization
We are seeking an experienced Senior Software Engineer to contribute to the training and evaluation of next-generation artificial intelligence systems. This role is ideal for engineers with strong backend development expertise, excellent problem-solving abilities, and a proven track record of building reliable, scalable, and high-performance software.
You will design realistic and challenging coding tasks that assess AI systems' ability to implement features, fix bugs, refactor existing code, and optimize application performance. You will also develop deterministic verification mechanisms that accurately validate solutions, accepting reasonable implementations that meet the requirements while rejecting incorrect or incomplete submissions.
The role requires strong software engineering judgment, extensive debugging experience, and a solid understanding of testing methodologies. Previous AI experience is not required; your technical expertise and independent judgment are what matter most.
Requirements
Key Responsibilities
- Coding Task Development: Design challenging, real-world software engineering problems involving feature implementation, bug fixes, code refactoring, and performance optimization.
- Reference Solution Development: Create high-quality reference implementations that demonstrate correct, efficient, and maintainable approaches to solving technical problems.
- Automated Verification: Build deterministic verifiers, validation scripts, and comprehensive test cases to evaluate solution correctness and ensure reliable task assessment.
- Backend Engineering: Apply expertise in backend development, scalable services, APIs, asynchronous processing, and distributed systems to create realistic engineering challenges.
- Debugging and Troubleshooting: Analyze complex software defects, investigate system bottlenecks, and identify root causes of unexpected behavior.
- Testing Strategy and Quality Assurance: Design effective testing strategies, develop and maintain large test suites, diagnose flaky tests, and improve overall software quality.
- Code Review: Evaluate code for correctness, readability, maintainability, performance, security, and adherence to engineering best practices.
- Refactoring and Optimization: Develop tasks that assess an AI system's ability to improve legacy code, simplify complex implementations, and optimize resource utilization.
- Technical Reasoning Assessment: Evaluate AI-generated reasoning related to architecture, debugging, algorithms, system design, and implementation decisions.
- Technical Documentation: Document task requirements, expected behavior, test coverage, solution approaches, and verification criteria clearly and consistently.
- Continuous Improvement: Analyze model failures and refine coding tasks and evaluation methods to improve the quality and reliability of AI-generated software.
Required Qualifications
- Strong professional experience in software engineering, particularly backend development and production-grade systems.
- Proficiency in one or more programming languages, including Java, Node.js, Go, Rust, TypeScript, Python, C++, or similar technologies.
- Proven experience designing, building, and maintaining scalable services and APIs.
- Strong understanding of data structures, algorithms, system design, and fundamental computer science concepts.
- Extensive experience with debugging, troubleshooting, performance optimization, and code review.
- Practical experience with automated testing, test design, and software quality assurance.
- Ability to identify implementation defects, edge cases, inefficient algorithms, and architectural weaknesses.
- Strong understanding of code maintainability, refactoring, reliability, and software engineering best practices.
- Excellent analytical, problem-solving, and technical communication skills.
- Ability to work independently and exercise sound technical judgment when evaluating complex engineering problems.
Must-Have Skills
- Senior Software Engineering
- Backend Development
- Scalable Services and API Design
- Java, Node.js, Go, Rust, TypeScript, Python, or C++
- Data Structures and Algorithms
- Debugging and Root-Cause Analysis
- Automated Testing and Test Strategy
- Code Review and Refactoring
- Performance Optimization
- Technical Problem-Solving
Good-to-Have Skills
- Large-Scale Test Suite Design and Maintenance
- Flaky Test Diagnosis and Resolution
- Test Automation Frameworks
- Continuous Integration and Continuous Delivery (CI/CD)
- Distributed Systems and Microservices
- System Architecture and Design Patterns
- Regression, Integration, and Unit Testing
- Deterministic Evaluation and Automated Verification
- AI Coding Agent Evaluation
- Legacy System Modernization
- Production Reliability and Performance Engineering
Preferred Qualifications
- Demonstrated success delivering scalable, reliable, and high-performance backend applications.
- Experience resolving complex software defects and improving system performance.
- Proven ability to implement complex features and refactor large codebases.
- Strong background in test strategy, automated validation, and improving engineering quality standards.
- Experience establishing testing best practices across engineering teams.
- Ability to translate complex engineering requirements into clear, reproducible coding tasks and evaluation criteria.
Selection Process
- Application: Submit your application and complete the required screening questions.
- AI Interview: Complete an approximately 30-minute interview focused on your technical expertise and problem-solving approach.
- Technical Assessment: Complete a technical assessment if required.
- Hiring Manager Review: Your technical background, assessment results, and suitability for the project will be reviewed.
- Onboarding: Selected candidates receive the necessary instructions and resources to begin their assignments.
Candidates are expected to complete interviews and assessments independently, without assistance from AI tools or other individuals. The same principle applies to project assignments, where the quality of the work depends on each engineer's own expertise and judgment.
Compensation Structure
Compensation is output-based, with payment provided for tasks that meet the defined project specifications and acceptance criteria. Completion time may vary based on task complexity, individual experience, and workflow.
Minimum submission requirements apply, and participants are expected to complete a specified minimum number of tasks per week.
Start Timeline and Availability
The selection process typically moves quickly, with roles often filled within approximately 48 hours. Successful candidates should be prepared to begin their first tasks within 24–48 hours of completing onboarding.