Other open roles
Software Engineer – AI Code Evaluation & Benchmarking (US candidates only)
Work Expert (WE) is an independent publishing and referral website. We are not a recruiter, hiring manager, agent or employer, and we are not affiliated with or endorsed by Turing. Applying takes you to the platform's own website, where we may be recorded as the referring source. We may receive a referral fee at no additional cost to you. Read our full affiliate disclosure →
Apply within 30 days of clicking - Turing only credits applications inside its 30-day attribution window.
About the Role
We are looking for experienced Software Engineers to help evaluate, benchmark, and improve the coding capabilities of frontier AI models. In this role, you will assess AI-generated code, validate solutions against real-world software engineering tasks, identify correctness and quality issues, and contribute to the development of high-quality evaluation datasets and benchmarks. This position is ideal for engineers who enjoy code review, debugging, problem-solving, and applying strong software engineering judgment to complex technical scenarios. Your work will directly contribute to measuring and improving the performance of advanced AI coding systems. What Does Day-to-Day Look Like?
What You'll Do
- Review and evaluate AI-generated code for correctness, efficiency, maintainability, and adherence to requirements.
- Analyze software engineering tasks and validate whether proposed solutions meet expected outcomes.
- Debug code, reproduce issues, and verify fixes across different programming environments.
- Assess model-generated explanations, reasoning, and implementation approaches for technical accuracy.
- Create, refine, and maintain evaluation datasets, benchmarks, and grading rubrics for coding tasks.
- Identify edge cases, failure modes, and areas where AI systems struggle with software engineering problems.
- Document findings clearly and provide structured feedback to improve evaluation quality and consistency.
- Collaborate with project teams to establish quality standards and evaluation methodologies.
You're a Good Fit If You
- Required skills: Python, Git, Docker
- Bachelor's or Master's degree in Computer Science, Software Engineering, or a related technical field.
- 3+ years of professional software engineering experience.
- Strong proficiency in one or more of the following languages: Python, Java, C/C++, Go, Swift, Objective-C, PHP, or SQL.
- Strong understanding of data structures, algorithms, software design principles, and debugging methodologies.
- Experience performing code reviews and evaluating code quality in production or large-scale codebases.
- Ability to analyze complex technical problems and assess solution correctness with minimal supervision.
- Familiarity with version control systems (e.g., Git) and modern software development workflows.
- Strong written communication skills and attention to detail.
Role Highlights
Pay & Payout
Independent contractor engagement through Turing. Apply via the link below - Turing handles vetting, onboarding and payment.
Common Questions
Opens Turing in a new tab. You pay nothing; we may earn a referral fee.
Get similar jobs every week
Similar roles
More coding work you may qualify for.
DevOps and Cloud Infrastructure Engineer
Build and manage secure cloud infrastructure for PII detection workflows.
ML and Data Engineer — Data Quality & PII Compliance
Improve data quality and compliance for enterprise data pipelines.
ML and Data Engineer — Data Quality & PII Compliance
Validate data quality and compliance for enterprise data pipelines.