Other open roles

View all opportunities

Senior Software Engineer — AI Evaluation & Benchmarks

$100
per hour
Hourly contract Remote Last verified 15 Nov 2026
Listed on Alignerr ↗
alignerr.com
Apply

Referral link. You pay nothing; we may earn a fee.

Affiliate disclosure

Work Expert (WE) is an independent publishing and referral website. We are not a recruiter, hiring manager, agent or employer, and we are not affiliated with or endorsed by Alignerr. Applying takes you to the platform's own website, where we may be recorded as the referring source. We may receive a referral fee at no additional cost to you. Read our full affiliate disclosure →

About the Role

Alignerr is hiring experienced software engineers to design and build the coding benchmarks and data pipelines that evaluate frontier AI models: the systems that decide whether an AI can really reason, debug and write production-quality software. It's a fully remote hourly contract for 3 months, with full-time availability preferred.

What You'll Do

  • Design and implement coding benchmarks that evaluate frontier AI models on real-world programming tasks.
  • Build and maintain scalable data pipelines for AI evaluation workflows.
  • Analyze AI-generated code for correctness, reliability and edge-case failures.
  • Create structured evaluation scenarios that test reasoning, debugging and code quality.
  • Work with large code repositories and multi-language environments.
  • Give detailed technical feedback on model performance and failure patterns.

You're a Good Fit If You

  • 4+ years of professional software engineering experience (non-negotiable).
  • Experience at a high-growth tech company or top-tier software organization.
  • Expert proficiency in Python.
  • Proven experience designing and implementing LLM coding benchmarks and data pipelines.
  • Strong command of Git and modern development workflows.
  • Bilingual or native English speaker with strong written communication.
  • Best match: senior or lead engineers; extra languages (JavaScript, Go, C++), CI/CD and unit testing, security or open-source background.

Role Highlights

Pay (stated by Alignerr)
$80-$100/h
Contract
Hourly contract
Location
Remote
Category
Coding
Posted
yesterday
Job ID
WE-8620

Pay & Payout

Independent hourly contract through Alignerr (powered by Labelbox). Alignerr pays weekly: work from Monday to Sunday is paid by the following Friday.

Common Questions

Alignerr has a reliability rating of reliable based on the listings we track, with an onboarding time of 1-2 weeks. See our full Alignerr review for the details behind that rating.

Applying takes you to Alignerr's own website, where the application is completed. We are not a recruiter or employer and do not process applications directly - we may be recorded as the referring source.

$80-$100/h - Independent hourly contract through Alignerr (powered by Labelbox). Alignerr pays weekly: work from Monday to Sunday is paid by the following Friday.

Design and implement coding benchmarks that evaluate frontier AI models on real-world programming tasks. Build and maintain scalable data pipelines for AI evaluation workflows. Analyze AI-generated code for correctness, reliability and edge-case failures.

Apply on Alignerr

Opens Alignerr in a new tab. You pay nothing; we may earn a referral fee.

Get similar jobs every week

What are you interested in?
Coding & software engineering

We only use your email to send job alerts. Read our Privacy Policy.

$80-$100/h
Referral link · you pay nothing
Apply on Alignerr

About Alignerr for this role

We track 2 open Alignerr roles on AI Work Expert. This role lists $80-$100/h. Our reliability rating for Alignerr is reliable, with an onboarding time of 1-2 weeks.