Referral link. You pay nothing; we may earn a fee.
Affiliate disclosure
Work Expert (WE) is an independent publishing and referral website. We are not a recruiter, hiring manager, agent or employer, and we are not affiliated with or endorsed by Alignerr. Applying takes you to the platform's own website, where we may be recorded as the referring source. We may receive a referral fee at no additional cost to you. Read our full affiliate disclosure →
About the Role
Alignerr is hiring experienced software engineers to design and build the coding benchmarks and data pipelines that evaluate frontier AI models: the systems that decide whether an AI can really reason, debug and write production-quality software. It's a fully remote hourly contract for 3 months, with full-time availability preferred.
What You'll Do
Design and implement coding benchmarks that evaluate frontier AI models on real-world programming tasks.
Build and maintain scalable data pipelines for AI evaluation workflows.
Analyze AI-generated code for correctness, reliability and edge-case failures.
Create structured evaluation scenarios that test reasoning, debugging and code quality.
Work with large code repositories and multi-language environments.
Give detailed technical feedback on model performance and failure patterns.
You're a Good Fit If You
4+ years of professional software engineering experience (non-negotiable).
Experience at a high-growth tech company or top-tier software organization.
Expert proficiency in Python.
Proven experience designing and implementing LLM coding benchmarks and data pipelines.
Strong command of Git and modern development workflows.
Bilingual or native English speaker with strong written communication.
Best match: senior or lead engineers; extra languages (JavaScript, Go, C++), CI/CD and unit testing, security or open-source background.
Independent hourly contract through Alignerr (powered by Labelbox). Alignerr pays weekly: work from Monday to Sunday is paid by the following Friday.
Common Questions
Alignerr has a reliability rating of reliable based on the listings we track, with an onboarding time of 1-2 weeks. See our full Alignerr review for the details behind that rating.
Applying takes you to Alignerr's own website, where the application is completed. We are not a recruiter or employer and do not process applications directly - we may be recorded as the referring source.
$80-$100/h - Independent hourly contract through Alignerr (powered by Labelbox). Alignerr pays weekly: work from Monday to Sunday is paid by the following Friday.
Design and implement coding benchmarks that evaluate frontier AI models on real-world programming tasks. Build and maintain scalable data pipelines for AI evaluation workflows. Analyze AI-generated code for correctness, reliability and edge-case failures.
We track 2 open Alignerr roles on AI Work Expert. This role lists $80-$100/h. Our reliability rating for Alignerr is reliable, with an onboarding time of 1-2 weeks.