Other open roles
AI Benchmark & Red-Teaming Specialist
Referral link. You pay nothing; we may earn a fee.
Work Expert (WE) is an independent publishing and referral website. We are not a recruiter, hiring manager, agent or employer, and we are not affiliated with or endorsed by Turing. Applying takes you to the platform's own website, where we may be recorded as the referring source. We may receive a referral fee at no additional cost to you. Read our full affiliate disclosure →
Apply within 30 days of clicking - Turing only credits applications inside its 30-day attribution window.
About the Role
Based in San Francisco, California, Turing is the world’s leading research accelerator for frontier AI labs and a trusted partner for global enterprises deploying advanced AI systems. Turing supports customers in two ways: first, by accelerating frontier research with high-quality data, advanced training pipelines, plus top AI researchers who specialize in coding, reasoning, STEM, multilinguality, multimodality, and agents; and second, by applying that expertise to help enterprises transform AI from proof of concept into proprietary intelligence with systems that perform reliably, deliver measurable impact, and drive lasting results on the P&L.
What You'll Do
- Obscure Fact Discovery: Deeply research niche topics to identify factual ground truths that cannot be surfaced via simple, direct search queries.
- Source Verification: Rigorously verify facts against primary or highly credible sources, cross-check for contradictory evidence, and document exact source URLs.
- Task Construction: Draft clear, answerable benchmark tasks designed specifically to expose structural weaknesses in model research strategies.
- Red-Teaming & Model Testing: Execute tasks against designated SOTA models to confirm they reliably trigger intended model failures.
- Failure Diagnosis & Debugging: Trace model run logs, search trajectories, and intermediate outputs to pinpoint exactly where and why the model failed (distinguishing genuine model limitations from ambiguous prompts).
- Pattern Scaling & Review: Apply validated "model-breaking" research patterns across new domains and, for strong performers, review tasks authored by peer trainers.
- Bachelor or higher degree from relevant engineering domain is a must for this position.
- Prior experience specifically designing prompt-injection or reasoning-trap tasks that break SOTA models.
You're a Good Fit If You
- Required skills: LLM
- Core Experience: 1+ years in investigative research, fact-checking, AI benchmark task authoring, red-teaming, or technical QA review.
- Benchmark Familiarity: Proven experience authoring, testing, or reviewing tasks for AI benchmarks such as GAIA, BrowseComp, SimpleQA, SWE-bench, Terminal-Bench, or Humanity’s Last Exam (HLE).
- Analytical & OSINT Skills: Intermediate-to-advanced web search and Open Source Intelligence (OSINT) skills; ability to navigate academic archives, local records, or obscure databases.
- Model Trajectory Debugging: Strong analytical capability to inspect model reasoning chains, search logs, and source selection to diagnose root causes of failure.
- Precision Writing: Ability to author clear, unambiguous benchmark prompts with bulletproof, primary-sourced answer keys.
Role Highlights
Pay & Payout
Independent contractor engagement through Turing. Apply via the link below - Turing handles vetting, onboarding and payment.
Common Questions
Opens Turing in a new tab. You pay nothing; we may earn a referral fee.
Get similar jobs every week
About Turing for this role
We track 229 open Turing roles on AI Work Expert. Across 21 hourly listings, the typical Turing rate runs from $15 to $15 an hour; this role lists Pay varies. Our reliability rating for Turing is very reliable, with an onboarding time of 5-14 days.
Similar roles
More model evaluation work you may qualify for.
Project Lead - AI Training
Lead the end-to-end delivery of large-scale multimodal AI annotation projects while managing clients, teams, quality, timelines, and budgets. The role requires making quick, data-d
Digital Imaging Specialist/ Digital Artist
Edit, annotate, and modify existing images based on briefs, references, and guidelines.
Merchandising / Category Managers Specialist
Benchmark Task Authoring: Design realistic, high-difficulty retail exercises, assortment planning scenarios, promotional strategy models, and planogram challenges with gold-standar