Work Expert (WE) is an independent publishing and referral website. We are not a recruiter, hiring manager, agent or employer, and we are not affiliated with or endorsed by Mercor. Applying takes you to the platform's own website, where we may be recorded as the referring source. We may receive a referral fee at no additional cost to you. Read our full affiliate disclosure →
About the Role
Join a leading AI lab's cutting-edge GenAI team to be at the core of the AI revolution, where your expertise fuels the development of the most advanced AI models. A leading AI lab is building and evaluating frontier models and needs experienced machine learning and natural language processing practitioners to act as ground-truth experts. You will author challenging, real-world tasks, generate reference solutions, and evaluate model outputs to surface reasoning and capability gaps in a target model.
What You'll Do
Task design and development: Design challenging, real-world ML and NLP problems drawn from your area of expertise (e.g., model training and evaluation, language understanding and generation, retrieval, applied ML pipelines) that target specific capability gaps in a frontier AI model.
Spec and golden-solution generation: Integrate the problems into an agentic development environment, preparing all necessary components using Python.
Evaluation and analysis: Evaluate the target model's performance on your tasks.
Headroom identification: Identify tasks where the target model fails, and classify the nature of the failure.