Other open roles
LLM Trainer - Agent Function call
Work Expert (WE) is an independent publishing and referral website. We are not a recruiter, hiring manager, agent or employer, and we are not affiliated with or endorsed by Turing. Applying takes you to the platform's own website, where we may be recorded as the referring source. We may receive a referral fee at no additional cost to you. Read our full affiliate disclosure →
Apply within 30 days of clicking - Turing only credits applications inside its 30-day attribution window.
About the Role
This position is within a project with one of the foundational LLM companies. The goal is to assist these foundational LLM companies in enhancing their Large Language Models. One way we help these companies improve their models is by providing them with high-quality proprietary data. This data serves two main purposes: first, as a basis for fine-tuning their models, and second, as an evaluation set to benchmark the performance of their models or competitor models. For example, in the case of Agent Completion (AC) data generation, your task will be to simulate high-quality multi-turn conversations between a user and a smart assistant that utilizes function-calling tools to accomplish user goals. You will craft these dialogues by playing both the assistant and the user, while simulating tool use where necessary to guide the assistant through complex decision-making and real-world reasoning scenarios.
What You'll Do
- Design multi-turn conversations that simulate real interactions between users and AI assistants using apps like calendar, email, maps, and drive.
- Emulate both the user and the assistant, including the assistant's tool calls (only when corrections are needed).
- Carefully select when and how the assistant uses available tools, ensuring logical flow and proper usage of function calls.
- Craft dialogues that demonstrate natural language, intelligent behavior, and contextual understanding across multiple turns.
- Generate examples that showcase the assistant’s ability to gracefully complete feasible tasks, recognize infeasible ones, and maintain engaging general chat when tools aren’t required.
- Ensure all conversations adhere to defined formatting and quality guidelines, using an internal playbook.
- Iterate on conversation examples based on feedback to continuously improve realism, clarity, and value for training purposes.
- Collaborate with peers and reviewers to maintain consistency and high standards in deliverables.
You're a Good Fit If You
- Required skills: Python, Java, JavaScript
- Strong general technical reasoning skills and the ability to model real-world assistant behavior using tool-based APIs.
- Ability to break down complex tasks and simulate realistic dialogues that reflect user expectations and assistant limitations.
- Experience in any programming language or tech stack is acceptable; a strong grasp of APIs, data formats (e.g., JSON), and logical thinking is more critical than specific toolsets.
- Excellent written communication skills in English, with a focus on clarity, tone, and instructional coherence.
- Creativity and attention to detail in crafting realistic scenarios and responses.
- Experience working with or around LLMs, virtual assistants, or function-calling frameworks is a plus.
- Ability to follow detailed guidelines and formatting standards with high consistency.
- 3+ years of overall professional experience in a technical or analytical field.
Role Highlights
Pay & Payout
Independent contractor engagement through Turing. Apply via the link below - Turing handles vetting, onboarding and payment.
Common Questions
Opens Turing in a new tab. You pay nothing; we may earn a referral fee.
Get similar jobs every week
Similar roles
More model evaluation work you may qualify for.
Senior Quality Lead
Guide QA teams in enhancing quality for AI annotation projects.
Quality Operations Lead
Manage quality operations for AI annotation projects and lead QA teams.
Quality Manager
Lead quality strategy for AI training data and annotation programs.