Other open roles
Behavioral Health Expert, AI Safety and Model Evaluation
Work Expert (WE) is an independent publishing and referral website. We are not a recruiter, hiring manager, agent or employer, and we are not affiliated with or endorsed by Mercor. Applying takes you to the platform's own website, where we may be recorded as the referring source. We may receive a referral fee at no additional cost to you. Read our full affiliate disclosure →
About the Role
Help a leading AI lab ensure its models respond to people with care, balance and sound judgment. A leading AI lab is seeking behavioral health experts to help evaluate and improve how its AI models handle sensitive, everyday conversations. People increasingly turn to AI for support with relationships, family dynamics, emotional wellbeing, personal beliefs and difficult life decisions. These conversations rarely involve crisis content, but they are exactly where a model's judgment matters most: whether it stays balanced, avoids taking sides, resists telling people what they want to hear, respects a person's beliefs without endorsing or dismissing them, and understands the limits of its role. You'll review these interactions, assess whether the model's responses are appropriate, neutral and safe, and help the lab's researchers define what good looks like. If you bring professional experience in mental health, counseling, social work or behavioral science, and you can articulate clearly why one response serves a person better than another, this role is for you. This is a part-time commitment of at least 20 hours per week, with the option to increase to up to 40 hours per week. This is a W-2 employment position with Cincinnatus LLC, with the opportunity to be placed at a leading AI lab as part of their extended workforce.
What You'll Do
- Evaluate conversations between users and AI models across topics such as relationship and family advice, emotional wellbeing, spiritual and metaphysical questions, and unconventional or unfounded beliefs, and assess whether responses are neutral, appropriate and safe.
- Identify patterns of concern, including excessive agreement, taking sides, reinforcing distorted or unfounded beliefs, moralizing, or overstepping into clinical or directive advice, and document them with clear written rationale.
- Develop rubrics, guidelines and reference responses that define a balanced, supportive and appropriately bounded reply, grounded in established practice from counseling and behavioral health.
- Design test scenarios and conversations that probe how models handle sensitive but non crisis topics.
- Collaborate with the lab's researchers and fellow experts to keep evaluation standards consistent, calibrated and well documented.
You're a Good Fit If You
- A degree in psychology, counseling, social work, behavioral health, behavioral science, human services or a closely related field, or equivalent professional experience in the mental health field.
- 3+ years of professional experience supporting people in a mental health, counseling or social services setting, for example as a therapist, counselor, clinical social worker, psychologist, psychiatric nurse, case manager, crisis counselor, peer support specialist or mental health advocate. Clinical licensure (for example LMFT, LCSW, LPC, LMHC, PsyD or PhD) is valued but not required.
- Demonstrated ability to remain neutral and nonjudgmental across diverse perspectives, relationships, belief systems and worldviews, and to explain the reasoning behind a professional judgment.
- Working familiarity with concepts such as sycophancy, cognitive distortions, healthy boundaries and client centered approaches such as motivational interviewing.
- Ability to engage reliably for at least 20 hours/week during weekdays.
- Strong written communication skills and the ability to deliver precise, well structured written feedback.
- Nice to have: a background in AI safety, applied ethics, trust and safety or content policy; experience in couples, family or relationship counseling; familiarity with spiritual care, religious or alternative belief communities, or the psychology of misinformation and conspiracy belief; prior experience evaluating, annotating or red teaming AI systems. You don't need all of these to apply.
Role Highlights
Pay & Payout
Independent contractor engagement. Apply via the link below - Mercor handles onboarding and pays weekly via Stripe or Wise.
Common Questions
Opens Mercor in a new tab. You pay nothing; we may earn a referral fee.
Get similar jobs every week
Similar roles
More model evaluation work you may qualify for.
Primary Care Physician (MD) - Clinical Documentation & AI Evaluation
Mercor is hiring practicing Primary Care Physicians (MD) on behalf of a healthcare AI partner building advanced clinical documentation and decision-support tools. In this role you
Cardiologist – Clinical Consultant (ATTR Cardiac Amyloidosis)
An AI healthcare company that automates administrative and pharmacy workflows to help patients access life-changing medications and specialty treatments is seeking an experienced
Cardiology NP / PA / RN – ATTR Clinical Consultant
An AI healthcare company that automates administrative and pharmacy workflows to help patients access life-changing medications and specialty treatments is seeking an experienced