AI Safety Specialist - Evaluation Expert

About the role

Mercor connects creative and technical talent with leading AI research labs. This is a remote contract position as an AI Safety Practitioner, paid $60-$70 per hour.

Key responsibilities

• Evaluate AI-generated responses for safety, factual accuracy, policy compliance and overall quality.
• Review content involving misinformation, political persuasion, self-harm, violence, cyber, biosecurity and other sensitive domains.
• Apply and refine evaluation rubrics for RLHF, SFT and AI safety benchmarking.
• Identify unsafe outputs, hallucinations, reasoning failures and policy violations.
• Provide structured feedback to improve model alignment and safety performance.
• Collaborate with AI researchers and safety teams on ongoing evaluation initiatives.

Required profile

• Bachelor's degree or higher in Journalism, Communications, Psychology, Sociology, Public Policy, Law, Biology, Chemistry, Computer Science or a related discipline.
• 5+ years of professional experience in AI Safety, Trust & Safety, journalism, public policy, scientific research, security or a related field.
• Excellent written English, critical thinking and analytical reasoning skills.
• Ability to consistently evaluate nuanced and policy-sensitive scenarios.

Preferred

• Experience with AI Safety, RLHF, SFT, Trust & Safety or AI evaluation.
• Familiarity with safety policies, content moderation or evaluation rubric development.
• Experience reviewing complex, high-risk or ambiguous content.

Applications are processed daily through an online process: upload your resume, complete a short AI interview based on your resume and submit a form.