Domain Expert Reviewer
Job at a glance
Domain Expert Reviewer | Contractor | 40 hrs/week | Worldwide Remote Turing is seeking a highly qualified Domain Expert Reviewer to support quality assurance for Large Language Model (LLM) evaluation projects. This is an 8-week contractor engagement requiring 40 hours per week with at least 4 hours of PST overlap. You will apply your subject matter expertise to ensure the accuracy, consistency, and reliability of AI evaluation data across academic, research, and industry disciplines.
About Turing Based in San Francisco, California, Turing is the world's leading research accelerator for frontier AI labs and a trusted partner for global enterprises deploying advanced AI systems. Turing accelerates frontier research with high-quality data, advanced training pipelines, and top AI researchers specializing in coding, reasoning, STEM, multilinguality, multimodality, and agents. Key Responsibilities Review and validate domain-specific prompts within your area of expertise.
Evaluate completed tasks for factual accuracy, reasoning quality, completeness, and compliance with project guidelines. Identify factual inaccuracies, logical inconsistencies, hallucinations, outdated information, and low-quality annotations. Ensure prompts are challenging, relevant, and aligned with project objectives. Provide clear, constructive, and evidence-based feedback to contributors to improve task quality.
Maintain consistency across reviews by following established quality standards and guidelines. Escalate ambiguous or complex cases and document review findings as needed. Collaborate with project managers and AI teams to continuously improve evaluation quality and review processes. Minimum Qualifications Master's degree or higher in a relevant academic or professional discipline. Minimum 3 years of professional, research, teaching, or industry experience in your area of expertise.
Prior experience reviewing, evaluating, editing, or quality-checking domain-specific content is preferred. Excellent written English communication and strong analytical skills. Sharp attention to detail with the ability to identify factual, logical, and contextual errors. Engagement Details Commitment: 40 hours per week with at least 4 hours PST overlap Employment Type: Contractor assignment (no medical/paid leave benefits) Duration: 8 weeks