Video Evaluation Generalist

About this role

Mercor is seeking detail-oriented writing experts to contribute to a high-impact video AI research project with a leading lab. We're looking for detail-oriented annotators to evaluate short video clips. In each task you'll watch two videos side by side, compare them against a provided rubric, and score them. This is a short-term, flexible opportunity for professionals with strong academic backgrounds and a knack for instructional clarity. Ideal for those who enjoy distilling complex concepts into well-crafted text. * * * ### **Job Details:** - **Evaluate Paired Video Outputs:** Watch two short video clips presented side by side and compare them against a provided rubric across dimensions such as prompt adherence, motion quality, visual artifacts, and physical plausibility. - **Score and Justify Preferences:** Assign rubric scores to each clip, select an overall preference between the two, and document a brief written justification supporting the decision where prompted**.** - **Maintain Scoring Consistency:** Apply the same standard across high task volumes, holding calibration to the rubric steady so that judgments remain reliable and comparable throughout a production run. ### **Minimum Qualifications:** - BS or BA from a reputable institution completed or in progress. - Strong writing and critical thinking skills. - Ability to work independently and meet deadlines. ### **Preferred Qualifications:** - 2+ years of experience in teaching or research. ### **Application & Onboarding Process:** - Complete an AI-led interview, this should take around 15 minutes. - If selected, you will be invited to work on the project. ### **More Details About This Role:** - Expect to contribute at least **20 hours per week**. - Expect a commitment of around 1.5 weeks. - You’ll be working in a structured project environment with clear goals and tools.

Skills & domains

• ai-training
• rlhf
• sme
• annotation