Generalist - English & Urdu

About this role

**PLEASE NOTE: You must take the Bilingual Competency interview in Urdu to be considered for this role.** **Location**: Global **Type**: Contract Work **Fluent Language Skills Required:** Urdu (native fluency) and English (strong proficiency) **Why this role matters:** Your job is to assess Urdu AI-generated responses and identify specific strengths and areas of improvement for these responses – your work will be used to create the "perfect AI-generated response" at a later stage of this project. Note the analysis you create will be in English. **What You'll Do** - Conduct fact-checking using trusted public sources and external tools - Generate high-quality human evaluation data by identifying response strengths, areas for improvement, and factual inaccuracies - Assess reasoning quality, clarity, tone, and completeness of responses - Ensure model responses align with expected conversational behavior and system guidelines **Who You Are** - You hold a **Bachelor's degree** - You are a **native speaker** in **Urdu** - You have **significant experience using large language models** (LLMs) and understand how and why people use them - You have **excellent writing skills in English** and can clearly articulate nuanced feedback - You have **strong attention to detail** and consistently notice subtle issues others may overlook - You have a background or experience in domains requiring **structured analytical thinking** (e.g., research, policy, analytics, linguistics, engineering) **Nice-to-Have Specialties** - Prior experience with **RLHF, model evaluation, or data annotation work** - Experience writing or editing **high-quality written content** - Experience comparing multiple outputs and making **fine-grained qualitative judgments** **What Success Looks Like** - You identify factual inaccuracies, reasoning errors, and communication gaps in model responses - You produce clear, consistent, and reproducible evaluation artifacts - Your feedback leads to measurable improvements in response quality and user experience

Skills & domains

• ai-training
• rlhf
• sme
• annotation
• Data Analysis