Senior Software Engineers

About this role

Join us in helping advance the next generation of AI models! We are looking for experienced Software Engineers to contribute to a cutting-edge AI evaluation project. This project plays a key role in improving how frontier language models understand, reason about, and generate production-quality software. You'll evaluate AI-generated solutions through the lens of an experienced software engineer, helping shape the future of AI-assisted software development. * * * ## **Project Timeline** - **Project Duration:** Approximately 1 week - **Expected Timeline:** July 8–14 - **Expected commitment:** at least 10 hours - Candidates must be located in **US/ Canada** due to the need for real-time collaboration with the project team. * * * ## **Background & Qualifications** **YOE** - 3+ years of professional software engineering experience (or equivalent open-source experience) **Technical Expertise** - **JavaScript / TypeScript:** Node.js, React, TypeScript, modern web application development - **Go:** Backend services, distributed systems, REST/gRPC APIs, cloud-native development (Kubernetes experience is a plus) - **Python:** Backend development, open-source libraries, DevOps/infrastructure automation, testing frameworks **Software Engineering** - Experience working with large production codebases - Ability to quickly understand unfamiliar repositories and software architectures - Experience reviewing code for readability, maintainability, and software design - Strong engineering judgment and a solid understanding of software engineering best practices **GitHub & Open Source** - Strong understanding of Git and GitHub workflows (e.g., branching, Issues, Pull Requests, and Code Reviews) - Experience navigating open-source repositories and collaborating through GitHub - Familiarity with open-source development workflows is highly preferred - Familiarity with SWE-Bench or SWE-Bench Pro is a plus **Communication** - Strong English communication skills * * * ## **Responsibilities** - Become familiar with assigned open-source codebases and their engineering practices - Review code changes, technical discussions, and related software artifacts - Evaluate AI-generated code for correctness, quality, maintainability, and engineering best practices - Provide structured evaluations with clear technical reasoning - Collaborate with fellow technical reviewers to ensure high-quality and consistent evaluations * * * **Note:** This is **not a software development role**. Rather than writing code, you'll leverage your software engineering expertise to evaluate AI-generated solutions and help improve the capabilities of state-of-the-art coding models.

Skills & domains

• ai-training
• rlhf
• sme
• annotation
• Software Engineering