Listing note. Listings come from public job feeds and can change or expire—check the original source and eligibility requirements before applying.

All listings

Technology

Research Engineer, Benchmarks

Clera

AshbyVisa / relocation
WhatsApp
Location
Singapore, Singapore
Type
FullTime
Salary
Salary not specified
Posted
18 Nov 2026
ABOUT THE ROLE Join a small, technical team building benchmarks to evaluate frontier AI agents on realistic, domain-specific workflows. You will own evaluation design and implementation, helping ensure benchmark results are rigorous, credible, and relevant to real-world performance. WHAT YOU'LL DO - Design, implement, and maintain benchmarks for evaluating frontier agents on domain-specific tasks. - Work with subject-matter experts to translate real workflows into realistic tasks and evaluation criteria. - Build reliable infrastructure for running models and agents against benchmark tasks at scale. - Develop metrics and analyses to assess benchmark difficulty, reliability, and failure modes. - Validate whether benchmark results correlate with real-world evaluations and user needs. - Write clear documentation and reports for technical audiences. WHAT WE'RE LOOKING FOR - Two or more years of experience in research engineering or machine learning engineering, building AI benchmarks, evaluation infrastructure, or agent environments. - Technical education or equivalent grounding, with strong first-principles reasoning about task design, scoring, and failure modes. - Proficiency in Pytho
ABOUT THE ROLE Join a small, technical team building benchmarks to evaluate frontier AI agents on realistic, domain-specific workflows. You will own evaluation design and implementation, helping ensure benchmark results are rigorous, credible, and relevant to real-world performance. WHAT YOU'LL DO - Design, implement, and maintain benchmarks for evaluating frontier agents on domain-specific tasks. - Work with subject-matter experts to translate real workflows into realistic tasks and evaluation criteria. - Build reliable infrastructure for running models and agents against benchmark tasks at scale. - Develop metrics and analyses to assess benchmark difficulty, reliability, and failure modes. - Validate whether benchmark results correlate with real-world evaluations and user needs. - Write clear documentation and reports for technical audiences. WHAT WE'RE LOOKING FOR - Two or more years of experience in research engineering or machine learning engineering, building AI benchmarks, evaluation infrastructure, or agent environments. - Technical education or equivalent grounding, with strong first-principles reasoning about task design, scoring, and failure modes. - Proficiency in Python, Docker, and Linux, including experience operating infrastructure for AI models or agents. - Hands-on experience designing and running evaluations for AI agents or large language models, with relevant, focused experience in benchmarks or evaluation. - Experience developing metrics, statistical analyses, or validation studies, and translating workflows across domains into benchmark tasks. - Strong technical writing and communication skills. Published papers or technical writing on benchmarking, evaluation, or model failure modes are valued. - Experience with reinforcement learning pipelines, data generation, or agent evaluation is a plus. - Comfort working independently in an early-stage, fast-paced environment and navigating open-ended problems. COMPENSATION & BENEFITS Base salary: $100,000 to $170,000 annually. Visa sponsorship is available. LOCATION On-site in Singapore, Singapore.