Technology
Senior Software Engineer, Model Evaluation
Clera
Ashby
- Location
- San Francisco, United States
- Type
- FullTime
- Salary
- Salary not specified
- Posted
- 19 Nov 2026
ABOUT THE ROLE Join an early-stage AI team building agents for manufacturing operations. You will create benchmarks and train models against real procurement and supply-chain workflows, helping ensure agents can handle customer-facing tasks reliably. WHAT YOU'LL DO - Design and build evaluation suites for supplier sourcing, qualification, compliance documents, purchase orders, freight, and order tracking. - Work with procurement and manufacturing experts to scope tasks, write reference answers and rubrics, and build graders. - Train and fine-tune models, using evaluation results to guide experiments. - Identify agent failures, improve production systems, and measure whether changes work. - Help establish the team's research and evaluation practices. WHAT WE'RE LOOKING FOR - At least 3 years of experience, operating at a senior or staff level. - Hands-on experience designing benchmarks and evaluations, including authoring tasks, reference solutions or rubrics, and quality checks. - Practical experience training and fine-tuning models, not only using models through APIs. - Ability to work closely with non-ML domain experts and translate their knowledge into measurable tasks. - Supply
ABOUT THE ROLE Join an early-stage AI team building agents for manufacturing operations. You will create benchmarks and train models against real procurement and supply-chain workflows, helping ensure agents can handle customer-facing tasks reliably. WHAT YOU'LL DO - Design and build evaluation suites for supplier sourcing, qualification, compliance documents, purchase orders, freight, and order tracking. - Work with procurement and manufacturing experts to scope tasks, write reference answers and rubrics, and build graders. - Train and fine-tune models, using evaluation results to guide experiments. - Identify agent failures, improve production systems, and measure whether changes work. - Help establish the team's research and evaluation practices. WHAT WE'RE LOOKING FOR - At least 3 years of experience, operating at a senior or staff level. - Hands-on experience designing benchmarks and evaluations, including authoring tasks, reference solutions or rubrics, and quality checks. - Practical experience training and fine-tuning models, not only using models through APIs. - Ability to work closely with non-ML domain experts and translate their knowledge into measurable tasks. - Supply-chain, procurement, manufacturing, or early-stage startup experience is helpful. COMPENSATION & BENEFITS Base salary is $180,000 to $250,000 annually, plus equity. Visa sponsorship details were not provided. LOCATION Hybrid in San Francisco, California, United States.