Senior Research Scientist, Model Evaluation

About the role

Evaluation is critical to scaling intelligence. Cohere is looking for a Senior Research Scientist to create next‑generation evaluation methods and infrastructure that measure large language model (LLM) progress.

Key responsibilities

• Create ambitious new evaluation benchmarks that push the limits of our models.
• Work with cross‑functional teams to translate model feedback into trustworthy, repeatable evaluations.
• Conduct research to advance state‑of‑the‑art LLM evaluation methods, including training LLM judges and refining data synthesis pipelines.
• Build scalable, reusable tools for analyzing model performance.

Required profile

• Enjoy rapidly building prototypes that demonstrate LLM capabilities and developing resources to measure them.
• Spend extensive time reviewing complex data and LLM outputs to ensure high data quality.
• Be obsessive about rigorously measuring AI capabilities and aligning measurements with desired outcomes.
• Possess strong software engineering skills.

What we offer

• Weekly lunch stipend of $75 (or equivalent).
• Full health, dental, and mental‑health benefits.
• RRSP matching, 401K, and pension scheme.
• 100% parental leave top‑up for up to 6 months.
• Annual enrichment benefits for arts, culture, fitness, and workspace improvements.
• Education and learning stipend for conferences, courses, and coaching.
• Six weeks of paid vacation (30 working days).
• $500 home office stipend and travel budget for office visits.