This job is no longer available
This job expired on 27/10/2026. It no longer accepts applications.
Directeur.rice, Évaluations
LoiZéro · Montréal
Job description
About the role
LoiZéro, a nonprofit dedicated to designing safe‑by‑design AI systems, is seeking a Director of Evaluations to build and lead its Evaluation team. This founding position will define world‑class evaluation practices for the organization’s flagship Scientist AI and ensure independent verification of model capabilities and safety.
Key responsibilities
- Define the evaluation strategy and roadmap, prioritising metrics in close collaboration with research and product teams.
- Recruit and grow an Evaluation team of 8‑10 specialists in research, engineering, dataset design, benchmarking and red‑teaming within the first 3‑6 months.
- Maintain independence from core research and product streams to avoid conflicts of interest.
- Design novel benchmarks that enable fair, apples‑to‑apples comparison between Scientist AI and frontier LLMs.
- Oversee creation of new datasets, tasks and virtual environments to assess performance, safety, causality, and resistance to adversarial attacks.
- Lead evaluation of Scientist AI when deployed as a guardrail, ensuring compliance with harm‑prevention specifications, explainability, and robustness against jailbreaks, prompt injection and data poisoning.
- Establish and manage automated and manual red‑team programmes, both internal and with external partners.
- Drive development of internal tools and infrastructure for large‑scale evaluation.
Required profile
- Proven experience leading evaluation or safety teams in AI/ML contexts.
- Strong track record of designing benchmarks, datasets and red‑team exercises.
- Ability to operate independently from product and research agendas.
- Excellent strategic thinking and team‑building skills.
Required skills
What we offer
- Founding leadership role in a mission‑driven nonprofit.
- Opportunity to shape industry‑leading AI safety standards.
- Collaborative environment with top AI researchers.
Questions fréquentes
Why are you reporting this job?
Explore further
Salaries, guides and searches in Canada.
Salaries by job title
A question about this job?
Ask it here: you will get the full job summary by e-mail, right away.
Boost your chances
Upload your CV — we will match you with relevant openings.
Analyzing your CV...
LoiZéro
Montréal
Related job offers
-
Technical Artist (Game Development)
People Can Fly Montréal -
Senior AI Programmer
People Can Fly Montréal -
IT Support Manager
People Can Fly Montréal -
Analyste en assurance qualité (Maintenance applicative) – Hybride
ISAAC Instruments Saint-Bruno-de-Montarville -
Quality Assurance Analyst (Application Maintenance) - Hybrid
ISAAC Instruments Saint-Bruno-de-Montarville