Site Reliability Engineer, Inference Infrastructure

About the role

Cohere is seeking a Site Reliability Engineer to join its Model Serving team, responsible for delivering large language models via API endpoints. You will work on high‑performance, low‑latency, and highly available machine‑learning infrastructure that powers advanced NLP applications.

Key responsibilities

• Build self‑service systems that automate deployment and operation of language‑model services, including custom Kubernetes operators.
• Automate observability, resilience, and troubleshooting tools to help developers meet defined SLOs.
• Participate in on‑call rotation and take actions required to achieve service reliability targets.
• Collaborate with internal developers to influence infrastructure roadmap and share knowledge through reviews.

Required profile

• 5+ years of engineering experience operating production infrastructure at large scale.
• Proven ability to design highly available distributed systems on Kubernetes with GPU workloads.
• Strong collaboration and troubleshooting skills for mission‑critical systems.

Required skills

• Kubernetes (including custom operators)
• GPU/TPU workloads and accelerator performance characteristics
• Multi‑cloud platforms: GCP, Azure, AWS, OCI
• Linux‑based computing environments and resource‑cost management
• Distributed systems design
• Programming in Golang or C++ for high‑performance servers

What we offer

• Weekly lunch stipend of $75 (or local equivalent)
• Full health, dental and mental‑health benefits
• RRSP matching, 401K, pension scheme and parental‑leave top‑up
• 6 weeks paid vacation and travel budget for office visits
• Home‑office stipend and co‑working benefit for remote employees