Senior Software Engineer – GPU Infrastructure (HPC)

About the role

Cohere is a security‑first enterprise AI company building foundation models and end‑to‑end products for real‑world business problems. The internal infrastructure team creates world‑class tools to train, evaluate and serve these models, working closely with AI researchers on cutting‑edge workloads.

Key responsibilities

• Build and scale ML‑optimized HPC infrastructure by deploying and managing Kubernetes‑based GPU/TPU superclusters across multiple clouds.
• Optimize AI/ML training performance and cost by collaborating with cloud providers and leveraging RDMA, NCCL and high‑speed interconnects.
• Troubleshoot and resolve complex infrastructure bottlenecks, performance degradation and system failures.
• Design self‑service tools and intuitive interfaces that enable researchers to monitor, debug and optimise their training jobs.
• Drive innovation by translating emerging research needs (e.g., JAX, PyTorch, distributed training) into robust, scalable solutions.
• Champion observability, automation and infrastructure‑as‑code best practices across the organization.
• Mentor teammates through code reviews, documentation and cross‑team collaboration.

Required profile

• Deep expertise in ML/HPC infrastructure, including GPU/TPU clusters and distributed training frameworks.
• Proven experience operating Kubernetes at scale for AI workloads.
• Strong programming skills in Python and Go, with a preference for open‑source contributions.
• Low‑level systems knowledge of Linux internals, RDMA networking and performance optimisation.
• Track record of collaborating closely with AI researchers or ML engineers.
• Self‑directed problem‑solving ability in a fast‑paced environment.

Required skills

• Python
• Go
• Kubernetes
• GPU and TPU clusters
• RDMA
• NCCL
• JAX
• PyTorch
• TensorFlow
• Linux
• High‑speed interconnects

What we offer

• Weekly lunch stipend of $75 (or equivalent).
• Full health, dental and mental‑health benefits.
• RRSP matching, 401(k) and pension scheme.
• 100 % parental‑leave top‑up for up to six months.
• Annual enrichment budget for arts, culture, fitness and workspace improvements.
• Education stipend for conferences, courses and coaching.
• Six weeks of paid vacation and travel budget for office visits.
• Remote‑friendly work model with a $500 home‑office stipend.