Software Engineer, GPU Inference

About the role

Cerebras is building a new generation of disaggregated AI inference systems that combine GPU‑accelerated prefill with ultra‑fast decode on its Wafer‑Scale Engine. We are looking for a Software Engineer to productionize and optimise the GPU serving stack, ensuring reliability, numerical correctness, observability and top‑tier performance.

Key responsibilities

• Productionize and maintain the GPU inference stack, including API services, model‑serving workers, vLLM, PyTorch, ROCm and rack‑scale GPU infrastructure.
• Establish operational practices for the AMD GPU fleet such as deployment, upgrades, health‑checking, capacity management and automated recovery.
• Define and monitor service‑level indicators, improve fault isolation, graceful degradation and incident response for GPU‑backed inference.
• Profile and optimise time‑to‑first‑token, throughput, tail latency, GPU utilisation and memory efficiency under production workloads.
• Build validation and regression infrastructure to ensure numerical correctness and model quality across software and hardware releases.

Required profile

• 5+ years of software engineering experience with ownership of complex production systems.
• Hands‑on experience building or optimizing inference systems for large language or multimodal models on GPUs.
• Strong C++ and Python programming skills, including multithreading, concurrency and performance‑critical code.
• Experience with high‑performance model‑serving frameworks such as vLLM, Triton or similar.
• Deep understanding of GPU execution, memory movement, kernel launches and profiling methodologies.
• Proficiency with Linux, containers, Kubernetes (or comparable orchestration), CI/CD and latency‑sensitive services.

Required skills

• C++
• Python
• Multithreading and concurrency
• vLLM or equivalent model‑serving framework
• AMD ROCm ecosystem (HIP, RCCL, rocprofiler)
• Linux, containers and Kubernetes
• Performance profiling and GPU optimisation
• Distributed systems debugging