Staff+ Software Engineer, ML Sampling Path

About the role

Anthropic is looking for a Staff+ Software Engineer to own the ML Sampling Path that powers Claude’s safety systems. You will design, build, and operate the backend services that process every token on the generation path, ensuring low latency and high reliability at global scale.

Key responsibilities

• Design, implement, and operate backend systems that handle token generation for Claude requests, including the streaming contract with the API and inference engines.
• Define and maintain SLOs and error budgets for latency, time‑to‑first‑token, and availability; lead incident response and post‑mortem analysis.
• Ship rapid yet safe changes to the hot path using canary and gradual rollouts, latency gating, and fast rollbacks while keeping per‑token cost flat.
• Set technical direction for the sampling path, lead design reviews, make trade‑off decisions with inference and research teams, and mentor engineers.

Required profile

• 8+ years of industry software engineering experience building high‑QPS, globally‑scaled systems.
• Proven track record of operating production services, handling incidents, and driving post‑mortem remediation.
• Deep understanding of distributed systems, replication, consistency trade‑offs, failure modes, and SLO management.
• Experience designing systems for graceful degradation and predictable failure handling.

Required skills

• High‑throughput backend engineering.
• Distributed systems design.
• Latency and reliability engineering.
• Familiarity with LLM inference and transformer‑based models (a plus).

What we offer

• Competitive annual compensation ranging from $320,000 to $485,000 USD.
• Opportunity to work on mission‑critical AI safety infrastructure at a fast‑growing AI research company.
• Collaborative environment with leading researchers and engineers.