What's the tech stack?

Joblaze extracted these technologies from the posting: Cadence, Go, Kafka, Kubernetes, NATS, Python.

What seniority level is this role?

Together AI targets staff-level candidates for this position.

Staff Software Engineer, Inference / Compute Infrastructure Engineering London or Amsterdam at Together AI

Joblaze summary

In this role, the Staff Software Engineer focuses on developing systems that automate the lifecycle management of hardware for AI inference clusters, enabling teams to provision resources with a single API call. Key skills include proficiency in programming languages like Go or Python, experience with durable workflow orchestration, and a strong understanding of event-driven architectures. This position is ideal for seasoned engineers with a product mindset who have previously built internal platforms or APIs for other engineering teams. Together AI, a research-driven company, emphasizes innovation in AI infrastructure and values transparency in its operations.

Joblaze insights

Listed about a month ago — first seen on Joblaze August 20, 2026. Last confirmed on Together AI's careers page October 8, 2026.
Python appears in 45.6% of 472 comparable staff ai/ml roles; SQS appears in 0.6% of 472 comparable staff ai/ml roles.

Quick facts

What's the tech stack?: Joblaze extracted these technologies from the posting: Cadence, Go, Kafka, Kubernetes, NATS, Python.
What seniority level is this role?: Together AI targets staff-level candidates for this position.
Is this full-time or contract?: Full-time for this Staff Software Engineer, Inference / Compute Infrastructure Engineering London or Amsterdam role at Together AI.

From the original posting

About the Role

We're looking for a software engineer to build the Kubernetes-native control plane that provisions and runs our GPU inference fleet. You'll design a manifest-driven API where the inference team declares what they need, whether that's a cluster, a model deployment, or a capacity change, and our controllers handle the reconciliation, provider/runtime selection, and lifecycle management underneath, so the inference team never has to know or care which specific serving stack, scheduler, or hardware pool is doing the work. You'll also build the systems that keep the fleet efficient, not just running, including defragmentation and rebalancing logic that consolidates scattered workloads back into contiguous capacity, and scheduling/bin-packing improvements that push GPU utilization up without hurting latency. The core value we're after is decoupling the people building on top of the platform from the operational and runtime complexity underneath, while squeezing more usable capacity out of the same hardware. You'll build the controllers, reconciliation loops, and self-service surface (API/CLI, not tickets) that make that decoupling real, plus the event-driven health, remediation, and utilization systems that keep it running and efficient without a human in the loop. Strong candidates have hands-on experience with Kubernetes controller/CRD patterns, have built or operated a platform API that abstracts multiple backends behind one interface, understand GPU scheduling and capacity efficiency (fragmentation, bin-packing, right-sizing), and think about GPU infrastructure as software to be engineered.

A product mindset - you've built internal platforms or APIs consumed by other engineering teams and care about the developer experience of what you ship.You build it, you own it. You are not only responsible for delivering the software but also for operating and supporting it in production.

Responsibilities

Build the provisioning state machine: design and implement the software that models the full lifecycle of a physical host from discovery, inference bring-up to GPU driver/CUDA stack, health validation, and decommission/RMA — as explicit, versioned states and transitions.
Build the self-service API: design declarative APIs and a control plane so the inference team can request, scale, and tear down inference clusters with one API call — no ticket, no human in the loop.
Automate self-healing: detect degraded or failed nodes, drain them safely, trigger repair or replacement, and reintroduce healthy capacity into the pool automatically.
Own reliability of the pipeline: idempotency, retries, rollback, and drift detection so the provisioning system is as dependable as any other production service.
Partner with the inference/ML platform team: understand the cluster shapes they need — topology, interconnect, scheduling constraints — and encode them as first-class abstractions in the platform.
Engineer it like software: strong typing, automated tests, code review, versioning, and CI/CD for infrastructure code — this is a product, not a collection of Ansible playbooks.

Requirements

Core requirements (all levels):

Strong software engineering background in Go, Python, Rust, or similar — you write and test real software for a living.
Experience with durable workflow orchestration tools such as Temporal, Cadence, or equivalent to run long-lived, manifest-driven workflows that survive failures and resume mid-execution.
Experience building software control planes or orchestration systems that model state and reconcile it over time (e.g., Kubernetes controllers/operators, custom reconciliation loops, workflow engines).
Experience with event-driven systems — designing and building software around message queues, event streams, or pub/sub (e.g., Kafka, NATS, SQS) rather than polling or cron-driven scripts.
A product mindset. You’ve built internal platforms or APIs consumed by other engineering teams and care about the developer experience of what you ship.

Nice to have:

Exposure to bare-metal provisioning (PXE/iPXE, Redfish/IPMI, BMC) and/or networking fundamentals (VLANs, BGP, fabric design), or GPU/accelerator infrastructure.
Experience with GPU cluster software stacks (NCCL, CUDA, InfiniBand/RoCE).
Prior work at a hyperscaler, GPU cloud, or datacenter-scale infrastructure organization.
Systems programming in Rust or Go.

Please see our privacy policy at https://www.together.ai/privacy.

Standard company text repeated across Together AI's postings is omitted here.

Staff Software Engineer, Inference / Compute Infrastructure Engineering London or Amsterdam

About the Role

Responsibilities

Requirements

Similar positions