What does 99.9% uptime mean for inference?

Every inference provider slaps an uptime number on their pricing page, and most customers nod along without asking what it actually covers. Together AI's latest breakdown argues that's backwards: a 99% SLA and a 99.99% SLA aren't the same promise with more decimals, they're entirely different failure domains requiring entirely different architecture.

The company lays out three tiers. Getting to 99% means surviving a single node dying — a GPU fault, a driver crash, a thermal spike — which requires fast health checks and quick rescheduling within one data center. But that ceiling is the building itself: lose power or cooling at that facility and no amount of in-DC redundancy saves you. Getting to 99.9% means surviving a full data center outage, which Together says only works if a provider runs live traffic across two facilities simultaneously, not a cold standby that's never been tested under real load. And 99.99% means surviving an entire region going dark, which demands reserved idle capacity sitting in a failover region right now, not just the theoretical ability to route there.

The more interesting argument is about who actually owns the hardware. Together runs inference for Cursor, Decagon, Cartesia, and Yutori, and claims that when something breaks at 3 a.m., providers renting GPU capacity from a hyperscaler or neocloud can't fix the problem themselves — they file a ticket and wait in a queue, then that provider files their own ticket upstream. Together says owning the stack top to bottom, what it calls chip-to-token visibility, collapses that into one ticket instead of two queued ones.

There's also a quieter point buried in the technical detail: GPU failures don't always look like failures. An ECC error in VRAM can silently corrupt weights while the server keeps answering requests, just with untrustworthy output. A thermal event can degrade quality long before triggering an alert. Together's pitch is that spotting the real signal inside a misleading symptom is the actual reliability work, and it's a lot less glamorous than the nines on a marketing page suggest.

The piece ends with a list of questions worth asking any inference vendor, and the questions are pointed on purpose: does your SLA get measured at the load balancer or at successful completion, do client retries get counted against uptime, is failover tested live or just on paper. Together is obviously selling its own infrastructure here, but the underlying advice holds regardless of vendor — an uptime number without an architecture explanation behind it is just marketing copy.