Amazon’s EKS Auto Mode and related platform components were instrumented end-to-end for GPU inference pod startup, revealing six sequential bottlenecks that add up to an 8-minute time-to-first-token response on a 70B-class model. For the 203 GB model, downloading weights from S3 took 423 seconds (about 92% of startup time), partly due to a pattern that left 98% of bandwidth idle. Configuration changes and platform features cut cold-start latency from 8–15 minutes to under 1 minute (with warm-node restarts dropping to under 30 seconds), mainly by fixing S3 weight loading, CUDA kernel compilation caching, and some node/image startup steps.
Nvidia launched Nvidia Personal AI Router (PAIR), an open source software router that lets idle home Macs and PCs run small local models for agent workflows using subagents. In Nvidia’s example, two RTX 5090 PCs with 32 GB RAM each sped up work with five subagents by about 1.6x. It changes agent execution by routing requests to eligible local machines behind a single interface rather than pooling GPUs or splitting one inference across computers.
ChatGPT, Grok, and Claude all suffered service issues at the same time, disrupting users across multiple features.
At around 11AM ET, ChatGPT began returning errors and its status page reported elevated errors across ChatGPT and Codex.
The outages are blocking logins and core chatbot use while also degrading functions like file uploads, voice mode, search, deep research, and image generation.
Every AI story that matters,
in your inbox by 8am.
TLDRocket reads all relevant sources, removes duplicate coverage, and summarises the
day in two minutes. Follow companies and topics for alerts, or get the
briefing in Slack. Free, no spam, unsubscribe anytime.
Reading TLDRocket needs no cookies, and the readership counts we rely on come from
our own cookieless analytics. Google Analytics is the exception: it sets cookies and
reports to Google, so it stays switched off until you allow it. You can change your
mind any time from “Cookie settings” in the footer.
Strictly necessary
Session security and form protection (tldrocket-session,
XSRF-TOKEN, 2 hours). The site cannot work without them,
so they need no consent.
Always on
Google Analytics 4 (_ga,
_ga_<id>, up to 2 years). Measures which
stories and sections readers use. Google acts as a third-party processor and may
store the data outside the EU. No advertising, no profiling, no data sold.
