Amazon SageMaker HyperPod inference experienced long pod cold-start times because each new pod had to pull the vLLM/LMI container image from Amazon ECR and download model weights from the configured remote source before it could serve requests. For a 600+ GB model such as DeepSeek-R1, the article says this can take 30 minutes or more for a single request and repeats on every scale-out event. Amazon launched model caching that pre-loads weights to node NVMe (about 7 GB/s) and can pre-pull images so pods typically start serving in seconds instead of tens of minutes, with fallbacks to normal downloading when cache-ready capacity isn’t available.
OpenAI temporarily disabled new sign-ups for its $200-per-month Pro subscriptions because demand for its new model Astra is straining its infrastructure. The pause starts while Astra, launched on September 3, rolls out across Pro, Plus, Enterprise, and Business. OpenAI says API access and lower-cost Go and Plus plans remain available, with priority given to existing users.
Cloudera and Mistral announced a partnership to let enterprises run Mistral AI models on Cloudera’s hybrid data platform while keeping data and model control inside their own environments. The deal targets Cloudera’s 30 exabytes of customer-managed data for sovereign AI deployment. This expands on-prem, private/public cloud, and fully air-gapped inference options and enables enterprises to train and own custom models on proprietary data.
LandingAI released Agentic Document Extraction (ADE) Gen2, rebuilding its document intelligence stack around the DPT-3 model family and Gen2’s tree-based document output.
Together’s kernels team gained access to the NVIDIA Vera Rubin NVL72 platform and updated ThunderKittens to run NVFP4 and FP8 GEMMs on it by adding Rubin-specific ISA and memory/pipeline features. The NVFP4 GEMM kernels improve from about 42.1% of the roofline to over 22 PFLOPS. These changes increase attainable performance by doubling the K step, expanding tensor memory and shared memory use, and adjusting tiling and commit/collector behaviors so tensor cores stay fed.
Together announced public preview of preemptible GPU compute for Together GPU Clusters on Kubernetes, where interruptible nodes can be reclaimed from unused capacity.
Every AI story that matters,
in your inbox by 8am.
TLDRocket reads all relevant sources, removes duplicate coverage, and summarises the
day in two minutes. Follow companies and topics for alerts, or get the
briefing in Slack. Free, no spam, unsubscribe anytime.
Reading TLDRocket needs no cookies, and the readership counts we rely on come from
our own cookieless analytics. Google Analytics is the exception: it sets cookies and
reports to Google, so it stays switched off until you allow it. You can change your
mind any time from “Cookie settings” in the footer.
Strictly necessary
Session security and form protection (tldrocket-session,
XSRF-TOKEN, 2 hours). The site cannot work without them,
so they need no consent.
Always on
Google Analytics 4 (_ga,
_ga_<id>, up to 2 years). Measures which
stories and sections readers use. Google acts as a third-party processor and may
store the data outside the EU. No advertising, no profiling, no data sold.
