CoreWeave is preparing a user conference called Fully Connected while describing services and infrastructure aimed at cutting the cost and effort needed to deploy AI at production scale. CoreWeave says its autonomous improvement for AI agents can reduce costs by over 40% and speed up training by about 1.4 times without loss in quality. The event coverage and interviews are expected to focus on how CoreWeave and its partners tackle operational challenges like validation, reliability, and observability for increasingly complex AI models.
PrismML launched Bonsai 2 27B, a second-generation ultra-compact multimodal AI model designed to run on PCs and some high-end mobile devices. It reduces a Qwen3.8 27B model from about 56 GB in 16-bit uncompressed form to about 5.9 GB while retaining about 98.2% of capabilities. Model users can now run this AI locally with lower memory, compute, and power needs, with weights released under Apache 2.0.
Kubernetes-focused updates are expanding to support production AI inference while infrastructure vendors and users debate whether Kubernetes can accurately account for inference costs. The article cites a measured 60% reduction in processing the cost for 1 million tokens for China Merchants Bank’s Kubernetes-based setup. It pushes teams toward evolving scheduling and memory/serving layers (and better telemetry and platform readiness) so token-level economics can drive inference decisions.
Hugging Face’s developers showed that deploying Hugging Face models on Amazon SageMaker AI can fail when coding agents choose stale or wrong serving containers, while their provided “agent skills” guide the workflow to a working endpoint. A Qwen3-0.6B deployment to a single ml.g5.xlarge instance in us-east-1 failed health checks multiple times when the agent initially picked TGI before switching to vLLM. Using the skills changes the process by selecting the correct container image URI from the AWS Deep Learning Containers catalog, adding autoscaling and CloudWatch alarms, and providing a verified teardown path to avoid billed GPU time from failed endpoints.
Amazon announced SageMaker HyperPod Inference Gateway, a Kubernetes-native GPU-aware routing addon for EKS that replaces naive load balancer routing for LLM inference. It reports reducing first-token latency by up to 82% versus Kubernetes round-robin, including a benchmark where TTFT P99 improved by 98% in bursty traffic for Llama-3.1-70B. The change is that inference requests get steered to the best-suited model pod using real-time GPU signals and routing rules (including KV cache state, queue depth, and LoRA adapter residency), rather than round-robin balancing.
The article says AI agents can review pull requests quickly in testing but slower in production because the execution environment changes while the agent logic stays the same. It points to p99 latency increasing first when infrastructure is provisioned for average load rather than the bursty peaks created when tool results return. As a result, teams need agent-specific infrastructure that handles long step chains reliably and scales for bursty, sequential workflows to keep latency and costs predictable.
Crusoe announced an initial closing of its $3.9 billion Series F funding round to expand its vertically integrated AI infrastructure and AI-factory buildout. The round values Crusoe at a $30.9 billion post-money valuation. The new capital will scale its programs and support buildout of its own AI factories, including large campuses and modular Crusoe Spark units, to accelerate Crusoe Cloud growth.
Crusoe announced a $3.9 billion Series F funding round, raising its post-money valuation to $30.9 billion. The deal follows a roughly year-earlier Series E that valued the company at $10 billion, implying it tripled value in 10 months. The new capital will support projects like Abilene and expand Crusoe’s AI data-center and cloud offerings, with further board changes and potential IPO discussions.
A global fintech scaled its AI coding assistant traffic by running GLM 5.2 on Together’s Dedicated Model Inference to handle spiky, engineering-hours demand. It reported p50/p90/p95 usage at about 81K/163K/178K tokens and RPS at 3/6/7, and a specific incident involved 192-second requests queued behind a 2.3M-token pending-prefill backlog. Engineers gained self-service endpoint control with a metrics API and live configuration/model updates, replacing ticket-based coordination and reducing multi-minute queuing.
Every AI story that matters,
in your inbox by 8am.
TLDRocket reads all relevant sources, removes duplicate coverage, and summarises the
day in two minutes. Follow companies and topics for alerts, or get the
briefing in Slack. Free, no spam, unsubscribe anytime.
Reading TLDRocket needs no cookies, and the readership counts we rely on come from
our own cookieless analytics. Google Analytics is the exception: it sets cookies and
reports to Google, so it stays switched off until you allow it. You can change your
mind any time from “Cookie settings” in the footer.
Strictly necessary
Session security and form protection (tldrocket-session,
XSRF-TOKEN, 2 hours). The site cannot work without them,
so they need no consent.
Always on
Google Analytics 4 (_ga,
_ga_<id>, up to 2 years). Measures which
stories and sections readers use. Google acts as a third-party processor and may
store the data outside the EU. No advertising, no profiling, no data sold.
