Google is developing a specialized chip called Frozen v2 designed specifically for its Gemini AI model, which would hardwire parts of Gemini's architecture while keeping weights updatable. The chip is projected to deliver six to ten times more tokens per watt compared to Google's current AI chips. If successful, this approach could significantly reduce inference costs for developers using Gemini while establishing a trend toward model-specific silicon rather than general-purpose accelerators.
Anthropic made Claude Fable 5 a permanent feature of Max and Team Premium subscriptions at 50% usage limits starting July 20, after weeks of temporary extensions while the company scaled inference capacity. Employees worked intense hours to bring additional compute infrastructure online to support broader access to the model. The move reflects how serving frontier AI models has become a key constraint in subscription pricing, with Anthropic also introducing local Indian rupee pricing for Claude to expand in its second-largest market.
Netflix built an in-house system to serve large language models using vLLM and NVIDIA Triton, integrating it into their existing JVM-based serving infrastructure rather than using external APIs. The platform supports both gRPC and OpenAI-compatible HTTP endpoints, with deployment strategies including red-black and versioned rollouts to handle model updates without dropping requests. The system implements constrained decoding via vLLM's logits processor interface to generate compliant outputs by construction, though this required optimization across vLLM versions to handle batching efficiently at scale.
Every AI story that matters,
in your inbox by 8am.
TLDRocket reads all relevant sources, removes duplicate coverage, and summarises the
day in two minutes. Follow companies and topics for alerts, or get the
briefing in Slack. Free, no spam, unsubscribe anytime.
Reading TLDRocket needs no cookies, and the readership counts we rely on come from
our own cookieless analytics. Google Analytics is the exception: it sets cookies and
reports to Google, so it stays switched off until you allow it. You can change your
mind any time from “Cookie settings” in the footer.
Strictly necessary
Session security and form protection (tldrocket-session,
XSRF-TOKEN, 2 hours). The site cannot work without them,
so they need no consent.
Always on
Google Analytics 4 (_ga,
_ga_<id>, up to 2 years). Measures which
stories and sections readers use. Google acts as a third-party processor and may
store the data outside the EU. No advertising, no profiling, no data sold.
