NVIDIA announced CUDA Rust to let Rust be used for writing GPU kernel bodies, filling the gap where Rust could already launch CUDA kernels but not define the kernel code. The cutile-rs Tile track is already on crates.io and runs on stable Rust 1.89+ (requiring CUDA 13.3), while cuda-oxide is in early alpha with a pinned nightly. Both projects use Rust ownership/borrowing to reject buffer aliasing at compile time, with Tile currently more deployable than SIMT due to alpha status.
China’s exports rose 25% year-on-year in August and expanded its trade surplus to $119.1 billion as shipments of autos and high-tech goods—especially semiconductors—increased. Exports to the U.S. reached $42.5 billion in August, up 34.4% from a year earlier. The report links the trade strength to China moving up the value chain in AI infrastructure and industrial automation, while U.S. and other officials intensify focus on imbalances ahead of late-September U.S.-China talks.
Amazon SageMaker AI benchmarked and recommended GPU instance configurations for serving two 30B-class Mixture-of-Experts LLMs on SageMaker Inference, comparing G5, G6, and G7 (Blackwell) options. In the Qwen3-Coder-30B test on the ml.g7.12xlarge endpoint, output token throughput reached 391.3 tok/s with an average request latency of 1,315.8 ms. The results led the article to conclude that G7 can deliver better price-performance, so users can either run direct endpoint benchmarks or use SageMaker AI generative AI inference recommendations to pick a deployment configuration.
Dell Technologies reported a major earnings surge tied to record AI server demand and positioning it to support companies running AI on their own infrastructure. It booked $60.9 billion in AI server orders in a single quarter. The company raised its full-year outlook and its results point to accelerating enterprise shifts toward on-premise AI, boosting growth across servers and storage and expanding margins.
llm-d, an open-source inference framework led by IBM Research, Red Hat, and Google, demonstrated efficient serving of agentic LLM workloads on existing enterprise GPUs by avoiding redundant context processing and scaling different inference stages separately. It deployed GLM-5.2, about 753B parameters (39B active), on 544 NVIDIA H100 GPUs and reached over 6.6 million output tokens per minute at peak with up to 3,000 concurrent coding agents and zero preemptions. The result is that operators can self-host an open-weight frontier-scale mixture-of-experts model for long-context agents on H100 fleets with software changes that improve latency/throughput and cut per-token costs versus equivalent commercial API pricing.
The talk “From S3 to GPU in One Copy” explains how SpiralDB’s Vortex columnar format enables training data to be streamed from S3 into GPUs with projection and filter pruning instead of decompressing/reprocessing everything. It reports S3-to-GPU scans at around 30 times faster than Parquet. As a result, only selected columns/segments are read and decompressed for each iteration, reducing movement and decision overhead during ML training.
vLLM explains how speculative decoding in its serving stack uses a draft component to propose multiple tokens and a target-model pass to verify them before committing accepted outputs. The post notes that standard autoregressive decoding needs 4 sequential decode steps to produce 4 output tokens, while speculative decoding can reduce target-model verification rounds when candidates are accepted. It changes generation by committing multiple tokens per target verification step and discarding later candidates after the first rejection, with observed output-token throughput varying by drafting method, proposal length, and AMD MI300X/MI355X workloads.
Carmen Alfonso Rico joined UK AI chip startup Fractile as VP of business operations and said it could become Britain’s first one trillion dollar company. Fractile is raising $600m at a $5.6bn valuation, according to Bloomberg. The company is positioning itself to compete with Nvidia in AI inference hardware, with specialized chips targeted for data centre deployment in 2027.
Every AI story that matters,
in your inbox by 8am.
TLDRocket reads all relevant sources, removes duplicate coverage, and summarises the
day in two minutes. Follow companies and topics for alerts, or get the
briefing in Slack. Free, no spam, unsubscribe anytime.
Reading TLDRocket needs no cookies, and the readership counts we rely on come from
our own cookieless analytics. Google Analytics is the exception: it sets cookies and
reports to Google, so it stays switched off until you allow it. You can change your
mind any time from “Cookie settings” in the footer.
Strictly necessary
Session security and form protection (tldrocket-session,
XSRF-TOKEN, 2 hours). The site cannot work without them,
so they need no consent.
Always on
Google Analytics 4 (_ga,
_ga_<id>, up to 2 years). Measures which
stories and sections readers use. Google acts as a third-party processor and may
store the data outside the EU. No advertising, no profiling, no data sold.
