TLDRocket
Sign in

AI Infrastructure

1728 summarised stories about AI Infrastructure, each linking back to the original source. Browse all topics →

+ Follow this topic

Thursday, 10 September 2026

Meet Redis LangCache: A Managed Semantic Cache That Cuts LLM API Costs by Up to 90% and Returns Cache Hits Up to 15x Faster

MarkTechPost 1 month ago 39

Redis launched LangCache, a managed semantic caching service that matches prompts by meaning and returns stored LLM responses without making a new model call on a cache hit. LangCache reports up to 90% lower LLM API costs and cache-hit responses up to 15x faster, with a demo showing 0.37 seconds vs 2.232 seconds for a paraphrased query. Applications can replace per-request LLM calls with a REST two-step loop (search before calling, store after misses), changing latency and spending by shifting cost mainly to avoided output tokens while requiring tuned thresholds and TTL/monitoring to prevent incorrect matches.

NVIDIA Details BioNeMo Inference Runtime (BioIR): 2.90x Higher Boltz-2 Folding Throughput and 58.5K Residues per GPU-Hour on 8xH100

MarkTechPost 1 month ago 15

NVIDIA released BioNeMo Inference Runtime (BioIR), a Python library that speeds up supported protein-structure prediction inference on NVIDIA GPUs while keeping the standard PyTorch workflow. In a matched benchmark on 8xH100 for 1,000 human dimers, BioIR delivered 58.5K successfully folded residues per GPU-hour versus 20.2K for an open-source torch-compiled Boltz-2 implementation. The throughput improves by 2.90x on this dataset and hardware, and NVIDIA estimates energy for 1 million comparable targets drops from 35 MWh to 11 MWh (8-GPU TDP equivalent).

Jensen Huang explains why Nvidia will grow an astounding 70% next year

TechCrunch 1 month ago 21

Nvidia CEO Jensen Huang said Nvidia expects its AI-related revenue growth to continue through next year despite rising GPU competition. He reiterated a forecast of 70% year-over-year revenue growth for next year. Nvidia’s guidance implies the company is positioning for sustained demand across AI labs and systems, including GPUs and data-center infrastructure, rather than expecting a slowdown.

Video and image search in Amazon Bedrock Knowledge Base using Marengo 3.0

Amazon Web Services 1 month ago 13

Amazon Bedrock Knowledge Bases added TwelveLabs Marengo Embed 3.0 as an embedding model for video and image semantic search. It encodes content into a 512-dimensional vector space, and the walkthrough uses a 10-minute clip of the 2022 FIFA World Cup final. Teams can now create and sync managed multimodal knowledge bases that ingest MP4/MOV and JPEG/PNG and return ranked natural-language search results with segment timestamps instead of managing a multi-step pipeline themselves.

OpenAI puts Pro subscriptions on hold due to Astra demand

TechCrunch 1 month ago 49 ● 3 sources

OpenAI temporarily disabled new sign-ups for its $200-per-month Pro subscriptions because demand for its new model Astra is straining its infrastructure. The pause starts while Astra, launched on September 3, rolls out across Pro, Plus, Enterprise, and Business. OpenAI says API access and lower-cost Go and Plus plans remain available, with priority given to existing users.

Sakana AI, SCSK, and Sumitomo Corporation Sign Comprehensive Business Partnership to Accelerate AI-Driven Social Implementation for Japan’s Industrial Transformation and Social Problem Solving

Sakana AI 20

Sakana AI, SC SK, and Sumitomo Corporation agreed to a comprehensive business partnership to support Japanese industrial transformation and solve social issues via AI use. The partnership will start with real enterprise cases that require strict reliability, quality, and information security, covering AI application PoCs and production-ready design, implementation, and operations. The three companies will jointly push customers from problem definition through technical evaluation, real-world deployment, impact verification, and continuous improvement, and will also explore an AI-orchestration-model-based service for vulnerability diagnosis to fixes in cybersecurity.

Mistral wants open-weight AI to compete at the frontier. It just raised $3.5 billion to do it.

The New Stack 1 month ago 20 ● 10 sources

Mistral announced it raised a €3 billion Series D to expand open-weight frontier AI research, training compute, and infrastructure. The funding round adds $3.5 billion (US dollars) and values the company at over €21 billion post-money. As a result, Mistral says it will keep building the full stack—models plus compute and production tools—to reduce customer dependence on a single vendor.

Amazon Quick is now generally available on desktop

Amazon Web Services 1 month ago 49 ● 3 sources

Amazon Quick desktop application is now generally available on macOS and Windows for enterprise use. It launched generally available on desktop “today.” The update adds a consolidated mobile activity feed on iOS and Android that merges email, calendar, CRM, and messaging so agents handle routine items in the background.

Red Hat AI 3.5 tackles the GPU queue that can stall AI pilots

The New Stack 1 month ago 49

Red Hat released Red Hat AI 3.5, expanding its platform for multi-tenant AI workloads with tenant isolation and priority-aware scheduling. The release adds a priority decision for every GPU request, plus new fairness scheduling and admission control for shared GPU inference. As a result, teams can move AI pilots into a governed enterprise architecture with pre-deployment safety benchmarking, real-time observability, and per-user token showback, reducing stalls from the GPU queue.

OpenAI Astra Changes Everything, But Not How You Think

YouTube 1 month ago 16 ● 19 sources

Corey and Grant discuss what changes when the AI model is no longer the clearest bottleneck, covering OpenAI Astra and related techniques and systems. They spend 90+ minutes on the topic. The focus shifts to factors like reasoning depth, agent behavior, cyber capabilities, and the supporting hardware and tools rather than just model quality.

Arlequin AI raises €28M to build novel AI models that learn complex relationships at scale

SiliconANGLE 1 month ago 20 ● 3 sources

Arlequin AI SAS announced it raised €28 million to develop a new AI model architecture built on topological neural networks rather than the graph-based networks used in most large language models. The Series A round totaled €28 million (about $32 million) and was exclusively backed by European investors, co-led by Redalpine and OTB Ventures. The funding will support scaling its platform and proprietary model, expanding its team, and accelerating commercial deployment across Europe and worldwide.

Skild AI Taps NVIDIA Physical AI to Teach Robots New Tasks From a Single Video

NVIDIA Blog 1 month ago 45

Skild AI launched its S1 robot foundation model that uses a single video demonstration to learn and perform new long-horizon tasks without updating model weights. It reports up to 10-minute unfamiliar task execution and an about 66% step success rate versus 9% for a similar system. As a result, robots can be adapted to changing factory work with in-context learning rather than retraining and post-training for each new task.

French Investors Value China’s Moonshot AI, Maker of Kimi K3, at $30 Billion

Trending Topics 1 month ago 7 ● 2 sources

French investors via a pooled SPV bought into Moonshot AI, the China-based company behind the Kimi model family, ahead of its planned Hong Kong listing. The SPV put about €10 million into the last financing round, valuing Moonshot AI at a $30 billion pre-money level. This shifts capital toward open-weight model hosting in Europe and supports efforts to self-deploy Moonshot’s Kimi models with privacy-focused technologies.

Physical AI Takes the Wheel: How the World’s Robotaxi Leaders Are Building With NVIDIA Technologies

NVIDIA Blog 1 month ago 43

NVIDIA says robotaxi leaders are building and scaling commercial driverless fleets using its end-to-end platform for AI training, simulation/validation, and in-vehicle computing. Uber plans to reach 28 cities by 2028 while using NVIDIA DRIVE Hyperion. As more programs adopt NVIDIA’s three-computer architecture, robotaxi deployment shifts toward standardized compute, simulation, and safety validation workflows across fleets at scale.

How AvioBook builds turnaround insights from operational data with Amazon Bedrock AgentCore

Amazon Web Services 1 month ago 42

AvioBook used Amazon Bedrock AgentCore to prototype “Connected Analytics,” turning turnaround “flightroom” operational records into plain-language, evidence-backed answers for airline managers and OCC dispatchers. The article cites that reducing average turnaround by 2 minutes can be worth about $240,000 a month for a mid-size carrier flying 200 flights a day. The system changes operations from scrolling and manual cross-referencing toward role-based multi-agent querying plus delay code validation that flags inconsistent codes with evidence for human review.

Midlands SMEs outpace UK on profitability, Sage data shows

Startups Magazine 14

Sage reported that Midlands SMEs are growing faster than the rest of the UK on profitability based on anonymised SME accounting data. East Midlands businesses saw profit growth of 18.8% in the year to Q2 2026, versus a 5.3% national average. The focus shifts to which regions and firms are sustaining stronger performance, alongside continued uptake of digital tools including AI.

d-Matrix Adopts NVIDIA NVLink Fusion for Rack-Scale XPU Deployment

NVIDIA Blog 1 month ago 49

d-Matrix announced it will connect its next-generation Raptor XPUs to NVIDIA’s AI infrastructure using NVIDIA NVLink Fusion within the MGX rack architecture. NVLink Fusion is claimed to deliver 3x lower XPU-to-XPU latency than off-the-shelf Ethernet. As a result, d-Matrix can deploy its custom silicon via NVIDIA’s standardized rack-scale platform, rather than building rack-scale infrastructure itself.

CoreWeave launches new engineering service to help enterprises implement physical AI

SiliconANGLE 1 month ago 12

CoreWeave launched a new Physical AI Field Engineering service that sends specialized engineers to work with enterprise engineering teams on AI projects. The service starts with a workshop and CoreWeave says it can cut testing times by between 17% and 35%. This shifts AI implementation toward using customer workflows and data end to end, from model training to compute setup and integration into operational tools.

Introducing IBM and NASA's new foundation model for the Moon

IBM Research 1 month ago 15

IBM and NASA released the NASA-IBM Lunar Foundation Model, an open-source AI model that integrates decades of lunar mapping data across different sensors, angles, and spatial scales. The model’s ice-prospecting task cut error by 22% versus a SwinV2 transformer. It changes lunar research by enabling faster, reusable modeling and improving crater, ice, and volcanic feature detection for Artemis planning and follow-on simulations.

Spain's top-funded tech companies in H1 2026

Tech.eu 1 month ago 38

Spain’s top-funded tech companies raised about €1.7 billion in H1 2026, with fundraising concentrated in a small set of leading firms. The ten biggest rounds together made up about 68% of the total funding. The mix of industries receiving the most capital included space (~€354.3M), travel (~€266.1M), medical/healthcare (~€195.1M), and artificial intelligence (~€150.9M), alongside funding across early and later stages and debt financing (~19%).

Powering AI is an architecture problem

MIT Technology Review 1 month ago 4

Transmission-line faults and surge arrester failures in Ashburn, Virginia caused more than 3 gigawatts and about 1,500 megawatts of AI data-center load to drop off the grid within seconds, showing the issue was grid architecture rather than power supply. The article cites an early-2026 full-scale test at the National Laboratory of the Rockies, where the system cleared ERCOT large-load voltage ride-through requirements with room to spare. By moving AI UPS and power conditioning from low-voltage, battery-based, bypass-heavy setups to medium-voltage, inline systems near substations, load swings become harder to notice and backup power can also support grid programs.

Belgium spin-out IVEX raises €5M to cut car safety testing time, eyes US and Asia

Tech Funding News 1 month ago 6

IVEX raised a €5M Series A to bring its car safety-testing AI platform to the US and Asia. The platform cuts a single batch footage review from 400 hours to eight hours, with engineers supervising each result. As a result, IVEX plans to expand coverage of NCAP protocols and build regulatory and commercial presence in North America and Asia.

Multimodal models need video. We froze time to give it to them

posthog.com 1 month ago 11

Replay Vision built a rasterizer to convert rrweb session replays into MP4 video so multimodal models could correctly “watch” them. It since March 2026 has rasterized more than 370 years of recordings into roughly 3.5 million videos. By gating Chrome’s frame rendering with HeadlessExperimental.beginFrame and freezing the rrweb player’s virtual time, the service eliminates skipped or frozen outputs so CPU contention only changes job duration, not the resulting video.

Cloudera and Mistral Partner to Bring Specialized, Sovereign Intelligence to Enterprise Data

Mistral AI 1 month ago 32 ● 10 sources

Cloudera and Mistral announced a partnership to let enterprises run Mistral AI models on Cloudera’s hybrid data platform while keeping data and model control inside their own environments. The deal targets Cloudera’s 30 exabytes of customer-managed data for sovereign AI deployment. This expands on-prem, private/public cloud, and fully air-gapped inference options and enables enterprises to train and own custom models on proprietary data.

KVector secures £3.8m to develop innovative antenna technology

Startups Magazine 37

KVector, a University of Birmingham spinout, raised funding to commercialise super-efficient antenna technology aimed at steering signals without thousands of individually controlled components. The company secured £3.8m, alongside an Innovate UK grant of £950,000, to support recruitment and development of radar and communications products with defence-sector partners. The raise speeds up product development and expands KVector’s team while it continues operating in stealth around specific partners and applications.

Nvidia and Palantir fine-tune a 30B Nemotron model for Nvidia’s supply chain. It beats a model 18 times its size.

The New Stack 1 month ago 28

Nvidia and Palantir announced they are applying their “sovereign AI” partnership inside Nvidia by fine-tuning a supply-chain AI model for Nvidia’s own operations. The fine-tuned Nvidia Nemotron 3.5 Lightning uses 30B parameters and it achieved 86.7% accuracy versus 55.5% for a 550B Nemotron model (about 18 times larger). Nvidia says this demonstrates specialization on a focused task and plans to extend the approach to other industries while letting customers train and run tailored versions with data and weights kept under their control.

DeepSeek AI Released DeepSeek-V4.1-Flash with 1M Context, FP4 KV Cache, and Cross-Layer Attention Reuse

MarkTechPost 1 month ago 19 ● 5 sources

DeepSeek AI released DeepSeek-V4.1-Flash, a multimodal Mixture-of-Experts model designed to reduce long-context KV-cache strain in LLM serving. The global KV cache footprint is 890 bytes per token with a 1M-token context window, and the model ships as open weights under an MIT license. Persistent KV caching is reduced (including FP4 quantization and bounded replay), enabling longer contexts to be more deployable with lower storage and bandwidth demands.

Async GRPO with LoRA across HF Jobs: a bucket, a proxy, and no NCCL

Hugging Face 1 month ago 16

AsyncGRPOTrainer in TRL v1.14 was updated to train a LoRA adapter and sync only the adapter to vLLM replicas running in separate Hugging Face Jobs, using a Storage Bucket mount plus a proxy rather than NCCL. Five runs using the same recipe for 500 steps fell from 3 h 27 min to 53 min. The setup changes training-inference coordination by persisting versioned adapters to shared storage, having the proxy route rollouts to replicas with matching KV prefixes, and requiring vLLM to load/unload adapters at runtime while checkpoints can resume after preemption.

To Infinity and Beyond: ThunderKittens Now on NVIDIA Vera Rubin NVL72!

Together AI 1 month ago 32

Together’s kernels team gained access to the NVIDIA Vera Rubin NVL72 platform and updated ThunderKittens to run NVFP4 and FP8 GEMMs on it by adding Rubin-specific ISA and memory/pipeline features. The NVFP4 GEMM kernels improve from about 42.1% of the roofline to over 22 PFLOPS. These changes increase attainable performance by doubling the K step, expanding tensor memory and shared memory use, and adjusting tiling and commit/collector behaviors so tensor cores stay fed.

The daily briefing

Every AI story that matters, in your inbox by 8am.

TLDRocket reads all relevant sources, removes duplicate coverage, and summarises the day in two minutes. Follow companies and topics for alerts, or get the briefing in Slack. Free, no spam, unsubscribe anytime.