TLDRocket
Sign in
Latest The new energy permitting bill to feed the AI boom unites fossil fuels... — Fortune ‘Never seen anything quite like it’: How rural Pennsylvania’s $165 mil... — Fortune Satya Nadella: ‘We Must Assume a Model Is Compromised’ — Trending Topics Can the AI industry persuade data center opponents by getting rid of N... — TechCrunch Flahy uses knowledge graphs to support AI-powered healthcare — SiliconANGLE Silicon Valley is falling in line with Trump’s attempt to rename AI — The Verge Claude Code found my app’s shared contract in two MCP calls. Then the... — The New Stack These execs think voice AI hasn’t reached its ChatGPT moment yet — TechCrunch

The AI intelligence platform

Every AI story that matters — and the intelligence behind it.

TLDRocket reads all relevant sources, removes duplicate coverage, and publishes a short neutral summary of every story, linking back to the original. Free, no spam, unsubscribe anytime.

Add to Slack

Every story also updates live profiles event timelines weekly rankings the AI Market Index

Tuesday, 24 October 2023

Exploring simple optimizations for SDXL

Hugging Face 2 years ago 33

Stability AI researchers tested optimization techniques for SDXL, their 3.5B-parameter image generation model, to reduce memory consumption and inference time. Running SDXL unoptimized required 28GB of memory and took 72.2 seconds to generate 4 images on an A100 GPU, but combining fp16 precision, scaled dot product attention, and torch.compile reduced inference time to 10.3 seconds while using 21.7GB of memory. Further memory reductions to 11.47GB are possible by adding VAE slicing and sequential CPU offloading, enabling the model to run on consumer GPUs at the cost of slower inference speed.

Read the full story
Model Optimization

The N Implementation Details of RLHF with PPO

Hugging Face 2 years ago 24

Researchers reproduced OpenAI's 2019 RLHF implementation with PPO by studying the original codebase and documented the engineering details needed to match its results on stylistic tasks. Their reproduction codebase produces learning curves nearly identical to OpenAI's original code when training on sentiment and descriptiveness tasks. The work provides a checklist of implementation details and a minimal reference implementation to help others understand RLHF engineering without requiring the specific hardware (8 V100 32GB GPUs) that OpenAI's original code demands.

Read the full story

Deploy Embedding Models with Hugging Face Inference Endpoints

Hugging Face 2 years ago 26

Hugging Face released Text Embeddings Inference (TEI), a service for deploying open-source embedding models on its Inference Endpoints platform with automatic scaling and security features. A benchmark of the BAAI/bge-base-en-v1.5 model on an Nvidia A10G instance achieved 450+ requests per second at a cost of $0.00000156 per 1,000 tokens, 64 times cheaper than OpenAI's embedding service. Developers can now deploy embedding models for retrieval-augmented generation tasks like semantic search and chatbots with minimal infrastructure management and significantly lower costs.

Read the full story
Hugging Face Inference Optimization Embedding Models

The daily briefing

Every AI story that matters, in your inbox by 8am.

TLDRocket reads all relevant sources, removes duplicate coverage, and summarises the day in two minutes. Follow companies and topics for alerts, or get the briefing in Slack. Free, no spam, unsubscribe anytime.