Soup (GitHub Repo)
GitHub
Soup lets you fine-tune LLMs with one command, even on a 4 GB laptop GPU. It’s still beta, but it’s already streaming layers and rejecting bad config keys.
Based on reporting by GitHub — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
Soup is trying to turn LLM fine-tuning from a weekend of setup pain into something closer to a normal command-line job. Install the CLI, write one YAML file, run soup train. That’s the pitch. And unlike a lot of “simple” ML tooling, this one is built around the idea that the messy parts should stay hidden without becoming magical or silent.
The most eye-catching claim is hardware fit. Soup says it can fine-tune an 8B model on a laptop GPU with 4 GB of VRAM by streaming frozen layers into the card one decoder layer at a time. On an RTX 3050 Laptop 4 GB, the project reports Llama-3.1-8B-Instruct with NF4 reaching 119.6 tok/s with 3.32 GB peak memory, and says the run matched a normal resident model exactly. The same setup was independently reproduced on an H100 at 113.00 tok/s with the same 3.32 GB peak. The project also says that specific tok/s figure came from v0.72.2, before a correctness repair in v0.73.0.
Version 0.75.0 is where the project gets opinionated about configuration. Unknown keys now fail the load instead of being silently dropped, so a typo like quantizaton stops the run and names the field you probably meant. That sounds harsh until you remember what the old behavior actually did: it let people believe a setting was active when it wasn’t. Soup is also enforcing backend honesty more aggressively, especially on MLX, where several accepted options were previously read, validated, and then ignored.
The release notes read like a catalog of sharp edges being filed down. Validation loss is now recorded and shown instead of vanishing. Web UI read endpoints and SSE now require auth through short-lived single-use tickets, and a training subprocess no longer hangs when nothing is reading its output. Support also moved forward on the dependency side: torch>=2.6.0 fixes the import dead-end seen with 2.5.1 and trl>=0.29, while Python 3.10 through 3.12 remain the supported range.
Soup is still beta, and it says so plainly. But the project is also unusually explicit about what it measures, what it rejects, and what each backend actually does. That combination is rare. A lot of ML tooling promises simplicity by hiding the cracks; Soup seems more interested in making the cracks visible and then refusing to step over them.
My take — AI-written commentary, not fact-checked reporting
This is the right kind of boring: configuration that fails loudly, metrics that don’t vanish, and backends that don’t pretend. The AI world has enough tools that smile politely while dropping your settings on the floor. Soup’s attitude is almost rude, which is usually a sign somebody finally got tired of being lied to by software.
Read more about this at: GitHub
