Launch HN: General Instinct (YC P26) – Frontier models on edge devices
Hacker News guanming0717
General Instinct, a YC-backed startup, released InstinctRazor, a tool for compressing large language models to run on edge devices with limited compute. They compressed Qwen3.5-122B from 245 GB to 48 GB while matching or exceeding the performance of smaller models like Gemma-4-26B on benchmarks. The technique enables frontier models to run on robotics and edge hardware with 7.6–8 GB peak VRAM usage, addressing the gap between datacenter-optimized models and resource-constrained physical systems.
Why it matters
Hey HN, Guanming and Bill here from General Instinct (https://general-instinct.com/).After years of working in robotics, we kept running into the same problem: the best models never fit the hardware we actually had available.The models that performed best were usually designed around datacenter assumptions: large GPUs, lots of memory bandwidth, and reliable network access. But most physical systems have the opposite constraints.That led us down the path of figuring out how much of a frontier model could be preserved while still making it practical to run on edge hardware.As part of that work, we recently open sourced InstinctRazor (https://github.com/General-Instinct/InstinctRazor)One result we're excited about is compressing Qwen3.5-122B-A10B, a roughly 245 GB BF16 MoE model, into a 48 GiB GGUF. The resulting model is actually smaller than Gemma-4-26B-A4B while outperforming it on benchmarks like MMLU-Pro and GPQA-D etc. we preserve the parts that are always active (router, norms, Gat
Related stories
Liquid AI Releases LFM2.5-2.6B: An On-Device Agentic Model With 128K Context, Tool Calling, And Open Weights
MarkTechPost · 3 weeks ago ·
44
Introducing Gemma 3n: The developer guide
Google DeepMind · 10 months ago ·
41