tencent/Hy3
Simon Willison Simon Willison ● Covered by 2 sources
Tencent released Hy3, a 295-billion-parameter mixture-of-experts model under Apache 2.0 license, following feedback from 50+ products that used its April preview version. The model uses 21 billion active parameters and supports a 256,000-token context length, with the full version requiring 598GB storage and a quantized version at 300GB. The model is available free on OpenRouter until July 21st and competes with open-source models having 2-5 times more parameters.
Why it matters
tencent/Hy3 New Apache 2.0 licensed model from Tencent in China: Hy3 is a 295B-parameter Mixture-of-Experts (MoE) model with 21B active parameters and 3.8B MTP layer parameters, developed by the Tencent Hy Team. Following the Hy3 Preview launch in late April, we gathered feedback from 50+ products and scaled up post-training with higher quality data. Today, we introduce Hy3, which outperforms similar-size models and rivals flagship open-source models with 2-5x parameters. It also shows significant gains in utility across various products and productivity tasks. The full-sized model is 598GB on Hugging Face, and the FP8 quantized one is 300GB. The context length is 256K. It's available for free on OpenRouter until July 21st. I had it "Generate an SVG of a pelican riding a bicycle" there and got this: Update: I'd forgotten about this but Max Woolf wrote about an earlier preview of this model back on May 26th: The mysterious Hy3 LLM is topping OpenRouter Model Rankings by a large margin.
