Qwen2.5-Max: Exploring the Intelligence of Large-scale MoE Model
Qwen ● Covered by 3 sources
Alibaba released Qwen2.5-Max, a Mixture-of-Expert language model pretrained on over 20 trillion tokens, which is now available through Alibaba Cloud's API and Qwen Chat. The model outperformed DeepSeek V3 on Arena-Hard, LiveBench, LiveCodeBench, and GPQA-Diamond benchmarks while achieving competitive results on MMLU-Pro. The API is accessible through Alibaba Cloud with OpenAI-compatible endpoints for developers to integrate the model into applications.
Why it matters
QWEN CHAT API DEMO DISCORD It is widely recognized that continuously scaling both data size and model size can lead to significant improvements in model intelligence. However, the research and industry community has limited experience in effectively scaling extremely large models, whether they are dense or Mixture-of-Expert (MoE) models. Many critical details regarding this scaling process were only disclosed with the recent release of DeepSeek V3. Concurrently, we are developing Qwen2.