Alibaba releases "Qwen2" — five sizes accelerate openness to developers worldwide
On June 7, 2024, Alibaba released “Qwen2,” the successor to the previous generation “Qwen1.5.” It’s offered in five sizes — 0.5B, 1.5B, 7B, 57B-A14B (MoE), and 72B — and expanded supported languages from English and Chinese to 27 languages. “We want Qwen to empower developers not just in China but around the world to build powerful and responsible AI applications,” said Alibaba Cloud AI’s chief architect.
Details
- Five sizes offered: In addition to standard 0.5B, 1.5B, 7B, and 72B models, a MoE model called “57B-A14B” — with 14B of its 57B total parameters active per token — was also included
- Expanded multilingual support: Added support for 27 additional languages beyond English and Chinese, substantially strengthening multilingual capability
- Context length: Qwen2-7B-Instruct and Qwen2-72B-Instruct support up to 128K tokens
- License change: The 0.5B, 1.5B, 7B, and 57B-A14B models were switched to the Apache 2.0 license (only the 72B model retained the previous Qianwen license). Alibaba described the change as intended to “increase openness and accelerate global adoption”
- Performance improvements: Significantly improved coding and math capabilities, with efficiency also boosted by adopting Group Query Attention (GQA)
What happened next
Qwen2 was succeeded by “Qwen2.5” just three months later, underscoring the rapid pace of the Qwen series’ development cycle. The major licensing opening helped drive the subsequent rapid global adoption of the Qwen family.