DeepSeek-V3 Released — 671B Parameters Trained for Roughly $5.58M

DeepSeek released “DeepSeek-V3,” a Mixture-of-Experts model with 671 billion total parameters. Released under a permissive license that also allows commercial use, the company presented benchmark results claiming it surpassed GPT-4o, Llama 3.1 405B, and Qwen 2.5 72B, catching the development community by surprise.

Details

• Model scale: A Mixture-of-Experts model with 671 billion total parameters
• Training cost: DeepSeek said the model was trained in about two months for roughly $5.58M, using a data center of NVIDIA H800 GPUs — chips constrained by U.S. export restrictions
• Performance: The company claimed results surpassing Llama 3.1 405B and GPT-4o on benchmarks including the coding competition site Codeforces
• License: Released under a permissive license allowing free downloading, modification, and commercial use
• Release date: Released on Thursday, December 26, 2024

What happened next

DeepSeek-V3’s pitch of low cost combined with high performance set the stage for the anticipation surrounding “DeepSeek-R1,” a reasoning-focused model released three weeks later. While the global reaction to V3 itself was still fairly limited at release, R1’s arrival expanded that impact dramatically, eventually affecting the stock prices of major tech companies worldwide.