DeepSeek open-sources its first large language model, "DeepSeek LLM 67B"

Chinese AI startup DeepSeek has open-sourced its first large language model, “DeepSeek LLM.” Available in two sizes, 7B and 67B, each with Base and Chat variants, the model was trained from scratch on a 2-trillion-token dataset including English and Chinese. At the time, DeepSeek positioned it as its first major product on “a new frontier of open-source LLMs.”

Details

• Model scale: A 67B-parameter model trained from scratch on 2 trillion tokens of mixed English and Chinese data
• Release format: Both the 7B and 67B Base and Chat versions were open-sourced for the research community
• Performance claims: DeepSeek’s own benchmark results claimed the model outperformed Llama 2 70B Base in areas including reasoning, coding, math, and Chinese-language understanding
• Comparison: The 67B Chat model reportedly showed benchmark performance surpassing GPT-3.5

What happened next

This release of DeepSeek LLM became the starting point for a series that would later include DeepSeek-V2, V3, and, in January 2025, DeepSeek-R1 — which shook the global AI market. DeepSeek’s core strategy of repeatedly releasing low-cost, high-performance open-source models was already consistent from this very first model.