Learn · Popular Open Source Models · The DeepSeek Breakthroughs
DeepSeek-V3 & DeepSeek-Coder-V2 (MoE)
In plain English
DeepSeek-V3 is a giant 671B model that only activates 37B weights for each word. That means you get the intelligence of a massive brain at the running speed of a small one! ⚡
What is actually going on
DeepSeek-V3 features a fine-grained Mixture-of-Experts architecture: 256 routed experts (activating top-8) plus 1 shared expert. Combined with Multi-Head Latent Attention (MLA) and FP8 mixed precision, it requires ~350GB RAM in Q4 quantization. It runs locally at 15-20 tok/s on 512GB Mac Studio M5 Ultra or clustered DGX Sparks.
Why it matters
Sparse MoE architecture decouples total knowledge capacity from per-token decode cost, making trillion-parameter capabilities feasible on desk-side iron.
Another way to think about it
A hospital with 256 specialized doctors where only the 8 most relevant specialists consult on any single question. 🏥👨⚕️
Check yourself
How many parameters are actively computed per token in DeepSeek-V3 / R1 (671B total)?
- 8B
- 37B
- 671B
- 1000B
Show answer
37B — While total parameters are 671B, only 37B active parameters (top-8 routed experts + 1 shared expert) are computed per token.