Learn · Foundations · Your First Mental Model
Parameters & Weights
In plain English
Parameters are the AI's brain cells! 🧠 When an AI learns, it adjusts billions of tiny knobs (weights) until it gets the right answers.
What is actually going on
Parameters are the numerical weights within the neural network's matrices, learned during training. In a 7B model, there are 7 billion of these floating-point numbers defining the model's behavior.
Why it matters
The parameter count directly dictates the model's capabilities, its RAM footprint, and the compute required for inference.
Another way to think about it
A recipe with 7 billion ingredients. You have to measure every single one perfectly! 🍲
Check yourself
What does the '7B' mean in 'Llama-3-7B'?
- 7 Bytes of size
- 7 Billion parameters
- 7 Basic functions
- 7 Billion tokens trained
Show answer
7 Billion parameters — It refers to the 7 billion numerical weights inside the model.
Parameters are fixed after training and do not change during inference.
- True
- False
Show answer
True — During inference (running the model), weights are read-only. They only change during training.
More parameters generally means:
- A smarter but larger and slower model
- A smaller, faster model
- A model that doesn't need RAM
- A model that only speaks English
Show answer
A smarter but larger and slower model — More parameters increase reasoning ability but at the cost of higher memory and compute requirements.