The Inference Pipeline
Understand how tokens are generated under the hood
3 modules6 lessons18 questions
01 How Generation Works
From Prompt to Response
3 qWhen you talk to the AI, it reads your whole message, thinks for a second, and then spits out the answer one word at a time. ⏱️
The Transformer Architecture
3 qTransformers are like a room full of experts who talk to each other to figure out the context of a sentence! 🗣️ They pay 'attention' to all words at once.
02 The Two Phases
Prefill — Reading Your Input
3 qThe Prefill phase is the AI speed-reading your prompt all at once! 📚 It builds a mental map of what you asked.
Decode — Writing the Answer
3 qThe Decode phase is the AI talking back to you, one word at a time. 🗣️ It has to think about everything it just said before saying the next word.