Prompt Caching
Reusing precomputed KV states for shared prefixes and system prompts
Performance
In plain English
Remembering the instructions from earlier so the AI doesn't have to re-read your 50-page codebase document every time you ask a follow-up question.
Technically
Storing the KV cache of invariant prefix tokens (e.g. system instructions, tool schemas, retrieved context) in RAM/VRAM. When subsequent requests share the prefix, prefill computation is skipped, reducing TTFT by up to 90%.