How the numbers on this site are made

There is a lot of hardware advice on the internet right now that was generated rather than learned. It reads fluently, cites nothing, and quietly invents a tokens-per-second number because a number was required in that position.

This site takes the opposite position: no figure here is typed in by hand, and the formula that produces all of them is published. That means you can disagree with it precisely, which is the only kind of disagreement worth having.

Two models, deliberately

The comparison table on the front page uses a deliberately simple model: six fixed workload profiles, and context folded into a flat 1.15× headroom factor. It exists to answer “roughly, how do these machines rank against each other.”

The calculator uses a sharper one. It reads each model’s real attention geometry out of its published config.json — layer count, KV head count, head dimension — and sizes the cache from that. Nothing is estimated from parameter count.

Where the two disagree above zero context, the calculator is right.

The thing most comparisons miss

Generating one token requires reading every active weight from memory. That makes single-stream decode bandwidth-bound, not compute-bound, which is why memory bandwidth predicts felt speed far better than TFLOPS does.

But the KV cache is also read — all of it, every token. So the real cost is:

decode = efficiency × bandwidth ÷ (activeWeights + kvCache)

At short context the cache is a rounding error and this collapses to the familiar bandwidth-over-weights estimate. At long context it dominates completely. Llama 3.3 70B at Q4 on an RTX PRO 6000 is about 28 tok/s at 8k and 15 tok/s at 128k — the cache has grown to 40 GB, matching the weights, and half of every token’s bandwidth now goes to re-reading conversation history rather than to the model.

Almost nothing else in this category shows you that, and it is the single most important fact when deciding how much bandwidth to buy.

The payoff for a good attention design

Run the calculator on DeepSeek-V3 and the reason multi-head latent attention exists becomes obvious. At 128k context:

A 671B model with a cache a quarter the size of a 70B model’s. MLA caches one compressed latent per token per layer instead of keys and values for every head. For DeepSeek-V3 that is 576 values per token per layer against Llama’s 2,048 — and it is why DeepSeek is far more practical at long context than its parameter count suggests, if you can hold the weights at all.

What the model gets wrong

Being specific about this matters more than the numbers themselves.

The software-efficiency constants are guesses. Prefill is estimated as a fraction of nameplate FP16 throughput, and that fraction is a per-vendor judgement about software maturity: 35% for NVIDIA, 15% for Apple, 10% for AMD, Intel and the startups. Those numbers encode CUDA’s head start, not silicon. They will age, and every prefill figure moves with them. This is the most contestable table on the site and it is labelled as such on the methodology page.

Multi-GPU scaling is not modelled. Every figure is for a single machine. Splitting a model across cards adds interconnect cost that does not appear anywhere here, including in the buy-vs-rent comparison.

Thermals are not modelled. Sustained throughput on a laptop, or on a blower card twenty minutes in, is lower than what you see.

Quantization quality is not modelled. Fitting a model is not the same as it being good. IQ2 will load things it should not run.

What would make it better

Measurements. Where a machine has actually been benchmarked, the measurement beats the model, and a post here will say so and show the numbers and the command that produced them.

That is what the rest of this blog is for.