Will it run?
Pick a model, a quantization and a context length. Every machine is sized against the model's real attention geometry — the KV cache is computed from its published config, not guessed at — and the context window is charged against both memory and bandwidth, because attention re-reads the whole cache for every token it generates.
| Machine | Price | Usable | Headroom | Verdict | Decode t/s | KV share | Prefill t/s | Max context |
|---|---|---|---|---|---|---|---|---|
96 GB · 1792 GB/s | $15,499 | 88.3 GB | 44.9 GB | comfortable | 28 | 6% | 312 | — |
512 GB · 1200 GB/s | $13,500 | 384 GB | 340.6 GB | comfortable | 18 | 6% | 64 | — |
48 GB · 864 GB/s | $3,899 | 44.2 GB | 0.8 GB | tight | 13 | 6% | 43 | — |
512 GB · 819 GB/s | $9,499 | 384 GB | 340.6 GB | comfortable | 13 | 6% | 30 | — |
128 GB · 614 GB/s | $4,500 | 96 GB | 52.6 GB | comfortable | 9 | 6% | 32 | — |
64 GB · 307 GB/s | $2,760 | 48 GB | 4.6 GB | comfortable | 5 | 6% | 16 | — |
128 GB · 273 GB/s | $5,199 | 117.8 GB | 74.4 GB | comfortable | 4 | 6% | 22 | — |
128 GB · 273 GB/s | $3,999 | 117.8 GB | 74.4 GB | comfortable | 4 | 6% | 22 | — |
32 GB · 1792 GB/s | $4,400 | 29.4 GB | -14 GB | impossible | — | — | — | — |
24 GB · 1008 GB/s | $3,499 | 22.1 GB | -21.3 GB | impossible | — | — | — | — |
24 GB · 960 GB/s | $1,460 | 22.1 GB | -21.3 GB | impossible | — | — | — | — |
32 GB · 640 GB/s | $1,700 | 29.4 GB | -14 GB | impossible | — | — | — | — |
24 GB · 576 GB/s | $1,400 | 22.1 GB | -21.3 GB | impossible | — | — | — | — |
32 GB · 512 GB/s | $1,399 | 29.4 GB | -14 GB | impossible | — | — | — | — |
32 GB · 456 GB/s | $1,300 | 29.4 GB | -14 GB | impossible | — | — | — | — |
24 GB · 456 GB/s | $650 | 22.1 GB | -21.3 GB | impossible | — | — | — | — |
48 GB · 273 GB/s | $2,199 | 36 GB | -7.4 GB | impossible | — | — | — | — |
Verdicts. Comfortable leaves more than 8% of usable memory spare.Tight fits but will not survive a longer prompt. Spills means the weights fit but this context does not — shorten it or quantize the cache. Impossible means the weights alone exceed usable memory, so no setting here saves it.
Buy or rent?
The cheapest cloud GPU with enough memory for this workload is 2× RTX A5000 on RunPod at $0.32/hr. Against RTX PRO 6000 Blackwell at $15,499 plus $0.102/hr of electricity, owning pays for itself after 71,096 hours of use — about 296.2 months at 8 hours a day. That is the cheapest way to rent this much memory, but splitting a model across 2 cards costs interconnect overhead these rates do not show — a single larger GPU will usually feel faster for the same money.
| Cloud option | Memory | $ / hour | $ / month at 8h/day | $ / month 24/7 | Go |
|---|---|---|---|---|---|
2× RTX A5000 RunPod · community | 48 GB | $0.32 | $77 | $230 | → |
RTX A6000 RunPod · community | 48 GB | $0.33 | $79 | $238 | → |
A40 RunPod · community | 48 GB | $0.35 | $84 | $252 | → |
2× RTX 3090 RunPod · community | 48 GB | $0.44 | $106 | $317 | → |
2× RTX A5000 RunPod · secure | 48 GB | $0.54 | $130 | $389 | → |
RTX A6000 RunPod · secure | 48 GB | $0.59 | $142 | $425 | → |
A40 RunPod · secure | 48 GB | $0.59 | $142 | $425 | → |
2× RTX 4090 RunPod · community | 48 GB | $0.68 | $163 | $490 | → |
Rates read from each provider's own pricing page on Nov 8, 2026 · Lambda's published rates take effect Nov 16, 2026. Electricity at $0.17/kWh. Multi-GPU rows assume you can split the model across cards, which costs interconnect overhead this model does not account for. Vast.ai is left out on purpose: its marketplace price moves per host, so any single number would be fiction.