GPTQ
Second-order error compensation quantization for GPUs
Formats
In plain English
A mathematical formula that rounds numbers down while making micro-adjustments to neighboring numbers so nothing gets distorted.
Technically
Accurate Post-Training Quantization for GPT models (Frantar et al., 2022). Uses inverse Hessian matrix information to minimize layer-wise output error during one-shot 4-bit weight quantization, enabling fast GPU dequantization kernels.