BitNet b1.58 (1-bit LLMs)
Ternary weight quantization {-1, 0, +1} removing matrix multiplication
Formats
In plain English
Replacing expensive calculator multiplication with simple addition and subtraction tally marks, letting tiny chips run giant models.
Technically
A 1.58-bit model architecture (Ma et al., 2024) where every weight is constrained to {-1, 0, +1}. Replaces matrix multiply-accumulate (MAC) operations in GEMM with pure integer additions, slashing energy consumption by >80% and shrinking model size to ~0.2 bytes per parameter.