04 · How Models Run
FP8 / INT8 / INT4
Low-precision inference formats
Lower-precision formats that make inference cheaper and faster. INT4 squeezes each number into 4 bits — heavy compression, used to run big models on small hardware.
Concrete example
An INT4 build of a large model can run on a single consumer GPU that the full version would never fit on.