In plain English
The numbers inside a model are normally stored at high precision, which is accurate but heavy. Quantisation rounds them to a coarser scale, often from sixteen bits down to eight or four. The file shrinks dramatically, the model needs far less memory, and it responds faster, at the cost of a small amount of accuracy. It is the main reason capable models can now run on a laptop or a phone rather than a datacentre.
Why it matters
It is what makes local and on-device AI practical, which in turn matters for privacy, cost, and working without an internet connection.
A worked example
A model that needs thirty gigabytes of memory at full precision might need under ten once quantised, bringing it within reach of an ordinary desktop machine.
Common confusion
Quantisation is not the same as distillation. Quantisation keeps the same model and stores it more roughly; distillation trains a smaller model to imitate a larger one.