Take the controls of a real, if tiny, model and watch every guess it gets wrong nudge it a little closer to getting it right.
What follows is a real model rather than an animation of one. It is a very small one that only looks at the single word before, and it learns from the 6 short sentences printed at the bottom of this console. Press step once and it takes one pass through every pair in that text, guessing and being corrected as it goes. Run it all the way to the end, then drag the step size and watch the whole run reshape itself.
the?satonthemat
This is one of the pairs it practises. The highlighted word is what it is given and the question mark is the word it has to guess. On every pass it works through every pair in the text like this one, guesses the hidden word, and is scored on how far off the guess was.
Each bar is one pass through the practice text and its height is how wrong the model was on that pass. A run that is learning slopes down towards the dashed line, and a run whose steps are too big bounces about above it instead.
what the model expectshow often that word really follows
Training slides the bars onto their marks, and that is the whole of it. Changing the word above only changes what you are looking at, because it practises every pair in the text either way.
No guess Every word it knows sits at 9% after “the”, so it has no favourite to offer yet. In the practice text, the word that follows “the” most often is “cat”.
How far every number moves on each correction, which engineers call the learning rate. Move this and the run is worked out again from the first pass at the new step size, so the whole shape of it changes in front of you.
Every number inside this model is currently zero, so it treats every word it knows as exactly as likely to come next as every other. It has read nothing and remembered nothing. Press step once to give it a single pass through the text.
6 short sentences and 11 different words. Every figure on this console comes out of these lines, so you can count the pairs for yourself.
The loop you have been running is all of it. A real model runs the same guess, score and nudge over an enormous amount of text, with an enormous pile of numbers, and it looks back over far more than the single word before. Nothing is filed away as a fact. What changes is the numbers, and once they have been corrected enough times the predictions read as fluent language.
A model that has only ever predicted the next word will happily carry on writing your question instead of answering it. A second and much smaller stage sorts that out. It is shown worked examples of the kind of answer people actually want, then tuned further by people comparing two of its answers and picking the better one.
Pre-training gives it language. Fine-tuning gives it manners.
The model on this page looks only at the single word before, learns from 6 short sentences and knows 11 words in total, so it is a toy version of the real thing. A real model runs the same loop over vastly more text with vastly more numbers, and it reads far more of the sentence at once. Provided by the Institute of AI for interest and learning.
The model is shown real text with the next bit hidden, it guesses, and it is scored on how far off it was. Nobody writes the rules of grammar down for it anywhere.
How badly a guess missed is boiled down to a single figure, the loss. Training is the business of driving that figure down, pass after pass after pass.
Nothing gets rewritten. Each pass moves every number a small step in the direction that would have made the right answer more likely, and the step size matters enormously.
A long first stage over an enormous amount of text gives the model language. A second and much smaller stage shows it examples of the answers people actually want, then has people compare its answers and tunes it towards the ones they prefer.
Get AI is a collection of games, experiences, and tools that make AI easier to understand from the Institute of AI.
