Chapter 26: Checkpoint: language
What to remember, a mixed quiz, and a classifier for your own messages.
Six chapters ago a network could not be handed a word. Since then you have built a tokenizer's dictionary, trained a model that invents names, watched a network learn where to look, followed a message through a chatbot, sorted 10,000 headlines, and seen four ways a model goes wrong. This is the place to check that it stuck, and then to try it on words that are yours.
Eight things to remember
- A language model never sees letters or words: text is chopped into tokens, each token is swapped for an id, and each id looks up a row of learned numbers called its embedding.
- The dictionary of tokens is learned by counting which neighbors appear together most, and an id is only a row number: the meaning lives in the row.
- A language model does one thing, which is to give every possible next token a probability, and writing is that one thing in a loop: predict, sample, append, repeat.
- What a language model writes follows the habits of its training data and is invented on the spot, not looked up, so it cannot tell you which of its sentences are true.
- Attention is a lookup with soft edges: a token's query is scored against every other token's key, softmax turns the scores into shares, and what comes back is a blend of values.
- A chatbot is a language model trained in stages to continue a conversation helpfully, and during a chat its weights are frozen: everything it "remembers" is text sitting in its context window.
- A decision model runs once and returns a typed answer with a probability, and what to do with that doubt is a human decision.
- A model learns whatever its data and its score reward, so one overall number can hide who it fails, and a confident voice is not evidence.
Mixed quiz
Seven questions from all over Part 3. If one trips you up, the "why" under it says which chapter to revisit.
A tokenizer gives one word the id 512 and another the id 513. What does that tell you about the two words?
Ids are handed out in the order the pieces were created. Whether two tokens are alike is decided by their embedding rows, and those are learned. See chapter 20.
Your name inventor wrote a name that turns out to belong to a real person, and that name was never in its training list. How did that happen?
The model has only a few thousand weights and no list to consult. It samples likely letters, and likely letters sometimes spell a real name. It has no idea which of its names are real. See chapter 21.
You lower the temperature of a language model from 1.0 to 0.2. What changes?
Temperature is a setting on the dice, not on the network. Low temperature gives safer, more repetitive writing. High temperature gives surprises, and more nonsense. See chapter 21 and chapter 23.
Before any training, the locker-room model spread its attention evenly over all the lockers. Why?
Softmax turns equal scores into equal shares. Only after training have the query and key weights grown enough for one locker's score to stand out. See chapter 22.
Halfway through a long chat, a chatbot "forgets" something you told it at the very start. What is the most likely reason?
The weights do not change while you chat. The window is the model's entire memory of the conversation, and it has a maximum length. See chapter 23.
A decision model says a message is spam with probability 0.55. What is the most sensible thing for a program to do with that?
The model supplies the doubt. Where to draw the line depends on whether a missed spam or a lost real message costs more. Asking again gives the same number: nothing about the model has changed. See chapter 24 and chapter 25.
A model that spots a rare illness is "99% accurate". Only 1 patient in 100 has the illness. What should you ask first?
A model that always says "healthy" would also be 99% accurate and would catch nobody. When the thing you are looking for is rare, accuracy cannot tell a useless model from a good one. See chapter 25.
Build it: a classifier for your own messages
In chapter 24 you trained a model on 10,000 headlines that somebody else collected. Now you are the one collecting. You will write about twenty short messages of two kinds that you choose, and train a tiny version of the same model to tell them apart.
- Pick two kinds of message that you could sort at a glance. Homework or fun. Excited or grumpy. Something you would say or something your sister would say.
- Write about ten of each. Short, lowercase, in your own words. Real ones are best.
- Predict: which single words do you think the model will lean on hardest?
- Run it, then type new sentences for it to sort.
The script starts with twenty made-up messages, ten about homework and ten about fun, so you can watch it work before you change anything. It is the model from chapter 24, shrunk: a dictionary of words, a row of 8 numbers for each word, an average, and one layer that turns the average into a score for each kind.
The script holds back 5 of the 20 messages and trains on the other 15. One of the held-back messages is "i forgot my math book at school", which is a homework message. What will the model say?
It says fun, 90% sure. With that message held back, the model has never seen "forgot" or "book", so they are invisible to it. It can see five words: i, my, math, at and school. Only "math" points toward homework. The other four turned up in its fifteen training messages only in fun ones ("i got a new bike", "at my house", "at the park after school"), and four small votes beat one.
Sort your own messages
This is my_messages.py. Press run with the starter messages first and read the output from top to bottom: the loss falling, the five held-back messages with the model's answer for each, the three scores, the words it leans on hardest, and what it makes of the sentences in TRY. Words in [brackets] are ones the model cannot see, because they were in none of its training messages.
What you should have seen
It learned something. The model has 546 weights. After 200 epochs it gets all 15 training messages right and 4 of the 5 held-back ones. Always answering "fun" would have scored 2 of 5, so it beat the baseline.
It leans on single words. Look at the list near the end of the output. Toward fun: saturday, pool, park, party. Toward homework: test, math, english, and also for and have, which are not about homework at all. They just happened to turn up in homework messages. That is the shortcut learning from chapter 25, in miniature.
You can fool it with one word. "we have pizza" comes out as homework, 97% sure, because "we" and "have" outvote "pizza". "homework is fun" comes out as homework, 98% sure, and look at the brackets: the model has never seen the word "fun" in a message, because "fun" was only ever the name of a pile.
It shrugs when it sees nothing. "fantastic news everybody" gets 50/50. Every word is in brackets, so the model has no evidence at all, and says so. That is the best thing it does.
Five messages is a blurry measuring stick. Change SEED and run it again. A different five are held back, the dictionary changes with them, and the held-back score moves. With a test this small, one message is worth twenty points. This is why real test sets have thousands of rows.
Now make it yours. Replace the lines in MESSAGES with your own two kinds, put your own sentences in TRY, and run it. A few things to notice, none of which mean you did it wrong:
- If one give-away word appears in most messages of one kind, the model will use that word and little else. Try writing messages that avoid it.
- A word it has never seen is invisible. Twenty messages is a very small dictionary.
- More messages help more than anything else you can change. Add ten and compare.
Text becomes ids, ids become rows of numbers, and a network turns those rows into a prediction: the next token, or an answer. What comes out follows the habits of the data it was trained on, and nothing else.
Show your classifier to someone and ask them to write a sentence that fools it. Then work out together which word did the damage (the bracketed words and the "leans on hardest" list are your clues) and add two or three training messages that would fix it. Does the fix hold, or did you just move the problem to a different word?
Next: seeing, creating, playing
Everything in this part was made of text. Part 4 is about the other things a network can take in and put out: pictures, and moves in a game. It starts in chapter 27, where you train a network on thousands of handwritten digits and then write some for it yourself.