Glossary
Every special word in the book, in plain language. In the chapters, words with a dotted underline show these same definitions when you hover or tap.
- accuracy
- The share of answers a model got right. Handy, but one accuracy number can hide who the model fails, and it means little when the thing you are looking for is rare.
- action
- One of the things an agent can choose to do. In the maze there are four: up, right, down and left.
- activation
- The “squisher” applied after the weighted sum — sigmoid, ReLU, and friends. It bends the line so the network can learn curves.
- Adam
- The most widely used training method: momentum plus an automatic step size for every weight.
- agent
- The one doing the acting. In reinforcement learning it is the learner: it looks at its situation, picks an action and collects the reward.
- AI agent
- A language model run in a loop with tools: it asks for an action, reads what came back, and decides what to do next until the job is done.
- AI winter
- A stretch of years when excitement about AI, and the money for it, drains away.
- alignment
- The work of making sure an AI system does what people actually meant, and not only what its training happened to reward.
- architecture
- The shape of a network: how many layers it has and how many neurons are in each one.
- attention
- How a token gathers information from other tokens: it scores each one for how useful it looks, then takes a blend that leans toward the highest scorers.
- attention head
- One set of query, key and value weights. A transformer layer runs several heads side by side, each free to look for something different.
- attention weight
- How much one token listens to another, from 0 to 1. A token's attention weights over all the others add up to 1.
- automatic differentiation
- Getting gradients without working out formulas by hand: the computer keeps notes during the forward pass and reads them backwards.
- backpropagation
- The bookkeeping that sends the error backward through the network to find each weight’s gradient.
- backward pass
- The second half of a training round: start at the loss and hand the blame back through the network, layer by layer.
- base model
- A language model straight out of pre-training. It continues documents and has not yet been taught to act as an assistant.
- baseline
- The score you would get with no skill at all, such as by guessing. A result only means something compared with it.
- batch size
- How many training examples the network looks at before each nudge of its weights.
- bias
- A neuron’s built-in lean — how eager it is to fire before any input arrives. A positive bias leans YES, a negative bias leans NO.
- biased
- A biased model works clearly worse for some groups than for others, usually because of what was (and wasn’t) in its training data. Not the same thing as a neuron’s bias.
- bigram
- A language model that predicts the next symbol from only the one symbol before it, using a table of counts.
- blame
- This book's nickname for a gradient: how fast the loss goes up when one number goes up a tiny bit.
- BPE
- Short for byte-pair encoding: build a token dictionary by repeatedly gluing together the most common pair of neighbors.
- byte
- The unit a computer stores text in: a number from 0 to 255. A plain English letter is one byte; most emoji take four.
- byte-pair encoding
- A counting recipe for building a token dictionary: start from single characters and keep gluing together whichever pair of neighbors appears most often.
- center of mass
- The balance point of a shape: the spot where it would balance on a fingertip.
- chain rule
- When one thing changes a second and the second changes a third, the overall rate is the two rates multiplied. Backpropagation uses it link by link.
- class
- One of the possible answers a network can choose between, like fruit, chips or cookie.
- computation graph
- The web of notes a framework keeps during the forward pass: every number, which numbers it was made from, and how.
- conditioning
- Giving a network extra information, such as a label or a text prompt, to steer what it makes.
- context window
- The most tokens a model can read at one time. Anything that does not fit in it, the model cannot see.
- controlled experiment
- A test where you change one thing and keep everything else the same, so you know what caused the difference.
- convolution
- Sliding a small filter across a picture and, at every stop, multiplying the pixels under it by the filter’s weights and adding them up.
- convolutional layer
- A layer made of several filters. Each one slides over the same input and produces its own feature map.
- convolutional network
- A network built from layers of filters that slide across a picture, usually with pooling in between. Often shortened to CNN.
- credit assignment
- Working out how much each weight contributed to a mistake, so that each one can be corrected by the right amount.
- cross-entropy
- A loss for networks that output probabilities: minus the log of the probability the model gave to the right answer, so confident mistakes cost the most.
- cutoff date
- The day a model’s training data ends. It has seen nothing that happened after it.
- data augmentation
- Training on deliberately altered copies of the examples, such as shifted or rotated pictures, so the network learns to ignore those changes.
- dead neuron
- A ReLU neuron that outputs 0 for every input. No blame flows back through it, so it never learns again.
- decision boundary
- The edge where a model’s answer flips from NO to YES. For a single neuron it is always one straight line.
- deep Q-network
- A neural network that does the job of a Q-table: the state goes in and an estimated value for each action comes out. Often shortened to DQN.
- derivative
- The slope of a curve at one point: how fast the output changes when the input changes a tiny bit.
- diffusion model
- A network that makes a picture by starting from pure noise and removing a little of it at a time. The idea behind the AI image generators that appeared in 2022.
- discount factor
- A number between 0 and 1, written γ, that says how much a reward one step in the future is worth compared with the same reward right now.
- dot product
- Multiply the matching entries of two lists, then add them all up — the same multiply-and-add a neuron does.
- dropout
- A training trick that switches off a random handful of neurons on every step, so the network cannot rely on any single neuron to memorize an example.
- early stopping
- Ending training at the moment the score on held-back data is at its best, before the network starts memorizing.
- embedding
- The row of learned numbers that a token id looks up. The id is only a row number; the meaning is in the row.
- embedding table
- A grid of learned numbers with one row for every token in the dictionary. A token's id says which row to read.
- end marker
- An extra symbol in the dictionary that means the text stops here. When the model samples it, the name is finished.
- end-of-text
- A special token that marks where a document stops. When a chatbot picks it, the reply is finished.
- episode
- One complete attempt in reinforcement learning, from the starting position until the goal is reached or time runs out.
- epoch
- One full pass through all of the training examples.
- exploitation
- Taking the action you currently believe is best, instead of trying something new.
- exploration
- Trying an action you are not sure about, to find out what happens.
- false alarm
- The model said yes and the truth was no. Also called a false positive.
- feature map
- The grid of numbers a filter produces: big where its pattern was found in the picture, near zero where it was not.
- filter
- In a network that looks at pictures: a small grid of weights that is slid across the picture to detect one pattern, such as an edge. Also called a kernel.
- fine-tuning
- Training an already-trained model some more on a smaller, special set of examples to change how it behaves.
- flatten
- To unroll a grid of numbers into one long list, row after row, so it can be fed to a layer of neurons.
- forward pass
- Running inputs through the network to produce an output. No learning happens yet.
- framework
- A toolkit for building and training neural networks. Its main trick is working out every gradient automatically.
- fully connected
- A layer where every neuron is wired to every input. It treats the inputs as a plain list and has no idea which ones are neighbors.
- generalize
- To do well on examples that were not in the training data. It is the whole point of learning a rule instead of memorizing answers.
- GPU
- A graphics chip with thousands of simple cores that can do huge numbers of small sums at the same time, which is what neural networks need.
- gradient
- The direction (and steepness) telling you how to change each weight to make the error smaller.
- gradient check
- Testing backpropagation by wiggling one weight a tiny bit, measuring how far the loss really moves, and comparing the two answers.
- gradient descent
- The learning rule: find the slope of the loss for every weight, then move each weight a small step in the downhill direction.
- hallucination
- When a language model states something false as if it were a fact, like a quote nobody said or a book that was never written.
- held-out names
- Names set aside before training and never trained on, used to test the model on names it has not seen.
- A layer of neurons in the middle of a network, between the inputs and the output. Nobody tells it what to mean.
- A neuron in a middle layer — not an input, not the final output. Where the network builds its own features.
- hyperparameter
- A setting chosen by the person building the network before training starts, like the learning rate or the number of hidden neurons.
- interpretability
- The science of opening up a trained network to find out what it has learned and how it gets its answers.
- kernel
- Another name for a filter: the small grid of weights a convolutional layer slides across a picture.
- key (in attention)
- The list of numbers a token wears as its label. When a query points the same way as a key, the match score is high.
- label noise
- Wrong answers hiding in the training data, like typos in an answer key.
- language model
- A model that looks at the text so far and gives a probability to every possible next token. Run in a loop, it writes.
- large language model
- A very big transformer trained on text to predict the next token. ChatGPT, Claude and Gemini are built on them.
- latent
- A compressed version of a picture: a much smaller grid of numbers that a network can turn back into pixels.
- learning rate
- How big a step to take when nudging the weights. Too big overshoots; too small crawls.
- linearly separable
- One straight line can split the YES examples from the NO examples. OR and AND are; XOR is not.
- LLM
- Short for large language model: a very big network trained on text to predict the next token.
- local minimum
- A dip in the loss landscape that is lower than everything right next to it but is not the lowest point overall.
- logit
- A raw score a network gives to one possible answer, before softmax turns the scores into probabilities.
- loss
- A single number measuring how wrong the network is right now. Training tries to make it small.
- loss landscape
- A picture of the loss for every possible setting of the weights: hills where the network is wrong, valleys where it does well.
- match score
- In attention, the dot product of one token's query with another token's key. Higher means listen more.
- matrix
- A grid of numbers — a list of lists. One matrix multiply does many weighted sums at once.
- max pooling
- Pooling that keeps only the biggest number in each block.
- mean squared error
- A score for how wrong a model is overall: square each example’s error, then average them. Often shortened to MSE.
- mini-batch
- A small handful of training examples used to work out one quick, roughly right gradient step.
- MNIST
- A famous collection of 70,000 handwritten digits, each 28 × 28 pixels, used to test image-reading networks since the 1990s.
- momentum
- Gradient descent with a memory: each step keeps most of the previous step's velocity, so steady slopes build up speed.
- multimodal
- Able to take in more than one kind of input, such as text, pictures and sound, in a single model.
- neuron
- The tiny machine: multiply each input by a weight, add the bias, then squish with an activation. One number comes out.
- noise
- Randomness with no pattern in it, like the static on an old TV tuned to no channel.
- noise level
- How much noise has been mixed into a picture. In this book's demo, level 0 is the clean picture and level 50 is almost nothing but noise.
- one-hot encoding
- Writing a choice as a list with a single 1 and the rest 0s, like [0,1,0] for "chips".
- open weights
- A trained model whose numbers have been published as a file, so that anyone can download it and run it on their own computer.
- optimizer
- The part of a framework that nudges the weights once their gradients are known. Adam is the usual choice.
- overfitting
- When a model memorizes the training examples instead of learning the general rule.
- padding
- In a convolutional network: a border of zeros added around a picture so the feature map comes out the same size as the picture.
- parameter
- Any number that training is allowed to change: every weight and every bias.
- perceptron
- An early learning machine from 1958: a single layer of neurons that adjusts its weights from examples.
- perplexity
- A loss turned into a number of choices by taking e to the power of the loss. A perplexity of 9 means the model is as unsure as if it were picking among 9 equally likely options.
- pixel
- One tiny square of a picture. To a computer it is just a number saying how bright (or how inky) that square is.
- policy
- An agent’s rule for which action to take in every state. In the maze it is the map of arrows.
- pooling
- Shrinking a feature map by replacing each small block of numbers with a single number.
- position signal
- Extra numbers added to each token that say where it sits in the text, because attention on its own ignores order. Engineers call it a positional encoding.
- pre-training
- The first and biggest stage of training a language model: predicting the next token across an enormous pile of text.
- precision
- Of everything the model flagged as a yes, the share that really was one. False alarms lower it.
- preprocessing
- Tidying raw data into the shape a model expects before the model sees it, the same way for training and for real use.
- prompt
- What you give a generative model to tell it what you want: a question for a chatbot, a description for an image generator.
- prompt injection
- Text hidden in a web page, file or tool result that is written to look like instructions, in the hope that a model reading it will obey.
- PyTorch
- The framework most AI researchers use to build and train neural networks, written mostly for Python.
- Q-learning
- A reinforcement learning method that keeps a value for every action in every state and corrects one value after each move, using the reward and the best value of the state it landed in.
- Q-table
- The table a Q-learning agent fills in: one Q-value for every action in every state.
- Q-value
- An agent’s current estimate of the total reward that one action, taken from one state, will lead to in the end.
- quantization
- Storing each weight with fewer bits, such as 4 instead of 16. The model file gets much smaller and its answers usually get only slightly worse.
- query
- In attention, the list of numbers a token uses to ask its question: here is what I am looking for.
- random seed
- The starting number a computer uses to make its random numbers. The same seed gives the same random numbers again, which lets you repeat an experiment exactly.
- reasoning model
- A language model trained to write out its working before it answers. The working is ordinary tokens that the model then reads back, like scratch paper.
- recall
- Of all the real yeses, the share the model caught. Missing a real one lowers it.
- reinforcement learning
- Learning by trial and error. An agent tries actions, gets rewards, and gradually works out which actions lead to more reward. Nobody shows it the right answer.
- ReLU
- A simple activation: keep positive numbers as-is, turn negatives into 0. Cheap to compute and the modern default.
- residual connection
- A shortcut that lets a signal skip past a layer, which keeps gradients alive in very deep networks.
- retrieval
- Searching a pile of documents for passages related to a question and pasting them into the context window, so the model can answer from text in front of it. Short for retrieval-augmented generation, or RAG.
- reward
- The number an agent gets back right after an action: points gained or lost. In reinforcement learning it is the only feedback there is.
- reward hacking
- When an agent finds a way to collect lots of reward that is not what its designers meant.
- RLHF
- Reinforcement learning from human feedback: people compare answers, and the model is nudged toward the kind they prefer.
- sampling
- Choosing the next token at random, with likelier tokens chosen more often, instead of always taking the top one.
- saturated
- Describes a sigmoid neuron pushed so far to one end that its curve is almost flat, so it barely learns.
- scaling law
- A measured rule for how a model's performance improves as its size, data and computing grow.
- self-play
- Training a game-playing agent by having it play against copies of itself.
- shortcut learning
- When a model learns an easy pattern that happens to go along with the right answer in its training data, instead of the real reason for the answer.
- sigmoid
- A curve that squishes any number into the 0–1 range, so the output reads as a confidence.
- softmax
- Turns a list of scores into probabilities that add up to 1 — used when there are several possible answers.
- special token
- A token that stands for a signal instead of text, such as end-of-text or the start of a speaker's turn.
- squared error
- A simple loss: the gap between the truth and the prediction, multiplied by itself, so bigger misses count much more.
- standard deviation
- A measure of how spread out a set of numbers is around their average.
- standardize
- To rescale a column of inputs so its average is 0 and its spread is about 1, which puts every input on an equal footing.
- state (in reinforcement learning)
- The situation an agent is in right now, as far as it can tell. In the maze it is the cell the creature is standing on.
- stochastic gradient descent
- Gradient descent that takes each step using a random mini-batch instead of the whole training set.
- System One model
- A model built to give a fast, typed decision (a yes or no, a choice, or a score) in a single pass through its network, instead of writing text.
- system prompt
- Instructions placed at the top of the context window by whoever built the app. They set the assistant's role and rules, and the user usually cannot see them.
- tanh
- An activation shaped like sigmoid that outputs between −1 and +1 instead of 0 and 1.
- temperature
- A sampling knob that divides every score before softmax. Low makes the favorite win almost every time; high evens out the odds.
- tensor
- A grid of numbers (a list, a table, or a stack of tables) that a framework does math on, and keeps notes on, as one unit.
- test set
- Examples held back and never trained on, used once at the very end to measure what the network really learned.
- threshold
- The line a score has to reach before the model’s answer counts as a yes. Moving it trades one kind of mistake for another.
- token
- A chunk of text, often a piece of a word, that has its own id number. Language models read and write tokens, not letters.
- tokenizer
- The tool that chops text into tokens and swaps each one for its id, and turns ids back into text. Its dictionary is built before training and never changes.
- tool call
- A request, written by a model in a fixed format, asking the app around it to do something the model cannot: run a search, do a calculation, open a file.
- top-k
- A sampling knob: only the k highest-scoring tokens may be picked, and the rest are cut.
- top-p
- A sampling knob: keep the likeliest tokens until their probabilities add up to p, and cut the rest. Also called nucleus sampling.
- training set
- The examples a network learns from. Its homework.
- transformer
- The network design inside modern language models: dozens of stacked layers, each one attention followed by ordinary neurons.
- truth table
- A table that lists every possible combination of inputs together with the answer for each one.
- universal approximation theorem
- A proof that one hidden layer can fit almost any shape if it is wide enough. It says nothing about how to find the weights.
- validation set
- A third pile of held-back examples, used for making choices during training (such as when to stop) so that the test set can stay unseen.
- value (in attention)
- The list of numbers a token hands over when it is listened to. The output of attention is a weighted blend of values.
- vanishing gradient
- When the blame shrinks at every layer on its way back through a deep network, so the earliest layers barely learn.
- vector
- Just a list of numbers, like [0.2, −1.4, 0.7].
- vocabulary
- The full list of tokens a model knows, each with its own id. This book mostly calls it the dictionary.
- weight
- An adjustable “importance” number on a connection. Training nudges it up or down so the network gets more answers right.
- weight decay
- A training trick that shrinks every weight a tiny bit on each step, which discourages the extreme weights that memorizing needs.
- weight sharing
- Using the same weights at every position of a picture, so a pattern learned in one place is recognized everywhere.
- weighted blend
- A mix where each ingredient is multiplied by its share and the results are added up. The shares add up to 1.
- weighted sum
- Every input times its weight, all added together (plus the bias). The single number that feeds into the activation.
- XOR
- “Exclusive or”: YES when exactly one of two inputs is on. The classic rule that a single neuron cannot learn.