Neuro 13
0 / 37 chapters finished

Glossary

Every special word in the book, in plain language. In the chapters, words with a dotted underline show these same definitions when you hover or tap.

accuracy
The share of answers a model got right. Handy, but one accuracy number can hide who the model fails, and it means little when the thing you are looking for is rare.
action
One of the things an agent can choose to do. In the maze there are four: up, right, down and left.
activation
The “squisher” applied after the weighted sum — sigmoid, ReLU, and friends. It bends the line so the network can learn curves.
Adam
The most widely used training method: momentum plus an automatic step size for every weight.
agent
The one doing the acting. In reinforcement learning it is the learner: it looks at its situation, picks an action and collects the reward.
AI agent
A language model run in a loop with tools: it asks for an action, reads what came back, and decides what to do next until the job is done.
AI winter
A stretch of years when excitement about AI, and the money for it, drains away.
alignment
The work of making sure an AI system does what people actually meant, and not only what its training happened to reward.
architecture
The shape of a network: how many layers it has and how many neurons are in each one.
attention
How a token gathers information from other tokens: it scores each one for how useful it looks, then takes a blend that leans toward the highest scorers.
attention head
One set of query, key and value weights. A transformer layer runs several heads side by side, each free to look for something different.
attention weight
How much one token listens to another, from 0 to 1. A token's attention weights over all the others add up to 1.
automatic differentiation
Getting gradients without working out formulas by hand: the computer keeps notes during the forward pass and reads them backwards.
backpropagation
The bookkeeping that sends the error backward through the network to find each weight’s gradient.
backward pass
The second half of a training round: start at the loss and hand the blame back through the network, layer by layer.
base model
A language model straight out of pre-training. It continues documents and has not yet been taught to act as an assistant.
baseline
The score you would get with no skill at all, such as by guessing. A result only means something compared with it.
batch size
How many training examples the network looks at before each nudge of its weights.
bias
A neuron’s built-in lean — how eager it is to fire before any input arrives. A positive bias leans YES, a negative bias leans NO.
biased
A biased model works clearly worse for some groups than for others, usually because of what was (and wasn’t) in its training data. Not the same thing as a neuron’s bias.
bigram
A language model that predicts the next symbol from only the one symbol before it, using a table of counts.
blame
This book's nickname for a gradient: how fast the loss goes up when one number goes up a tiny bit.
BPE
Short for byte-pair encoding: build a token dictionary by repeatedly gluing together the most common pair of neighbors.
byte
The unit a computer stores text in: a number from 0 to 255. A plain English letter is one byte; most emoji take four.
byte-pair encoding
A counting recipe for building a token dictionary: start from single characters and keep gluing together whichever pair of neighbors appears most often.
center of mass
The balance point of a shape: the spot where it would balance on a fingertip.
chain rule
When one thing changes a second and the second changes a third, the overall rate is the two rates multiplied. Backpropagation uses it link by link.
class
One of the possible answers a network can choose between, like fruit, chips or cookie.
computation graph
The web of notes a framework keeps during the forward pass: every number, which numbers it was made from, and how.
conditioning
Giving a network extra information, such as a label or a text prompt, to steer what it makes.
context window
The most tokens a model can read at one time. Anything that does not fit in it, the model cannot see.
controlled experiment
A test where you change one thing and keep everything else the same, so you know what caused the difference.
convolution
Sliding a small filter across a picture and, at every stop, multiplying the pixels under it by the filter’s weights and adding them up.
convolutional layer
A layer made of several filters. Each one slides over the same input and produces its own feature map.
convolutional network
A network built from layers of filters that slide across a picture, usually with pooling in between. Often shortened to CNN.
credit assignment
Working out how much each weight contributed to a mistake, so that each one can be corrected by the right amount.
cross-entropy
A loss for networks that output probabilities: minus the log of the probability the model gave to the right answer, so confident mistakes cost the most.
cutoff date
The day a model’s training data ends. It has seen nothing that happened after it.
data augmentation
Training on deliberately altered copies of the examples, such as shifted or rotated pictures, so the network learns to ignore those changes.
dead neuron
A ReLU neuron that outputs 0 for every input. No blame flows back through it, so it never learns again.
decision boundary
The edge where a model’s answer flips from NO to YES. For a single neuron it is always one straight line.
deep Q-network
A neural network that does the job of a Q-table: the state goes in and an estimated value for each action comes out. Often shortened to DQN.
derivative
The slope of a curve at one point: how fast the output changes when the input changes a tiny bit.
diffusion model
A network that makes a picture by starting from pure noise and removing a little of it at a time. The idea behind the AI image generators that appeared in 2022.
discount factor
A number between 0 and 1, written γ, that says how much a reward one step in the future is worth compared with the same reward right now.
dot product
Multiply the matching entries of two lists, then add them all up — the same multiply-and-add a neuron does.
dropout
A training trick that switches off a random handful of neurons on every step, so the network cannot rely on any single neuron to memorize an example.
early stopping
Ending training at the moment the score on held-back data is at its best, before the network starts memorizing.
embedding
The row of learned numbers that a token id looks up. The id is only a row number; the meaning is in the row.
embedding table
A grid of learned numbers with one row for every token in the dictionary. A token's id says which row to read.
end marker
An extra symbol in the dictionary that means the text stops here. When the model samples it, the name is finished.
end-of-text
A special token that marks where a document stops. When a chatbot picks it, the reply is finished.
episode
One complete attempt in reinforcement learning, from the starting position until the goal is reached or time runs out.
epoch
One full pass through all of the training examples.
exploitation
Taking the action you currently believe is best, instead of trying something new.
exploration
Trying an action you are not sure about, to find out what happens.
false alarm
The model said yes and the truth was no. Also called a false positive.
feature map
The grid of numbers a filter produces: big where its pattern was found in the picture, near zero where it was not.
filter
In a network that looks at pictures: a small grid of weights that is slid across the picture to detect one pattern, such as an edge. Also called a kernel.
fine-tuning
Training an already-trained model some more on a smaller, special set of examples to change how it behaves.
flatten
To unroll a grid of numbers into one long list, row after row, so it can be fed to a layer of neurons.
forward pass
Running inputs through the network to produce an output. No learning happens yet.
framework
A toolkit for building and training neural networks. Its main trick is working out every gradient automatically.
fully connected
A layer where every neuron is wired to every input. It treats the inputs as a plain list and has no idea which ones are neighbors.
generalize
To do well on examples that were not in the training data. It is the whole point of learning a rule instead of memorizing answers.
GPU
A graphics chip with thousands of simple cores that can do huge numbers of small sums at the same time, which is what neural networks need.
gradient
The direction (and steepness) telling you how to change each weight to make the error smaller.
gradient check
Testing backpropagation by wiggling one weight a tiny bit, measuring how far the loss really moves, and comparing the two answers.
gradient descent
The learning rule: find the slope of the loss for every weight, then move each weight a small step in the downhill direction.
hallucination
When a language model states something false as if it were a fact, like a quote nobody said or a book that was never written.
held-out names
Names set aside before training and never trained on, used to test the model on names it has not seen.
hidden layer
A layer of neurons in the middle of a network, between the inputs and the output. Nobody tells it what to mean.
hidden neuron
A neuron in a middle layer — not an input, not the final output. Where the network builds its own features.
hyperparameter
A setting chosen by the person building the network before training starts, like the learning rate or the number of hidden neurons.
interpretability
The science of opening up a trained network to find out what it has learned and how it gets its answers.
kernel
Another name for a filter: the small grid of weights a convolutional layer slides across a picture.
key (in attention)
The list of numbers a token wears as its label. When a query points the same way as a key, the match score is high.
label noise
Wrong answers hiding in the training data, like typos in an answer key.
language model
A model that looks at the text so far and gives a probability to every possible next token. Run in a loop, it writes.
large language model
A very big transformer trained on text to predict the next token. ChatGPT, Claude and Gemini are built on them.
latent
A compressed version of a picture: a much smaller grid of numbers that a network can turn back into pixels.
learning rate
How big a step to take when nudging the weights. Too big overshoots; too small crawls.
linearly separable
One straight line can split the YES examples from the NO examples. OR and AND are; XOR is not.
LLM
Short for large language model: a very big network trained on text to predict the next token.
local minimum
A dip in the loss landscape that is lower than everything right next to it but is not the lowest point overall.
logit
A raw score a network gives to one possible answer, before softmax turns the scores into probabilities.
loss
A single number measuring how wrong the network is right now. Training tries to make it small.
loss landscape
A picture of the loss for every possible setting of the weights: hills where the network is wrong, valleys where it does well.
match score
In attention, the dot product of one token's query with another token's key. Higher means listen more.
matrix
A grid of numbers — a list of lists. One matrix multiply does many weighted sums at once.
max pooling
Pooling that keeps only the biggest number in each block.
mean squared error
A score for how wrong a model is overall: square each example’s error, then average them. Often shortened to MSE.
mini-batch
A small handful of training examples used to work out one quick, roughly right gradient step.
MNIST
A famous collection of 70,000 handwritten digits, each 28 × 28 pixels, used to test image-reading networks since the 1990s.
momentum
Gradient descent with a memory: each step keeps most of the previous step's velocity, so steady slopes build up speed.
multimodal
Able to take in more than one kind of input, such as text, pictures and sound, in a single model.
neuron
The tiny machine: multiply each input by a weight, add the bias, then squish with an activation. One number comes out.
noise
Randomness with no pattern in it, like the static on an old TV tuned to no channel.
noise level
How much noise has been mixed into a picture. In this book's demo, level 0 is the clean picture and level 50 is almost nothing but noise.
one-hot encoding
Writing a choice as a list with a single 1 and the rest 0s, like [0,1,0] for "chips".
open weights
A trained model whose numbers have been published as a file, so that anyone can download it and run it on their own computer.
optimizer
The part of a framework that nudges the weights once their gradients are known. Adam is the usual choice.
overfitting
When a model memorizes the training examples instead of learning the general rule.
padding
In a convolutional network: a border of zeros added around a picture so the feature map comes out the same size as the picture.
parameter
Any number that training is allowed to change: every weight and every bias.
perceptron
An early learning machine from 1958: a single layer of neurons that adjusts its weights from examples.
perplexity
A loss turned into a number of choices by taking e to the power of the loss. A perplexity of 9 means the model is as unsure as if it were picking among 9 equally likely options.
pixel
One tiny square of a picture. To a computer it is just a number saying how bright (or how inky) that square is.
policy
An agent’s rule for which action to take in every state. In the maze it is the map of arrows.
pooling
Shrinking a feature map by replacing each small block of numbers with a single number.
position signal
Extra numbers added to each token that say where it sits in the text, because attention on its own ignores order. Engineers call it a positional encoding.
pre-training
The first and biggest stage of training a language model: predicting the next token across an enormous pile of text.
precision
Of everything the model flagged as a yes, the share that really was one. False alarms lower it.
preprocessing
Tidying raw data into the shape a model expects before the model sees it, the same way for training and for real use.
prompt
What you give a generative model to tell it what you want: a question for a chatbot, a description for an image generator.
prompt injection
Text hidden in a web page, file or tool result that is written to look like instructions, in the hope that a model reading it will obey.
PyTorch
The framework most AI researchers use to build and train neural networks, written mostly for Python.
Q-learning
A reinforcement learning method that keeps a value for every action in every state and corrects one value after each move, using the reward and the best value of the state it landed in.
Q-table
The table a Q-learning agent fills in: one Q-value for every action in every state.
Q-value
An agent’s current estimate of the total reward that one action, taken from one state, will lead to in the end.
quantization
Storing each weight with fewer bits, such as 4 instead of 16. The model file gets much smaller and its answers usually get only slightly worse.
query
In attention, the list of numbers a token uses to ask its question: here is what I am looking for.
random seed
The starting number a computer uses to make its random numbers. The same seed gives the same random numbers again, which lets you repeat an experiment exactly.
reasoning model
A language model trained to write out its working before it answers. The working is ordinary tokens that the model then reads back, like scratch paper.
recall
Of all the real yeses, the share the model caught. Missing a real one lowers it.
reinforcement learning
Learning by trial and error. An agent tries actions, gets rewards, and gradually works out which actions lead to more reward. Nobody shows it the right answer.
ReLU
A simple activation: keep positive numbers as-is, turn negatives into 0. Cheap to compute and the modern default.
residual connection
A shortcut that lets a signal skip past a layer, which keeps gradients alive in very deep networks.
retrieval
Searching a pile of documents for passages related to a question and pasting them into the context window, so the model can answer from text in front of it. Short for retrieval-augmented generation, or RAG.
reward
The number an agent gets back right after an action: points gained or lost. In reinforcement learning it is the only feedback there is.
reward hacking
When an agent finds a way to collect lots of reward that is not what its designers meant.
RLHF
Reinforcement learning from human feedback: people compare answers, and the model is nudged toward the kind they prefer.
sampling
Choosing the next token at random, with likelier tokens chosen more often, instead of always taking the top one.
saturated
Describes a sigmoid neuron pushed so far to one end that its curve is almost flat, so it barely learns.
scaling law
A measured rule for how a model's performance improves as its size, data and computing grow.
self-play
Training a game-playing agent by having it play against copies of itself.
shortcut learning
When a model learns an easy pattern that happens to go along with the right answer in its training data, instead of the real reason for the answer.
sigmoid
A curve that squishes any number into the 0–1 range, so the output reads as a confidence.
softmax
Turns a list of scores into probabilities that add up to 1 — used when there are several possible answers.
special token
A token that stands for a signal instead of text, such as end-of-text or the start of a speaker's turn.
squared error
A simple loss: the gap between the truth and the prediction, multiplied by itself, so bigger misses count much more.
standard deviation
A measure of how spread out a set of numbers is around their average.
standardize
To rescale a column of inputs so its average is 0 and its spread is about 1, which puts every input on an equal footing.
state (in reinforcement learning)
The situation an agent is in right now, as far as it can tell. In the maze it is the cell the creature is standing on.
stochastic gradient descent
Gradient descent that takes each step using a random mini-batch instead of the whole training set.
System One model
A model built to give a fast, typed decision (a yes or no, a choice, or a score) in a single pass through its network, instead of writing text.
system prompt
Instructions placed at the top of the context window by whoever built the app. They set the assistant's role and rules, and the user usually cannot see them.
tanh
An activation shaped like sigmoid that outputs between −1 and +1 instead of 0 and 1.
temperature
A sampling knob that divides every score before softmax. Low makes the favorite win almost every time; high evens out the odds.
tensor
A grid of numbers (a list, a table, or a stack of tables) that a framework does math on, and keeps notes on, as one unit.
test set
Examples held back and never trained on, used once at the very end to measure what the network really learned.
threshold
The line a score has to reach before the model’s answer counts as a yes. Moving it trades one kind of mistake for another.
token
A chunk of text, often a piece of a word, that has its own id number. Language models read and write tokens, not letters.
tokenizer
The tool that chops text into tokens and swaps each one for its id, and turns ids back into text. Its dictionary is built before training and never changes.
tool call
A request, written by a model in a fixed format, asking the app around it to do something the model cannot: run a search, do a calculation, open a file.
top-k
A sampling knob: only the k highest-scoring tokens may be picked, and the rest are cut.
top-p
A sampling knob: keep the likeliest tokens until their probabilities add up to p, and cut the rest. Also called nucleus sampling.
training set
The examples a network learns from. Its homework.
transformer
The network design inside modern language models: dozens of stacked layers, each one attention followed by ordinary neurons.
truth table
A table that lists every possible combination of inputs together with the answer for each one.
universal approximation theorem
A proof that one hidden layer can fit almost any shape if it is wide enough. It says nothing about how to find the weights.
validation set
A third pile of held-back examples, used for making choices during training (such as when to stop) so that the test set can stay unseen.
value (in attention)
The list of numbers a token hands over when it is listened to. The output of attention is a weighted blend of values.
vanishing gradient
When the blame shrinks at every layer on its way back through a deep network, so the earliest layers barely learn.
vector
Just a list of numbers, like [0.2, −1.4, 0.7].
vocabulary
The full list of tokens a model knows, each with its own id. This book mostly calls it the dictionary.
weight
An adjustable “importance” number on a connection. Training nudges it up or down so the network gets more answers right.
weight decay
A training trick that shrinks every weight a tiny bit on each step, which discourages the extreme weights that memorizing needs.
weight sharing
Using the same weights at every position of a picture, so a pattern learned in one place is recognized everywhere.
weighted blend
A mix where each ingredient is multiplied by its share and the results are added up. The shares add up to 1.
weighted sum
Every input times its weight, all added together (plus the bias). The single number that feeds into the activation.
XOR
“Exclusive or”: YES when exactly one of two inputs is on. The classic rule that a single neuron cannot learn.