Lesson 1 · Foundations

The Artificial Neuron

An artificial neuron takes several numerical inputs, multiplies each by a weight, adds a bias, and passes the result through a nonlinear function called an activation function. That's it. Most of the computation in a neural network — including GPT-2 — is this one repeated operation; the transformer adds a few others (attention, which multiplies activations together, softmax and LayerNorm), covered in Lesson 2.

The weight on each input encodes how much that input should influence the output, and its sign encodes whether the influence is excitatory or inhibitory — an analogy to biological synaptic strength that is more than superficial.

Computation z = w₁·x₁ + w₂·x₂ + w₃·x₃ + b
output = ReLU(z) = max(0, z)

Neuroscience parallel. The weighted sum z is a linear integration of inputs — analogous to dendritic summation. The nonlinearity (ReLU here) models the threshold-like firing behavior: no output below threshold, linear output above it. Unlike a real neuron, there is no refractory period, no spike timing, and no temporal dynamics — just an instantaneous scalar transformation.

Inputs

x₁ 1.00

x₂ 0.50

x₃ −1.00

Weights

w₁ 1.50

w₂ 0.80

w₃ −0.50

Bias

b 0.20

Contribution of each term (wᵢxᵢ)

w₁x₁

w₂x₂

w₃x₃

Pre-activation z

—

weighted sum + bias

Output ReLU(z)

—

Try pulling w₁ negative while keeping x₁ positive — you'll see that input actively suppress the output. Try making all inputs large and positive but setting the bias to a large negative value — the neuron requires a strong collective signal before it fires, like a high-threshold cell.

Critical point. A single neuron can only compute a linear decision boundary — it fires or not based on whether a hyperplane in input space is crossed. All the expressive power of neural networks comes from stacking many of these, with nonlinearities in between. That's next.

Activation Functions — the nonlinearity

The activation function is what separates neural networks from simple linear algebra. Without it, a stack of layers collapses to a single matrix multiplication — no matter how deep the network. The activation function breaks this, allowing the network to carve curved, non-linear boundaries through input space.

Different functions have different tradeoffs. ReLU is the workhorse of many modern networks and was used in the original Transformer's MLP layers. Sigmoid squashes output to (0,1) and was historically popular for gating and output probabilities. GELU (used in GPT-2 and GPT-3) is a smooth approximation to ReLU with a slight negative lobe; most current LLMs use SwiGLU.

Definitions ReLU(z) = max(0, z)
GELU(z) ≈ z · σ(1.702z) where σ is sigmoid
Sigmoid(z) = 1 / (1 + e^−z)
Tanh(z) = (e^z − e^−z) / (e^z + e^−z)

Why ReLU became dominant. It is cheap to compute, its gradient is either 0 or 1 (no saturation for positive inputs), and it induces natural sparsity — many neurons output exactly zero. Modern LLMs use variants like GELU or SwiGLU, which are smoother and slightly outperform ReLU empirically.

Neuroscience parallel. The sigmoid was originally motivated by the sigmoidal shape of neural firing-rate curves (f-I curves). ReLU is a better fit for a linear-threshold model. Neither captures spiking, adaptation, or neuromodulation — but both are far cheaper to differentiate through.

The "linear" option shows what happens with no nonlinearity — the curve is just the identity. A network of these, no matter how deep, computes only linear functions. All representation power is lost.

Neural Networks

A layer is just many neurons receiving the same input vector and each producing one output — their outputs become the next layer's input. A feedforward network stacks these layers left to right. Every layer applies: output = activation(W · input + b) where W is a weight matrix and b is a bias vector.

The number of neurons per layer is the layer's width. The number of layers is the depth. Both matter: width controls how much information can be represented at once; depth controls how many sequential transformations are applied.

a. A toy feedforward network with two layers of weights, each 4 units wide: every unit of one layer connects to every unit of the next, and each layer adds a bias per unit. b. GPT-2 small at the same level of description: tokens are embedded as 768-dimensional vectors that pass through 12 blocks, each adding the outputs of an attention layer and an MLP; the final vector is mapped to a probability for each of the 50,257 tokens. Each MLP is the same kind of two-layer network as in a, 768 → 3,072 → 768 units wide.

The parameter count grows quadratically with width. A layer of width 512 (512 inputs → 512 outputs) has 512 × 512 = 262,144 weights, plus 512 biases. GPT-2 XL has 1.5 billion such parameters across 48 layers with width 1,600 (GPT-2 small: 124 million, 12 layers, width 768). Fig. 3 compares the toy network's 40 parameters with that scale.

Layer operation (vectorized) h^(l) = activation( W^(l) · h^(l−1) + b^(l) )
W^(l) ∈ ℝ^d_out×d_in, b^(l) ∈ ℝ^d_out

Note: from Lesson 2 on, the transformer lessons write the same operation with row vectors, h W + b with W ∈ ℝ^d_in×d_out (the convention of GPT-2's code). The two forms are transposes of each other.

Reading the equation. This is just the single-neuron formula applied to an entire layer at once. A layer has many neurons, all reading the same input vector h^(l−1). Instead of computing each neuron's dot product separately, stack all their weight rows into a matrix W^(l) and do one matrix–vector multiply:

h^(l−1) — input vector (activations from previous layer), length d_in
W^(l) · h^(l−1) — matrix multiply: d_out rows × d_in columns times d_in vector → d_out pre-activations z, one per output neuron
b^(l) — bias vector; one scalar added to each neuron's z
activation(…) — applied element-wise (ReLU, GELU, SiLU…) → final output h^(l)
Superscript (l) — layer index; W⁽¹⁾ is layer 1's weights, W⁽²⁾ is layer 2's, etc. (h⁽⁰⁾ is the input). (These superscripts count weight layers from 1, with h⁽⁰⁾ the input; GPT-2's transformer blocks are numbered from 0, Block 0–Block 11.)

GPUs are extremely fast at this matrix multiply, which is why inference can run in milliseconds despite hundreds of millions to billions of parameters (GPT-2 small: 124M; XL: 1.5B).

Neuroscience parallel. A layer is like a cortical area: a population of neurons all receiving the same upstream projection and sending outputs downstream in parallel. Depth maps loosely to the cortical hierarchy — V1→V2→V4→IT in vision, for example. Key difference: feedforward networks have no recurrence within a layer, and in a transformer, "layers" are residual rather than replacing the representation entirely.

What "deep" actually means. The word "deep" in deep learning just means more than one hidden layer. A two-layer network (one hidden layer) can already approximate any continuous function on a bounded input range to arbitrary precision given enough width — but enormous depth is empirically far more parameter-efficient for complex structured tasks.

How Networks Learn

The weights of a network are not programmed — they are optimized. Given a dataset of (input, desired output) pairs, training adjusts every weight so the network's predictions match the targets. The machinery that does this is gradient descent applied to a loss function.

The loss measures how wrong the network currently is — for language modeling, it's cross-entropy between the predicted probability distribution over tokens and the actual next token. The gradient of the loss with respect to every weight tells us which direction to move each weight to reduce the loss. We take a small step in that direction, then repeat — hundreds of thousands of times in a typical LLM training run.

Gradient descent update rule w ← w − η · ∂Loss/∂w

η = learning rate (step size)
∂Loss/∂w = gradient of the loss w.r.t. weight w

The loss landscape below visualizes the loss as a function of two hypothetical weights. Click anywhere to place the current position, then watch gradient descent find a local minimum.

Backpropagation is the algorithm that computes the gradient of the loss with respect to every single weight in the network simultaneously, using the chain rule of calculus. For a network with hundreds of millions of parameters (124M in GPT-2 small) or more, this is computed in one backwards pass through the network — costing about twice as much as one forward pass. Without backprop, training modern networks would be computationally intractable. The full mechanics — chain rule, computational graphs, vanishing gradients — are covered in Lesson 4: Training, tab 3.

Neuroscience parallel 1 — plasticity. Gradient descent is often compared to synaptic plasticity, but the analogy is loose at best. Backprop requires a global error signal to be propagated backwards through the network — the "credit assignment problem." Biological synapses use local Hebbian-style rules, not global error backpropagation, though there are ongoing debates about biologically plausible approximations.

Neuroscience parallel 2 — the connectome. Once training ends, the full set of weights W^(l) and b^(l) is saved as a fixed checkpoint and frozen. Every forward pass during deployment reads the same numbers — nothing about a user's conversation changes them. Think of it as the model's connectome: the complete wiring diagram, fixed at a particular moment. Just as your connectome supports context-dependent, flexible computation without rewiring — different activation patterns, same synapses — a deployed LLM produces wildly different outputs from the same frozen weights by attending to different parts of the input. The key difference: biological synapses are locally plastic (LTP/LTD, neuromodulation, consolidation during sleep), so your connectome shifts slowly over time. A deployed model's weights are genuinely static between training runs — no usage, no feedback, no passage of time alters them.

Notice that with a high learning rate η, gradient descent can overshoot and oscillate. With a very low rate, it converges slowly. Real training uses Adam, an adaptive optimizer that maintains per-weight learning rates — effectively navigating the loss landscape much more efficiently.

Distributed Coding

In a neural network, information is not stored in single neurons — it is distributed across many neurons simultaneously. A concept is represented as a pattern of activations across a population, not a single "grandmother cell."

This is both a strength and a challenge for interpretability. The strength: distributed representations are far more expressive than local ones — d neurons can represent 2^d distinct binary patterns, not just d concepts. The challenge: you cannot read off what a network "knows" by looking at one neuron.

Representational capacity Local code: N neurons → N concepts
Sparse code: N neurons, k active → C(N,k) concepts
Dense code: N neurons → 2^N binary patterns (exponential)

Modern networks appear to use sparse, overcomplete representations — each input activates only a small fraction of a large set of features (directions in activation space), and the number of features exceeds the neuron count. The neurons themselves are not sparse: because many features must share them, individual neurons become polysemantic — responding to many unrelated concepts. This is called superposition, and it is believed to be a key reason interpretability is hard. Lesson 5, tab 3 returns to this in detail, and Lesson 7, tab 8 covers the tool used to untangle it.

Neuroscience parallel. Population coding is a fundamental concept in systems neuroscience — place cells, grid cells, and orientation-selective cells all use distributed population codes, and the geometry of activity patterns in high-dimensional space is directly analogous to what mechanistic interpretability calls "representational geometry." Methods like PCA and RSA — standard tools in systems neuroscience — apply directly to LLM activations.

One note on terminology: population codes span a density spectrum, from fully dense (all neurons respond graded to every stimulus) to fully sparse (only a handful of neurons fire for any given input). In LLMs the sparsity is in features, not neurons: neurons respond densely, but each input is thought to use only a few directions from a much larger dictionary. That is the sparse coding model of primary visual cortex: Olshausen & Field (1996) showed that learning a sparse code for natural images gives V1-like receptive fields, and Olshausen & Field (1997) extended it to an overcomplete dictionary, with more features than input dimensions, kept tractable by sparsity. The direct LLM analogue is the dictionary learned by a sparse autoencoder (SAE) on a layer's activations, covered in Lesson 7, tab 8. One difference matters: in V1 each dictionary element is thought to be a neuron of its own, so cortex has many more neurons than input fibres. An LLM is the reverse: it is thought to hold more features than it has neurons, so features must share neurons (superposition, Lesson 5, tab 3). Only the SAE, an analysis tool, gives each feature a unit of its own.

This is where systems and computational neuroscience expertise is directly relevant. The study of population dynamics, dimensionality, and representational geometry in neural recordings is the same mathematical framework applied to LLM activations. The difference: LLM activations are noiseless, perfectly reproducible, and the "stimulus" is fully controlled text. Every technique in Lesson 7 builds directly on this parallel.

Language as a Probability Distribution

A language model assigns a probability to every possible sequence of text. More usefully, it computes the probability of the next word (or token) given all preceding words. This conditional probability is the central object:

Core objective P(token_t | token₀, token₁, …, token_t−1)

"What is the probability of the next token, given everything so far?"

By the chain rule of probability, the probability of any full sequence factors into a product of these conditional probabilities. A model that computes one also implicitly defines a distribution over all possible texts.

P(The cat sat) =
  P(The) ×
  P(cat | The) ×
  P(sat | The cat)

The widget below shows illustrative conditional distributions over next tokens for different contexts (the numbers are made up to show the shapes, not read from a model). Notice how the context shapes the distribution — some completions concentrate the probability on one token, others spread it over many plausible options and a long tail.

This is not a lookup table. The model doesn't store precomputed probabilities for every possible context — there are more possible sentences than atoms in the universe. Instead, the network computes this distribution on the fly, using its parameters (124 million in GPT-2 small, hundreds of billions in the largest models) to generalize from training data to any context it encounters.

Neuroscience framing. Think of the conditional distribution as the model's "expectation" — the probabilistic prediction a sensory system makes before receiving the next stimulus. Predictive coding frameworks in neuroscience model perception as exactly this: a brain that continuously predicts its next input. The LLM makes this structure explicit and mathematically precise.

The key insight: if you build a model that predicts the next token well across all possible contexts, you have implicitly encoded an enormous amount about language structure, world knowledge, and reasoning — because all of those things constrain what comes next. We'll return to this in tab 10.

Tokenization — the atoms of language

Before a model can process text, the text must be converted into discrete units called tokens. Tokens are not words — they are subword pieces learned from the training corpus using an algorithm called Byte-Pair Encoding (BPE).

BPE starts with individual characters (in GPT-2, individual bytes), then iteratively merges the most frequent adjacent pairs into new tokens. Common words become single tokens (the, is, cat); rarer words are split into pieces (neuro + scient + ist); common prefixes and suffixes become reusable tokens. GPT-2 and Pythia use about 50,000 tokens, GPT-4 about 100,000.

Tokenization pipeline (GPT-2) raw text → [BPE tokenizer] → token ids → model

"Hello world" → ["Hello", " world"] → [15496, 995]
" neuroscientist" → [" neuro", "scient", "ist"] → [7669, 25346, 396]

Try typing scientific terms (neuroscientist, electrophysiology) and watch them split into pieces; some, like neuroscience and hippocampus, are common enough in the training text to be single tokens when preceded by a space, as they usually are mid-sentence (at the very start of the text, without the space, they split). Try common words and see them stay whole. This design choice means the vocabulary is compact (~50k for GPT-2) yet the model can handle any word — even made-up ones.

Tokenization has real consequences. Numbers are often split in small chunks ("1729" → ["17", "29"]), making arithmetic harder. Languages with different scripts tokenize less efficiently — a Chinese character might cost 2–3 tokens where an English word costs 1. Prompt sensitivity is partly a tokenization artifact: "cat" and " cat" (with a leading space) are different tokens.

Neuroscience parallel. Tokenization is a choice of the unit of analysis. Text is already discrete (bytes or characters); the tokenizer decides which chunks count as the basic events the model sees — much as an analysis of neural recordings must decide whether its unit is the single unit, multi-unit activity or the population. The choice of granularity shapes what is easy to see and learn in both cases.

Embeddings — tokens as vectors

Tokens are integers — discrete symbols with no inherent structure. Before the network can process them, each token is converted to a dense vector in a high-dimensional space called an embedding. GPT-2 small uses 768-dimensional embeddings (GPT-2 XL 1,600); the largest models use 12,000 or more.

The embedding matrix E is a learned parameter table with one row per vocabulary token. Looking up token ID i selects row i from this matrix, producing a dense vector e ∈ ℝ^d. The geometry of this space is meaningful — similar tokens tend to end up close together, and some semantic relationships become roughly consistent directions.

Embedding lookup E ∈ ℝ^|V|×d [GPT-2: |V| = 50,257 tokens; d = 768 (small) to 1,600 (XL)]

e = E[id] [select row id → vector e ∈ ℝ^d]

x_t = E[id_t] + P_t [token embedding + positional embedding]

cos(a,b) = (a·b) / (‖a‖·‖b‖) [similarity: 1 = same direction; higher = more similar]

E["king"] − E["man"] + E["woman"] ≈ E["queen"] [word2vec; see below]

Fig. 9 · The Embedding Matrix Lookup

Vocabulary

0 · the

1 · a

2 · cat

3 · dog

⋮

|V|−1

E ∈ ℝ^|V|×d

d dimensions →

→

e_cat ∈ ℝ^d

One learned value
per dimension

Row 2 of E is selected — the entire row becomes the embedding for "cat". All other rows are ignored. In GPT-2 small d = 768, so each embedding holds 768 learned floating-point values. (Toy vocabulary for illustration; in GPT-2, "cat" is row 9,246 and " cat" with a leading space is row 3,797.) The cell intensities here represent schematic learned weights; bright = large positive, dim = near zero.

Fig. 10 · Building the Transformer Input x_t = E[id_t] + P_t

Token embedding

E[id_t] — what token

Positional embedding

P_t — where in sequence

Transformer input x_t

identity + position, fused

Without a positional embedding, "the cat sat" and "sat cat the" produce the same set of token embeddings, and attention by itself treats its inputs as an unordered set — a bidirectional model could not tell the two apart. A causal (left-to-right) model like GPT-2 can still partly infer order, because each position sees a different number of earlier tokens (Haviv et al. 2022), but explicit positions make it much easier. P_t injects position t into every vector before the first attention layer. GPT-2 learns P from data; the original "Attention Is All You Need" (Vaswani et al. 2017) used fixed sinusoidal functions instead. Many recent models (e.g., Pythia, Llama) add no position vector at all: they use rotary position embeddings (RoPE), which rotate the queries and keys inside attention by a position-dependent angle. The element-wise sum x_t = E[id_t] + P_t means each dimension carries a blend of token identity and sequence position.

The schematic 2D map below places ~30 words by hand to show the kind of clustering seen when real embeddings are projected to two dimensions (for example with PCA): points close together stand for tokens the model represents similarly. Hover over any point to see the word, or click a category to highlight a cluster.

Fig. 12 · Linear Structure: king − man + woman ≈ queen (schematic)

The royalty offset (man→king) is nearly identical to (woman→queen). The gender offset (man→woman) is nearly identical to (king→queen), so the four points form a parallelogram. The axes are schematic directions, not principal components. The example comes from word2vec (Mikolov et al. 2013), where similar offsets recur across many analogies and emerge from co-occurrence statistics in training text — they are not hand-coded into the model.

Linear structure in embedding space. One of the most striking findings in embedding research: semantic relationships are encoded as roughly consistent directions. The classic example, from word2vec: vec("king") − vec("man") + vec("woman") ≈ vec("queen"). This geometric regularity is not programmed in — it emerges from training on language statistics. Two caveats: the analogy works only when the input words are excluded from the answers (otherwise the nearest vector is usually "king" itself; Nissim et al. 2020), and GPT-2's representations are anisotropic — they share a common direction, so even unrelated tokens can have clearly positive cosine similarity (Ethayarajh 2019).

Neuroscience parallel. Embeddings are the LLM analogue of a neural population code in a sensory area: each token is represented as a point in a high-dimensional activity space, where proximity encodes similarity. The structure of this space — its dimensionality, curvature, cluster geometry — is precisely what representational similarity analysis (RSA) and PCA-based methods characterize. The tools used on neural data apply here almost without modification.

Next-Token Prediction — the training objective

The model outputs a vector of logits — one real number per token in the vocabulary. These are converted to a probability distribution by the softmax function. During training, the loss is the cross-entropy between this predicted distribution and the true next token (a one-hot target).

Softmax & cross-entropy loss softmax(z)_i = exp(z_i) / Σ_j exp(z_j)

Loss = −log P(true token) = −log softmax(z)_true

Perfect prediction: Loss → 0
Random guess (GPT-2, 50,257 tokens): Loss = ln(50,257) ≈ 10.8

Adjust the logits below to see how softmax converts raw scores to probabilities, and how the temperature parameter sharpens or flattens the distribution.

Temperature controls randomness. Logits are divided by a temperature τ before the softmax. At τ=1, softmax is unchanged. τ < 1 sharpens the distribution — the top token becomes more dominant, output becomes more predictable. τ > 1 flattens it — all tokens become more equally likely, output becomes more random/creative. This is the "temperature" knob exposed in LLM APIs.

Neuroscience parallel. The softmax with temperature is mathematically equivalent to the Boltzmann distribution / Luce choice rule from decision neuroscience. Temperature is τ = 1/β, where β is the "inverse temperature" parameter of softmax policies in reinforcement learning models of neural decision-making. Low temperature = more deterministic policy; high temperature = more stochastic exploration.

Why Next-Token Prediction Works

It is not obvious that training a model to predict the next word should produce a system that can reason, write poetry, solve math problems, and explain scientific concepts. Yet it does — and understanding why is one of the deepest questions in the field.

The compression argument

To predict the next token well across a diverse corpus, the model must implicitly learn everything that constrains what comes next: grammar, facts about the world, reasoning patterns, style, social conventions. A model that learns to predict text must, at least partly, model the process that generated the text — people writing about the world. Next-token prediction is, in this sense, a proxy for world modeling, though how much of the world a model captures this way is debated.

Formal framing. Minimizing cross-entropy loss on text is equivalent to maximizing the log-likelihood of the data under the model — which is equivalent to minimizing the KL divergence between the model's distribution and the true data-generating distribution. A perfect model would reproduce the data distribution exactly.

Scaling laws

Empirically, model performance improves as a power law with compute, data, and parameters — across many orders of magnitude. This was established by Kaplan et al. (2020) and refined in the Chinchilla paper (Hoffmann et al., 2022), which showed the optimal compute allocation: for a given compute budget, scale model size and data tokens roughly equally.

Scaling law (Chinchilla form, Hoffmann et al. 2022) Loss(N, D) ≈ E + A/N^α + B/D^β

N = model parameters, D = training tokens
E ≈ 1.69 (irreducible loss, nats/token), α ≈ 0.34, β ≈ 0.28

Kaplan et al. (2020) fit separate power laws (N without embedding parameters):
Loss(N) = (N_c/N)^α_N, α_N ≈ 0.076 Loss(D) = (D_c/D)^α_D, α_D ≈ 0.095

Emergent capabilities

As models scale, some capabilities have been reported to appear sharply — essentially absent at smaller scales and present above a threshold. Examples include multi-step arithmetic, in-context learning (few-shot prompting), chain-of-thought reasoning, and code generation. Whether these jumps are real is debated: much of the apparent sharpness comes from all-or-nothing metrics such as exact match, and with continuous metrics the same abilities often improve smoothly (Schaeffer et al. 2023).

The alignment question. A language model trained on next-token prediction does not directly optimize for truthfulness, helpfulness, or factual accuracy — only for predicting the distribution of its training text. Post-training then shapes the model's behavior: supervised fine-tuning (SFT) on example responses, preference tuning (RLHF or DPO), and reinforcement learning on verifiable rewards such as correct answers to math or code problems. This distinction matters enormously for interpretation.

Connection to studying LLMs like brains. If next-token prediction implicitly encodes world models and reasoning, then the dynamics of this computation — how information flows through the residual stream layer by layer as a token sequence is processed — should reveal something about the internal structure of that world model. This is precisely the electrophysiology analogy: record the "neural activity" of the model mid-computation and study its geometry, not just its input-output behavior.

The next lesson moves inside the transformer architecture — the specific network design that makes large-scale language modeling tractable and powerful.

Key formulas — Lesson 1

      Chain rule:    P(x0…xT−1) = ∏t P(xt | x0…xt−1)

      Softmax:      P(i) = exp(zi/τ) / Σj exp(zj/τ)   [τ = temperature]

      Cross-entropy: Loss = −log P(true token)

      Perplexity:   PPL = exp(Loss)  [Loss averaged over tokens; effective branching factor]