A daily series
How It Works
Long-form chapters on how technologies, systems and ideas work: where they came from, how they evolved, and what actually happens under the hood.
11 chapters · Latest 20 Oct 2026 · RSS
Latest chapter · Artificial Intelligence
How LLMs Generate Text: Inference, Context And Sampling
How prompts become generated tokens through prefill, key-value caching and decoding, and why sampling and context change serving behavior.
Read chapter 011 →
Contents
- Chapter 001
From Perceptrons to LLMs
Seventy years of neural networks: from a single weighted sum in 1958 to models that write, translate and reason about code.
Artificial Intelligence - Chapter 002
How GPUs Work
Understanding GPU architecture and parallel computing: why a chip designed to shade pixels ended up training the world's AI models.
Computing - Chapter 003
Learning In Layers: Gradient Descent And Backpropagation
How multilayer networks learn representations through gradient descent and backpropagation, with a complete worked numerical example.
Artificial Intelligence - Chapter 004
How A Machine Learns To See: From Neocognitron And LeNet To AlexNet
How convolutional networks exploit the structure of images, and how architecture, data and GPUs came together to make AlexNet possible.
Artificial Intelligence - Chapter 005
When Networks Left The Spotlight
The AI winters, the rise of expert systems and the ideas from statistical learning that are still useful today.
Artificial Intelligence - Chapter 006
Machines With Memory: RNNs, LSTMs And The Problem Of Sequences
How recurrent networks process sequences, why their gradients break down and how the gates of an LSTM control memory.
Artificial Intelligence - Chapter 007
Words Turned Into Coordinates: Embeddings And The Representation Of Language
How a machine learns numerical representations of language, from word vectors to contextualised embeddings.
Artificial Intelligence - Chapter 008
Attention And Transformers: Queries, Keys And Values
How attention mixes information between tokens, and how transformers combine queries, keys, values, positions and masks.
Artificial Intelligence - Chapter 009
From Transformers To LLMs: Pretraining And Next-Token Prediction
How next-token prediction turns a transformer into a language model, and why pretraining enables useful transfer without guaranteeing truth.
Artificial Intelligence - Chapter 010
How LLMs Are Trained: Data, Tokenization And Scaling
How documents become token batches and parameter updates, and how data choices, tokenization and compute budgets shape LLM training.
Artificial Intelligence - Chapter 011
How LLMs Generate Text: Inference, Context And Sampling
How prompts become generated tokens through prefill, key-value caching and decoding, and why sampling and context change serving behavior.
Artificial Intelligence