All of AI engineering —
neuron to frontier, in public.
To measure every bit that moves, you have to understand every bit that moves. So we are learning the whole stack — from a single neuron to a frontier LLM to the GPU it runs on — and rendering each topic as an interactive chapter you can play with: the theory, the actual math, a live visualization with knobs you drag, and the original sources linked, not rehosted.
This is the credibility engine. The lessons are ours, the demos run the real math, and the results are measured — the same honesty we hold GPUs to.
I · Foundations
the machinery under everythingHow a model learns
A model is a function with knobs; learning is rolling downhill on a loss landscape.
▶roll a ball down a live loss surfaceopen chapter →Linear algebra — matmul
Vectors in, matrices of knobs, and the one operation that is ~90% of all the arithmetic.
▶watch a matrix multiply accumulate cell by cellopen chapter →Probability — the loss
A model outputs a distribution, not an answer; maximum likelihood is where every loss is born.
▶fit a bell curve to real data by handopen chapter →II · Deep learning
from one neuron to deep networksThe neuron & backprop
Wire neurons into a network, push data forward, send error backward — the chain rule at scale.
▶train a tiny net and watch its boundary formopen chapter →Training deep nets
Activations, initialization, normalization, and the optimizers — SGD → momentum → Adam.
▶race three optimizers across a loss surfaceopen chapter →Convolutions & vision
The kernel that slides, weight sharing, feature maps — LeNet to ResNet.
▶drag a kernel over an image, see the feature mapopen chapter →Sequences & memory
Recurrence, unrolling through time, and why long-range gradients vanish — then LSTM gates.
▶watch a gradient vanish as the sequence growsopen chapter →III · Transformers & LLMs
the architecture that ate the fieldAttention & the Transformer
Query, key, value: every token looks at every other and decides what matters.
▶move a query and watch what it attends toopen chapter →Tokenization & embeddings
Text → tokens via BPE, tokens → vectors, and meaning as geometry.
▶run BPE merges on your own stringopen chapter →The LLM assembled
Stack the decoder block: residual stream, positional encoding / RoPE, the KV cache.
▶assemble a GPT block and rotate RoPEopen chapter →Pretraining & scaling laws
Next-token prediction at scale, and the compute-optimal trade of params vs tokens.
▶slide the scaling-law knee (Chinchilla)open chapter →Alignment
From raw predictor to helpful assistant: SFT, reward models, RLHF, and DPO.
▶shape a reward and watch preference shiftopen chapter →Inference & decoding
Turning probabilities into text: temperature, top-k / top-p, and the KV cache at serve time.
▶twist the sampling knobs, resample the textopen chapter →IV · Systems & the endgame
where utilization starts to lieSystems & the GPU
Threads, warps, blocks, SMs; the memory hierarchy; the roofline and arithmetic intensity.
▶drag intensity across the memory/compute kneeopen chapter →Observability — the endgame
nvidia-smi vs the truth, DCGM, and the vertical trace that joins kernel → op → model → cluster.
▶watch %util read 100% while real work collapsesopen chapter →Chapters go live as their interactive is built and verified. The format is locked: theory + the actual math + a draggable live demo + linked sources. This page is the map; we are filling it in, in order.