LEARNING PATH
Prepare for AI systems
Use a model. Understand the systems that run it.
Start with complexity, memory hierarchies, CPUs, and concurrency before studying AI execution systems. See where model theory fits into the wider learning path.
For learners interested in the computers behind AI, retrieval, and inference performance
Where should you start when investigating slow inference or retrieval?
Scope and finish line
Every main reading in this course is published and available now.
- Steps
- 23
- Main readings
- 23
- Check questions
- 66
What you will learn to explain
- Use linear algebra, probability and differentiation to explain loss functions, training and neural network computations.
- Explain Transformer inference and memory use, batching, and retrieval and evaluation in RAG.
Final main reading: LLM Evaluation, Observability, and Security
Complete each step’s main reading and its check, where one is provided. Extra reading and experiments are optional.
See the final step ↓0 of 23 steps complete
Start learning →Apply the course in code
After an experiment, apply its rule in a short function and test the result.
Your learning steps
STEP 01Not started From Source Code to a Running Program
Goal: Explain how source code passes through a compiler, interpreter, and runtime before execution.
STEP 02Not started Complexity and Measured Performance
Goal: Derive time and space complexity using O, Omega, and Theta notation and explain how costs grow with input size.
Main reading: Complexity and Measured Performance →
Explore the mechanisms
STEP 03Not started Memory, Pointers, References, and Lifetimes
Goal: Diagram runtime memory using stacks, heaps, addresses, pointers, and references.
Main reading: Memory, Pointers, References, and Lifetimes →
Explore the mechanisms
STEP 04Not started CPUs, Instruction Cycles, and Pipelines
Goal: Trace instruction fetch, decode, execution, memory access, and write-back.
Main reading: CPUs, Instruction Cycles, and Pipelines →
Explore the mechanisms
- Measuring execution with perf stat
- IPC and instruction throughput: count useful work per cycle
- Branch predictors: choose a path before the condition is resolved
- Out-of-order execution and dependencies
- Branch prediction and misprediction cost
- SIMD and loop vectorization
- Inspect SIMD instructions and loop tails
STEP 05Not started Caches and the Memory Hierarchy
Goal: Explain the memory hierarchy of registers, caches, main memory, and storage, and why locality matters.
Main reading: Caches and the Memory Hierarchy →
Explore the mechanisms
- Locality of reference: closeness in time and space
- TLBs: caching address translations
- Cache coherence: coordinating copies of one memory location
- Trace cache placement from addresses
- Reason about performance ceilings with Roofline
- Compare row and column traversal order
- Compare stride, visit count and traversal time
- Compare starting alignment with valid C accesses
- Measure arithmetic work and nominal traffic
- Separate first and repeated timing observations
STEP 06Not started Virtual Memory, Page Tables, and the TLB
Goal: Split a virtual address into a page number and offset, and trace translation through page tables and the translation lookaside buffer.
Main reading: Virtual Memory, Page Tables, and the TLB →
Explore the mechanisms
STEP 07Not started Interleaving, Race Conditions, and Atomicity
Goal: Enumerate thread interleavings to detect race conditions and lost updates to shared state.
Main reading: Interleaving, Race Conditions, and Atomicity →
Explore the mechanisms
- Atomicity: choosing an indivisible operation
- Happens-before: causal and synchronization order
- Read memory ordering through publication and synchronization
- Check linearizability from invocation and response histories
- From unlinking a node to reclaiming it safely
- Design thread pools from slots, queues and dependencies
- Separate cancellation delivery from cleanup ownership
STEP 08Not started Vectors, Matrices, and Linear Maps
Goal: Represent multidimensional data and linear transformations using vectors, matrices, linear combinations, and bases.
Main reading: Vectors, matrices, and linear maps: read a transformation by columns →
STEP 09Not started Eigenvalues, Orthogonal Projection, and Matrix Decompositions
Goal: Explain eigenvalues and eigenvectors as quantities and directions preserved by a linear transformation.
Main reading: Eigenvalues, Orthogonal Projection, and Matrix Decompositions →
Explore the mechanisms
STEP 10Not started Conditional Probability, Random Variables, and Distributions
Goal: Model uncertain phenomena using sample spaces, events, conditional probability, and independence.
Main reading: Random variables and distributions: behavior beyond the average →
Explore the mechanisms
STEP 11Not started Samples, Estimation, Hypothesis Tests, and Evaluation
Goal: Explain the relationships between populations, samples, estimators, likelihood, bias, and variance.
Main reading: Samples, Estimation, Hypothesis Tests, and Evaluation →
Explore the mechanisms
STEP 12Not started Derivatives, Partial Derivatives, Gradients, and the Chain Rule
Goal: Interpret derivatives, partial derivatives, and gradients as rates of change and calculate them for functions of one or several variables.
Main reading: Derivatives and gradients: read local change →
Explore the mechanisms
STEP 13Not started Convexity, Objective Functions, and Gradient Descent
Goal: Formulate optimization problems using objective functions, constraints, local and global optima, and convexity.
Main reading: Gradient descent: test step size and convergence on a quadratic →
Explore the mechanisms
STEP 14Not started Supervised Learning, Models, and Loss Functions
Goal: Formulate supervised learning using features, labels, models, loss functions, and empirical risk.
Main reading: Supervised Learning, Models, and Loss Functions →
Explore the mechanisms
STEP 15Not started Generalization, Overfitting, and Evaluation Metrics
Goal: Explain the difference between training and generalization error, and distinguish overfitting, underfitting, bias, and variance.
Main reading: Generalization, Overfitting, and Evaluation Metrics →
Explore the mechanisms
STEP 16Not started Feature Representations, Dimensionality Reduction, and Clustering
Goal: Explain how feature engineering and representation learning affect distance, similarity, and predictive performance.
Main reading: Feature Representations, Dimensionality Reduction, and Clustering →
Explore the mechanisms
STEP 17Not started Neural Networks, Computation Graphs, and Backpropagation
Goal: Explain forward propagation using neurons, layers, activation functions, and computation graphs.
Main reading: Neural Networks, Computation Graphs, and Backpropagation →
STEP 18Not started Deep Learning Optimization and Model Architectures
Goal: Compare the update rules and roles of stochastic gradient descent, momentum, Adam, and normalization.
Main reading: Deep Learning Optimization and Model Architectures →
Explore the mechanisms
STEP 19Not started Tokenization, Self-Attention, and Transformers
Goal: Explain how text becomes tokens, embeddings, and positional information before passing through Transformer blocks.
Main reading: Tokenization, Self-Attention, and Transformers →
Explore the mechanisms
STEP 20Not started Pretraining, Fine-Tuning, Generation, and the KV Cache
Goal: Distinguish next-token pretraining, instruction tuning, and inference-time decoding.
Main reading: Pretraining, Fine-Tuning, Generation, and the KV Cache →
Explore the mechanisms
STEP 21Not started GPU Execution, Memory Bandwidth, Batching, and Parallelism
Goal: Explain how GPUs, Tensor Cores, kernels, and memory bandwidth determine model inference time.
Main reading: GPU Execution, Memory Bandwidth, Batching, and Parallelism →
Explore the mechanisms
STEP 22Not started Vector Search, Chunking, and Retrieval-Augmented Generation
Goal: Explain a RAG pipeline consisting of document ingestion, chunking, embedding, indexing, retrieval, reranking, and generation.
Main reading: Vector Search, Chunking, and Retrieval-Augmented Generation →
Explore the mechanisms
STEP 23Not started LLM Evaluation, Observability, and Security
Goal: Combine offline and online evaluation, traces, and quality metrics to observe failures in LLM systems.
Main reading: LLM Evaluation, Observability, and Security →
Explore the mechanisms
Progress is kept in this browser for the current language. You can read the steps in any order.