AI Systems
GPU Execution, Memory Bandwidth, Batching, and Parallelism
Analyze inference workloads in terms of accelerator work, data movement, and serving constraints.
Loading learning records…
Readings and practice
Batching: throughput gains and the cost of waiting
Unread · Latest correct 0/3
Practise these questionsAnswer history and review dates
Question 1
Not yet answered
Question 2
Not yet answered
Question 3
Not yet answered
GPU Execution, Memory Bandwidth, Batching, and Parallelism
Unread · Latest correct 0/3
Practise these questionsAnswer history and review dates
Question 1
Not yet answered
Question 2
Not yet answered
Question 3
Not yet answered
Inference-service SLOs: separate waiting from generation speed
Unread · Latest correct 0/3
Practise these questionsAnswer history and review dates
Question 1
Not yet answered
Question 2
Not yet answered
Question 3
Not yet answered
Calculating KV-cache capacity
Unread · Latest correct 0/3
Practise these questionsAnswer history and review dates
Question 1
Not yet answered
Question 2
Not yet answered
Question 3
Not yet answered
Calculating KV-cache size from model configuration
Unread · Latest correct 0/3
Practise these questionsAnswer history and review dates
Question 1
Not yet answered
Question 2
Not yet answered
Question 3
Not yet answered
Separating prefill and decode in LLM inference
Unread · Latest correct 0/3
Practise these questionsAnswer history and review dates
Question 1
Not yet answered
Question 2
Not yet answered
Question 3
Not yet answered
Quantization error and inference performance
Unread · Latest correct 0/3
Practise these questionsAnswer history and review dates
Question 1
Not yet answered
Question 2
Not yet answered
Question 3
Not yet answered
GPU memory bandwidth and arithmetic intensity
Unread · Latest correct 0/3
Practise these questionsAnswer history and review dates
Question 1
Not yet answered
Question 2
Not yet answered
Question 3
Not yet answered
Comparing inference batch size and latency
Unread · Latest correct 0/3
Practise these questionsAnswer history and review dates
Question 1
Not yet answered
Question 2
Not yet answered
Question 3
Not yet answered
Measuring quantization error
Unread · Latest correct 0/3
Practise these questionsAnswer history and review dates
Question 1
Not yet answered
Question 2
Not yet answered
Question 3
Not yet answered
Quantization versus pruning
Unread · Latest correct 0/3
Practise these questionsAnswer history and review dates
Question 1
Not yet answered
Question 2
Not yet answered
Question 3
Not yet answered