LLM Primer VI — Series Introduction & Index

Published on: 2026-04-22 Last updated on: 2026-07-06 Version: 2
LLM Primer VI — Series Introduction & Index

LLM Primer VI — Series Introduction & Index

A chapter-by-chapter walkthrough of LLM Primer VI: Scaling AI Systems — the volume that treats LLM inference as an engineering discipline where memory bandwidth, scheduling, and dollar signs collide.

Why this series exists

An H100 rented for four to eight dollars an hour delivers 989 TFLOP/s of BF16 compute. A single user, generating a single response in real time against a 70B model on that chip, consumes roughly 0.34 percent of that compute. The chip is 99.7 percent idle — not because the workload is small, but because the workload is memory-bandwidth-bound and the chip's compute units have nothing to do while HBM streams weights through them. Inference engineering is the discipline of finding work for those idle units. It is a discipline of batching, of KV cache accounting, of quantization, of speculative decoding, of scheduling, and eventually of platform, orchestration, and cost. Every technique in this volume is a variation on the same move: convert bandwidth-bound waste into throughput without breaking the latency the user actually feels.

The book in one sentence: LLM serving is a systems problem where a bandwidth-bound decoding loop, a KV cache the size of a small database, and a per-token bill combine to determine whether a product survives contact with real traffic.

Who I wrote this for

Platform engineers, SREs, and infrastructure architects who own LLM inference at scale — the people who get paged when latency spikes, whose budget line item is the GPU pool, and who have to explain to finance why last month's bill was double the month before. It is also written for ML engineers who trained the model and now have to serve it, and for backend engineers who suddenly find themselves the closest thing their team has to an inference specialist. The book assumes fluency with distributed systems and containers; it does not assume prior familiarity with GPU internals or the transformer's memory pattern. Those it builds up from first principles.

How to read it

The sixteen chapters divide into six parts. Chapters 1–2 name the workload — the autoregressive loop and the KV cache. Chapters 3–4 walk the silicon that runs it, from H100s to Groq's LPU. Chapters 5–6 walk the model-side compressions — quantization, pruning, distillation — that shrink the bandwidth burden. Chapters 7–9 walk the runtime techniques — batching, paged KV, speculative decoding — that hide idle time. Chapters 10–13 walk the serving stack — engines, platforms, disaggregation, autoscaling — that turns those techniques into a service. Chapters 14–16 walk the money: token economics, self-host versus API, and the cost-cutting moves that compound. The chapters can be read in order or dipped into by topic, but the mechanism-first framing depends on Chapters 1 and 2.

The 16-chapter walk

Between April 23 and May 8 the walkthrough posts one chapter per day. Each article distills the chapter's three key ideas into roughly a five-minute read, with the book chapter supplying the worked examples, code, and In Plain English sidebars.

Where this sits in the series: Volumes I–IV built up transformer mechanics, RAG, agents, and modality. Volume V walked application-level engineering — the deterministic wrapper around the probabilistic core. Volume VI is the layer beneath the wrapper: how the model call itself is served fast and cheap enough for the wrapper to be economical. Volume VII closes the series with AI Security — threat modeling, guardrails, and the regulation that is now shaping how all of the above must be deployed.

About this book and the series

The LLM Primer series is seven volumes written by Sho Shimoda, published on Amazon KDP and read chapter-by-chapter here on the ReceiptRoller blog. The series argues that building with LLMs is a systems discipline, and that the discipline is best learned by walking each layer of the stack in mechanism-first prose rather than checklist form. Volume VI is the infrastructure volume — the one that answers, layer by layer, the question of what has to be true about the hardware, the runtime, and the platform for an LLM feature to survive real traffic and a finance review.

Grab a copy. The book has the full worked examples, the runnable Python for calibration and continuous batching, the YAML for KServe and Grove, and the In Plain English sidebars that the walkthrough articles only summarize. LLM Primer VI on Amazon →
LLM Primer VI LLM inference scaling AI GPU serving vLLM TensorRT-LLM series intro
SHO
SHO
CTO of Receipt Roller Inc., he builds innovative AI solutions and writes to make large language models more understandable, sharing both practical uses and behind-the-scenes insights.