Chapter 6 — Fine-Tuning & Adaptation

This is Part 6 of a series walking through LLM Primer I: How Generative AI Works. Yesterday we looked at how a base model is trained. Today we look at what happens after: how a pretrained model becomes a useful product, and why the techniques for doing that have become more important than the underlying model in many cases.

Pretraining is broad. Adaptation is specific.

A freshly pretrained large language model is a strange beast. It has absorbed a vast amount of structure from text — grammar, facts, idioms, code patterns, citation styles, jokes, mistakes, ideologies, the whole soup — but it has no opinion about which of those things it should produce when. Ask it "what is the capital of France?" and it might answer "Paris" or it might continue as if it were a quiz book ("What is the capital of Germany? What is the capital of Italy?"). It hasn't been told that you wanted a direct answer.

Adaptation is the family of techniques that fix this. The pretrained model already knows nearly everything it will ever know in any useful sense; adaptation reshapes how and when that knowledge is expressed. Chapter 6 walks through the adaptation spectrum from the lightest touch to the heaviest.

The cheapest move: just write a better prompt

The lightest form of adaptation costs nothing and changes no parameters. You simply provide context in the prompt that nudges the model toward the kind of output you want. Show it two or three examples of the task and ask it to do another one. State the rules explicitly. Define the role you want the model to play.

This is called prompt-based adaptation, and when it works, it's the right answer. No training pipeline, no GPU bill, no risk of breaking something. The book explains when prompt-based adaptation hits its limits — which it does — and how to recognize that you've outgrown it.

Instruction tuning: teaching the model to follow you

Instruction tuning is the lightest form of actual fine-tuning, and it's the one that turned raw pretrained models into the responsive assistants you actually use. The idea is simple: you show the model many examples of instruction-response pairs ("Summarize this paragraph in one sentence: …"), and you train on those examples for a relatively short time.

After instruction tuning, the model has internalized the general skill of "follow what's in the prompt." It doesn't need explicit examples anymore. This is what makes a chatbot feel like a chatbot instead of an autocomplete on a typewriter.

Parameter-efficient fine-tuning

Full fine-tuning — updating every parameter in the model — is expensive and risks degrading the model's general capabilities. The last few years have produced a family of techniques that let you adapt a large model by training only a tiny fraction of the parameters, often less than 1%.

The most popular technique is called LoRA (Low-Rank Adaptation). It works by inserting small matrices alongside the model's frozen weights and training only those matrices. The result is a tiny adapter file — often a few megabytes — that, when combined with the base model, produces fine-tuned behavior. Variants like QLoRA push this even further by also quantizing the base model so the whole thing fits on a single accelerator.

This is the technique that makes domain-specific fine-tuning practical for organizations that aren't training their own base models. The book includes a table comparing the major parameter-efficient techniques side by side, so you can pick the right one for your use case.

Alignment: the post-training that really matters

This is the section I'm proudest of in the 2026 edition, because alignment is where the field has moved fastest and where most readers have the murkiest understanding. The core idea is this: even after instruction tuning, a model will sometimes produce outputs you don't want — incorrect, unsafe, off-policy, unhelpful. Alignment is the family of techniques that shapes the model's probability distribution to favor outputs humans would prefer.

The first widely-deployed approach was reinforcement learning from human feedback (RLHF). Human reviewers compare pairs of model outputs and indicate which is better. Those judgments train a reward model that predicts human preference. The base model is then fine-tuned with reinforcement learning, using the reward model as the training signal. This produced dramatic improvements but was famously unstable and expensive.

A second generation of methods — direct preference optimization (DPO) and its variants — simplified the pipeline. They treat the pairwise preferences as a direct supervised signal, eliminating the separate reward model and the reinforcement learning step. The result is more stable, cheaper, and reproducible.

A third family — constitutional methods and AI-feedback variants — replaces some of the human review work with comparisons generated by a stronger model judging outputs against a written set of principles. This scales alignment data far beyond what human review alone can produce.

What Chapter 6 sets up

By the end of Chapter 6, you understand the full adaptation pipeline: from prompt tricks to alignment. You can read announcements about new models and correctly distinguish between "they trained a new base" (rare and expensive) and "they updated the post-training" (common and impactful). And you have a framework for deciding which adaptation technique to use for your own work.

Next up — Chapter 7: Beyond Next-Token Prediction. Tomorrow we go past the basic generation loop. Embeddings, semantic search, retrieval-augmented generation (RAG), and the multimodal extensions that let models work with images and audio.