Chapter 17 — Future Threats and Emerging Defenses
Seventeenth and final post of the chapter-by-chapter walkthrough of LLM Primer VII: AI Security. The chapter that looks past the disciplines already mature enough to write down and names the ones the community is still figuring out — autonomous agents, multimodal attack surfaces, synthetic identity, and the AI-versus-AI arms race of mid-2026.
Why this chapter exists
Chapters 1 through 16 walked the security disciplines that are already mature enough to write down. Chapter 17 walks the ones that are still forming. The contrast matters. The earlier chapters answered "we know how to do this and the question is whether your organisation will." This one answers "the community is still figuring out what good looks like, and the answers next year may be different from the answers this year." Both kinds of work are real, and both belong in the engineer's repertoire. The first kind keeps systems safe today. The second kind keeps them safe in two years.
17.1 Autonomous agents multiply blast radius by the tool budget
An autonomous agent is a system in which a language model is given a goal, a set of tools, and the authority to call those tools in sequence without per-step human review. The model decides what to do next; tools execute; results feed back; the loop continues until the goal is reached or a stopping condition fires. AutoGPT and BabyAGI prototyped the pattern publicly in early 2023; LangChain formalised it; Anthropic's Claude computer use (October 2024) and OpenAI's Operator (January 2025) turned it into commercial products that give a model authority over a browser or a desktop. The security properties are different in kind from single-turn generation. A model that produces one completion has the authority to write text the application might act on. A model that operates a browser has, in the worst case, the authority to do whatever a browser can do, without a human review point. The mitigations composed from earlier chapters — tool allow-lists, capability tokens with narrow scope, per-tool policy checks outside the model, human confirmation for high-impact actions, hard step limits, budget caps — apply here with sharpened emphasis. The agent's autonomy is a design decision made in advance of any specific incident, and the design decides the ceiling on any compromise.
17.2 Multimodal surfaces widen the injection channel by orders of magnitude
A text-only model reads what the tokeniser recognises. A vision-language model reads images whose pixel-level bandwidth exceeds anything text can carry, and the surrounding application usually cannot inspect what an image contains in the way it can inspect a string. Schlarmann and Hein's 2023 paper "On the Adversarial Robustness of Multi-Modal Foundation Models" showed that human-imperceptible perturbations to an image could substantially change a VLM's text outputs — the classical adversarial-example attack adapted to a new modality. Text embedded in images (screenshots, memes, document scans) is another surface: OCR-detected instructions in an image can be indirect prompt injection through the visual channel. Audio-language models take spoken input; adversarial audio perturbations, some inaudible to humans, have been demonstrated. Video adds both channels together. Each modality is a new injection surface, and defensive work is only beginning to catch up: pre-VLM scanners that flag or strip OCR-detected text in images; provenance tags on multimodal inputs; alignment training that treats visual and audio channels with the same suspicion as text; and modality-aware output filtering. The pattern from earlier chapters — do not let trust boundaries collapse into a single input — extends to modalities the earlier chapters did not have to name.
17.3 Synthetic identity and AI-versus-AI reshape the perimeter
The first two sections treated the model as target. The third treats it as tool. A generative model whose outputs are indistinguishable from authentic human production undermines any security mechanism that depends on authenticity being detectable. Synthetic voice generated from minutes of speech powers vishing against finance staff who recognise the CEO's voice. Deepfake video fabricates footage of public figures. Synthetic text imitates a specific person's writing in business-email-compromise. None of these is exotic by 2026; the tools are commodity, the cost per generation is cents. The defensive response has been provenance infrastructure — C2PA for content credentials, watermarking research, cryptographic signing of authentic media — and identity-verification uplift where high-stakes decisions rest on channel authenticity. The wider defensive trajectory is AI-versus-AI: security systems built on language models, deployed to defend against attacks generated or amplified by other language models. Automated red-teaming — NVIDIA Garak, Microsoft PyRIT — substitutes model for human on the attacker side. Guardrails, safety classifiers, and anomaly detection substitute model for human on the defender side. The arms race is uncomfortable but it is the operating reality. AI assurance, the emerging discipline of showing that an AI system meets its requirements with defined confidence, is what tries to give the arms race a stable footing — continuous evaluation infrastructure, third-party certification, incident-disclosure norms, and the standards work the ISO/IEC 42001 and NIST AI RMF communities are extending.
The series ends here
Chapter 17 is the final chapter of LLM Primer VII, and with it the final chapter of the LLM Primer series as a whole. Volume I introduced the architecture of transformer-based language models from the attention mechanism outward. Volume II covered training, alignment, and the practical lifecycle of building one. Volume III examined retrieval-augmented generation and the surrounding data pipelines. Volume IV looked at evaluation, tooling, and the engineering practices that surround the model in production. Volume V worked through the agent and tool-use patterns this chapter has now treated under their adversarial aspect. Volume VI covered the inference infrastructure and scaling patterns at organisational scale. Volume VII, this one, has been about defending all of it. The seven volumes were written as a connected map. Readers who reach this chapter without having visited the earlier volumes are encouraged to circle back, because many of the security claims in this book rest on architectural details those volumes establish.
The companion volume, Physical AI, extends the map into embodied systems — robots, autonomous vehicles, and the physical-world deployments where the same probabilistic substrate now controls actuators and shares physical space with humans. The security concerns of this volume carry over with modifications: prompt injection through the vision channel becomes a safety concern with a metre of physical reach; the tool boundary is now a motor controller; adversarial inputs are objects placed in the operating environment. The disciplines from these seven volumes are prerequisites for that work rather than substitutes for it, and the arc from transformer attention through infrastructure through security continues into the physical world where the stakes become tangible.
Thanks for reading the walkthrough. The book itself carries the worked examples, the runnable code, the incident playbooks, and the In Plain English sidebars in longer form than these articles have room for. If any of it has been useful, the highest-impact next step is applying the framework to the specific system your organisation is defending — the threat model from Chapter 2, the layered mitigations from Chapters 4 and 5, the architecture patterns from Chapter 10, the observability from Chapter 11, and the organisational discipline from Chapter 15 are the load-bearing pieces.
