Series Foundations: What This Book Builds On
This is a reference for authors and editors, not a manuscript chapter. AI Red Team is Book 4 of The Transformed Architecture. It does not repeat the earlier volumes; it takes their conclusions as its starting point and asks what defenders do next. This document records those conclusions with stable identifiers (F1 to F24), the open questions the earlier volumes left for later work (Q1 to Q10), the artefacts and vocabulary the book reuses, and the tables of contents used for cross-references. BOOK_BLUEPRINT.md maps every chapter of this book to the foundations it builds on.
When a chapter builds on a foundation, it says so in the prose, refers to the earlier volume in plain text by chapter ("AI Platform and Security, Chapter 18"), cites the volume's bibliography entry, and summarises enough that the reader is not left with a gap.
The turn this book makes
The first three volumes treat AI mainly as something to build, govern and protect. Architecture as Code made architecture executable. AI Governance made governance executable and proportionate to usage mode and risk. AI Platform and Security built the platform, secured it and concluded that AI is now both a target and a source of threat, so every AI system must be designed twice: as something to protect and as something to contain. Its Chapter 18 ended with four architectural commitments against AI-enabled adversaries, the fourth of which was "using AI for defence, under oversight".
AI Red Team starts from that fourth commitment. It treats AI as an active participant in cyber defence across every Function of NIST CSF 2.0, from Govern to Recover, and through the three focus areas of the NIST Cyber AI Profile: securing AI systems, AI-enabled cyber defence and thwarting AI-enabled attacks. Red teaming, both with AI and of AI, runs through every Function, and the purple loop turns findings into verified controls. It keeps the earlier volumes' discipline: everything that matters is codified, versioned, tested and produces evidence, and every AI that acts does so inside boundaries that hold whatever anyone intends.
Foundations from Architecture as Code (Book 1)
| ID |
Conclusion |
Where it is made |
| F1 |
Threat handling belongs in the codebase: threat intelligence is ingested as STIX/TAXII through pipelines, threat models are generated from architecture code, response playbooks are state machines (subscribe, assess, act, document), and findings return as pull requests. "Celebrate the remediation pull request, not just the discovery." |
Chapter 9C, Risk as Code and Threat Handling as Code |
| F2 |
Risk is a versioned record with likelihood, impact, owner and expiring exceptions; a failing control metric fails the pipeline. |
Chapter 9C |
| F3 |
Policy as Code (OPA/Rego) with "assure once, comply many": a control is written once and mapped to many obligations, with evidence linked to enforcement. |
Chapters 10 and 11 |
| F4 |
Evidence as Code: evidence is machine-collected, signed and versioned, not assembled in audit season. |
Chapter 15A, Evidence as Code and Continuous Assurance |
| F5 |
Manual incident response cannot scale when attacks develop within minutes; compromised components are redeployed from known-good definitions, and MTTD, MTTR and false-positive ratio are the programme metrics. AI models must themselves be defended against adversarial machine learning. |
Chapter 9B, Advanced Security Patterns and Implementation |
| F6 |
The four security dimensions: threat modelling, zero trust, Policy as Code and risk assessment. |
Chapter 9, Security Fundamentals for Architecture as Code |
| F7 |
AI agents are structured participants under human direction: "agents draft; humans approve", and exceptions and risk acceptances carry a named human approval in the audit trail. |
Chapter 20, AI Agent Team for Architecture as Code Initiatives |
| F8 |
Security and compliance tests, including synthetic attack simulations, run in the pipeline against ephemeral environments. |
Chapter 13, Testing Strategies for Infrastructure as Code |
Foundations from AI Governance (Book 2)
| ID |
Conclusion |
Where it is made |
| F9 |
AI governance is effective when it is codified, evidenced and proportionate to usage mode and risk; the usage mode, not the brand, sets the governance (Mode 0 ambient AI to Mode 5 autonomous production agents). |
Chapters 2 and 25 |
| F10 |
Classification is triggered by change, not by the calendar: a new model version, a drift alert, an incident, a regulatory change or a move between modes. |
Chapter 8, AI Risk Management and Classification as Code |
| F11 |
Decisions are tiered: routine decisions at pipeline gates, exceptions at a design authority, high-risk go-live at the AI governance board, risk beyond appetite at the management body. |
Chapter 13, Decision Forums and Governance Bodies |
| F12 |
Evidence is a by-product of delivery; the evidence manifest ties the chain together; attestation is a person asserting, evidence is machine-generated. |
Chapter 11, Evidence, Assurance and Audit |
| F13 |
Govern every agent as a well-meaning insider: instructions lower the rate of unwanted behaviour, but only boundaries set a ceiling on its consequences. |
Chapters 19 and 25 |
| F14 |
The July 2026 OpenAI–Hugging Face incident failed through correlated controls: a shared credential, a package mirror that was also an egress path, and an alert that reached people who did not hold the stop decision. |
Chapter 19, A Cautionary Case: The OpenAI–Hugging Face Incident |
| F15 |
Patch decisions must be delegated in advance; a weekly change advisory board cannot sit in the path of known-exploited vulnerabilities, and patch velocity belongs in management reporting as a risk indicator. |
Chapter 24, Reflections on Each Part |
| F16 |
Human oversight must be designed, not declared: an interruption path, an override log and the authority to use them. Approvals given in seconds are a signal of rubber-stamping. |
Chapters 10 and 21 |
| ID |
Conclusion |
Where it is made |
| F17 |
A platform is under control when every model call and tool call passes a policy point outside the model, when containment holds whatever anyone intends, and when evidence is produced by normal operation. |
Chapter 25, Conclusions: Building and Securing AI Platforms |
| F18 |
AI is both a target and a source of threat; every AI system is designed twice, as something to protect and as something to contain; security operations must run faster than a monthly patch cycle. |
Chapters 18 and 25 |
| F19 |
Four commitments against AI-enabled adversaries: patch velocity as a measured service level (EPSS, KEV, EUVD and local exposure); detection at machine speed, with canaries, honeytokens and the authority to act; identity and approvals that resist synthetic media; and AI for defence under oversight, where defensive agents get scoped credentials, sandboxed verification and human approval, and human verification is the bottleneck. |
Chapter 18, When AI Is the Threat |
| F20 |
Prompt injection is a property of the architecture, not a patchable defect; controls sit outside the model; OWASP LLM Top 10 and MITRE ATLAS give the shared vocabulary; model loading is code execution. |
Chapter 16, Security for AI Systems |
| F21 |
Red teaming of AI systems runs continuously in CI: the external exercise is for discovery, and every finding becomes a versioned attack case with explicit thresholds, published as evidence against control IDs. DORA threat-led penetration testing is periodic; continuous suites complement it. |
Chapter 17, AI Security Engineering in Practice |
| F22 |
Red teaming is the apex of the evaluation pyramid: a scenario library in version control, an automated harness, a route for human findings into scenarios, and a finding–remediation–re-test path. Article 55 adversarial testing falls on providers of general-purpose AI models with systemic risk; deployers ask for the evidence and test the compound system themselves. |
Chapter 20, Evaluation and Testing of AI Systems |
| F23 |
Containment must not depend on intent: task-scoped credentials, default-deny egress, control artefacts outside the agent's reach, budgets and a stop path outside the system they stop, recorded approvals, and detection from telemetry and decoys rather than self-report. |
Chapter 19, When Your Own Agents Exceed Their Mandate |
| F24 |
AI incident management adds an AI classification step (AI Act Articles 3 and 73, DORA), an impact query against the event log and a path back to evaluation; the runbook stages are detection, triage, containment, notification, recovery and post-incident review. |
Chapter 21, Observability, Monitoring and Drift in Production |
Open questions left for this book
| ID |
Question |
Raised in |
| Q1 |
Version control captures what changed but not whether the reasoning behind a change was sound. How is AI-proposed security change (a detection, a playbook, a patch) reviewed for soundness? |
Architecture as Code (Chapter 27) |
| Q2 |
AI-enhanced security tooling and automated threat hunting are named as trends but never developed. |
Architecture as Code (Chapter 9B) |
| Q3 |
How do defenders plan for patch-to-exploit times measured in minutes, and will patch velocity become a supervised metric? |
AI Governance, Chapter 24 |
| Q4 |
How far can evaluation carry assurance when models can tell test settings from deployment? |
AI Governance, Chapter 24 |
| Q5 |
How do organisations prepare for a stream of smaller agent incidents of the same shape inside enterprises? |
AI Governance, Chapter 24 |
| Q6 |
The containment built for one's own agents is the same containment that limits an external one. How is that containment tested from the adversary's side? |
AI Governance, Chapters 19 and 23 |
| Q7 |
Adversary emulation that includes AI-enabled techniques is a natural extension of DORA threat-led penetration testing. What does it look like in practice, and how is it kept safe? |
AI Platform and Security, Chapter 18 |
| Q8 |
Human verification is the bottleneck in AI-assisted vulnerability discovery, and every patch is an exploit specification. How do defenders absorb the patch wave? |
AI Platform and Security, Chapter 18 |
| Q9 |
Security architecture will assume machine-speed adversaries and machine-speed insiders as the normal case. What does a security operation built for that assumption look like? |
AI Platform and Security, Chapter 25 |
| Q10 |
When may a defensive AI act without per-action approval, and who decides? |
AI Platform and Security, Chapter 18; AI Governance, Chapter 13 |
Vocabulary and artefacts to reuse
- Usage modes Mode 0 ambient AI, Mode 1 conversational assistants, Mode 2 embedded workplace assistants, Mode 3 agentic work tools, Mode 4 built AI applications, Mode 5 autonomous production agents (AI Governance, Chapter 2; AI Platform and Security, Chapter 2). This book reads the usage modes through a security lens (Chapters 1 and 6), adds the autonomy levels for security AI (Level 0 Advisory to Level 4 Autonomous), defined in Chapter 6, and organises its subject by NIST CSF 2.0 with the three focus areas of the NIST Cyber AI Profile (Chapter 3).
- Policy decision point outside the model and the AI gateway as the single seam (AI Platform and Security, Chapters 4 and 12).
- Containment profile, permission manifest and stop path (AI Platform and Security, Chapters 11 and 19).
- AI Decision Records (AIDR-NNNN, immutable and numbered); this book uses AIDR-0300 to AIDR-0399.
- Evidence manifest and evidence bundle; control catalogue identifiers such as
CTRL-GW-007 and CTRL-AGT-002.
- Threat model in
threat-model.yaml (STRIDE extended with ATLAS), the prompt-injection playbook PB-AI-001, the red-team plan RT-2026-11 and the scenario library RT-LIB-* (AI Platform and Security, Chapters 17 and 20).
- Horizon register and decision tiers (AI Governance, Chapters 13 and 24), including the horizon item
exploit-window-minutes owned by the CISO.
- Exceptions as expiring records (EXC-NNNN).
- Observable / Expected / Speculation labels for any outlook.
- The fictional organisations Nordvik Region and Hansa Mutual, with the security operations profile added for this book in
EXAMPLE_ORGANISATIONS.md.
Tables of contents for cross-references
Architecture as Code (Book 1): chapters most used
| Chapter |
Title |
| 9 |
Security Fundamentals for Architecture as Code |
| 9B |
Advanced Security Patterns and Implementation |
| 9C |
Risk as Code and Threat Handling as Code |
| 10 |
Policy and Security as Code in Detail |
| 11 |
Governance as Code |
| 13 |
Testing Strategies for Infrastructure as Code |
| 15A |
Evidence as Code and Continuous Assurance |
| 20 |
AI Agent Team for Architecture as Code Initiatives |
| 27 |
Conclusion |
AI Governance (Book 2)
| Part |
Chapter |
Title |
| I Foundations |
1 |
Introduction to AI Governance |
|
2 |
From Chat to Agents: How Large Language Models Are Used |
|
3 |
The EU AI Act and the European Regulatory Landscape |
|
4 |
Fundamental Principles |
|
5 |
Prerequisites and Readiness for Enterprise AI |
| II Sovereignty and Suppliers |
6 |
Geopolitics and Model Origin |
|
7 |
Procurement, Vendor Management and Open Models |
| III Risk, Governance, Trust and Assurance |
8 |
AI Risk Management and Classification as Code |
|
9 |
AI Governance as Code |
|
10 |
Trustworthy AI: Transparency, Fairness and Human Oversight |
|
11 |
Evidence, Assurance and Audit |
| IV Organisation and Leadership |
12 |
Organisational Design for AI |
|
13 |
Decision Forums and Governance Bodies |
|
14 |
Competence, Roles and the AI Architect |
|
15 |
Change Management and Responsible Adoption |
|
16 |
Management as Code for AI Programmes |
|
16A |
Governing AI Economics: Cost, Value and Portfolio Decisions |
|
17 |
AI in Public Authorities |
| V Experience |
18 |
Case Studies |
|
19 |
A Cautionary Case: The OpenAI–Hugging Face Incident |
|
20 |
Anti-Patterns |
|
21 |
Best Practices and Lessons Learned |
| VI Future and Conclusions |
22 |
Future Trends |
|
23 |
Nordvik Region and Hansa Mutual: Conclusions for Two Organisations |
|
24 |
Reflections on Each Part |
|
25 |
Conclusions: A Consolidated Analysis |
| Part |
Chapter |
Title |
| I Foundations |
1 |
Introduction to AI Platform and Security |
|
2 |
Usage Modes and the Platform: What Each Mode Demands |
|
3 |
Regulatory Requirements as Engineering Constraints |
|
4 |
A Reference Architecture for AI in Large Organisations |
|
5 |
AI Decision Records, Model Cards and the Architecture Repository |
| II Data, Models and DevSecMLOps |
6 |
Data Architecture for AI |
|
7 |
Model Lifecycle Management: MLOps and LLMOps as Code |
|
8 |
Setting Up an Effective DevSecMLOps Loop |
|
9 |
Risks and Vulnerabilities Across the DevSecMLOps Loop |
|
10 |
Foundation Models, Retrieval and Grounding |
|
11 |
Agentic Architectures and Orchestration |
|
12 |
Integrating AI into the Enterprise Landscape |
| III Infrastructure |
13 |
Infrastructure, Compute and European Digital Sovereignty |
|
14 |
Cloud AI Services versus On-Premises AI: Different Challenges |
|
15 |
Hybrid Agentic Work: Self-Hosted and Cloud Models for Sensitive Information |
| IV Security |
16 |
Security for AI Systems |
|
17 |
AI Security Engineering in Practice |
|
18 |
When AI Is the Threat: AI-Enabled Attackers and the Collapsing Exploitation Window |
|
19 |
When Your Own Agents Exceed Their Mandate: Containment That Does Not Depend on Intent |
| V Delivery and Operations |
20 |
Evaluation and Testing of AI Systems |
|
21 |
Observability, Monitoring and Drift in Production |
|
22 |
Cost, Capacity and Sustainability of AI |
|
23 |
From Pilot to Production: Scaling AI Across the Organisation |
| VI Conclusions |
24 |
Nordvik Region and Hansa Mutual: Platform and Security Conclusions |
|
25 |
Conclusions: Building and Securing AI Platforms |