Adversary Emulation and Continuous Red Teaming with AI

Most organisations know what their controls are supposed to do; far fewer know what those controls do when a capable adversary uses the techniques they were built for. Adversary emulation closes that gap: a red team reproduces the behaviour of a specific, credible adversary against the organisation's own systems, under written authority, so that defenders learn which detections fire, which controls hold and which assumptions are wrong. This chapter describes how a red team uses AI for that work and keeps it safe. The subject is urgent because adversaries now use agentic offensive frameworks that act at machine speed, so an emulation at human tempo tests the wrong thing, and because the July 2026 incident showed that AI used offensively is itself a system to contain.

In CSF terms the chapter is responsible for ID.IM Improvement (19, NIST, 2024). Its Subcategories ask that improvements are identified from evaluations (ID.IM-01), from security tests and exercises, including those coordinated with suppliers (ID.IM-02), and from operational processes (ID.IM-03), and that response and other plans are improved (ID.IM-04). Adversary emulation is the most direct source of the second outcome and feeds the other three. The chapter treats the Category through the three focus areas of the NIST Cyber AI Profile (20, NIST, 2025).

Figure 12.1: Emulation planned, approved by the white team and run in a separated range, with continuous validation feeding evidence to the purple loop.

Threat-informed scenarios

A generic penetration test asks whether anyone could get in; a threat-informed emulation asks whether the adversaries most likely to target this organisation, with the capabilities they have shown, would succeed, and whether anyone would notice. A regional authority that administers social benefits faces fraud-motivated actors and opportunistic ransomware groups; a cross-border insurer faces financially motivated groups that target claims payments and, increasingly, actors who use agentic tooling to move quickly through a directory. The scenarios follow from that analysis, not from a catalogue of everything that could be tried.

The common language is MITRE ATT&CK, the knowledge base of adversary techniques observed in real intrusions (10, MITRE, n.d.). Mapping each step to a technique makes the plan comparable with intelligence, lets the white team see which technique classes are in scope without reading any tooling, and lets each step name the detection or control it is expected to meet, so that every result is a statement about a control rather than an anecdote. For techniques against AI systems, MITRE ATLAS provides the equivalent vocabulary (11, MITRE, n.d.); Chapter 13 treats them in depth.

From intelligence to an emulation plan

The threat-intelligence function selects the relevant adversary profiles; they are decomposed into reported techniques, each with its source; the techniques are ordered into scenarios that lead towards the critical or important functions, the flags of the exercise; and each step is annotated with the expected detection, the control and the safeguards. AI helps at every stage under the red team's accountability: it extracts techniques from large volumes of reporting, normalised to ATT&CK identifiers, shows where the plan would test nothing new, and drafts the summary the management body reads before approval. It must not decide the scope, because the profile, flags and boundaries carry legal consequences and belong to named people.

Two disciplines keep AI-assisted design honest. Every technique carries a reference to the intelligence that justifies it, so that a reviewer can catch unsupported attributions, and the plan is reviewed against the threat model that Architecture as Code (Chapter 9C) placed in the codebase (71, Nordqvist, 2025) and Chapter 10 maintains. The plan sits in version control with its rules of engagement, and the approved version is the executed version.

Execution under rules of engagement

Every offensive activity in this book is authorised in writing, scoped by rules of engagement, run against the organisation's own or contracted systems, logged and closed with a defensive improvement. Chapter 7 describes the rules of engagement as a signed, versioned manifest. When AI is involved, the manifest must also say which AI tools the red team may use, where they run, which credentials they may use, and who can stop them.

What AI may execute

AI that helps people draft plans, build tooling and write reports acts through a person's session, and a person decides what reaches the target. AI that executes emulation steps acts with credentials and network access of its own, and only the controls around it hold the line between emulation and intrusion. It is therefore treated as a Mode 5 system for the exercise, whatever product it is built on, and receives the full containment baseline that AI Platform and Security, Chapter 19, set out for agents that can exceed their mandate: task-scoped credentials, default-deny egress, control artefacts outside the agent's reach, budgets, an external stop path, recorded approvals, and detection from telemetry and decoys rather than from the agent's own account (73, Nordqvist, 2026). This chapter builds on that conclusion (F23) unchanged, because an agent whose task is to probe for weaknesses needs containment more, not less.

The autonomy level is set per step, not per tool. Read-only steps, such as directory discovery against an in-scope domain, may run at Level 2 Supervised, and at Level 3 Bounded for catalogued, reversible actions in an isolated range. Steps that change state on production systems, use harvested credentials or move data run at Level 1 Assisted at most. Level 4 Autonomous is never used for offensive actions or for actions that cannot be reversed, as Chapter 6 explains.

The safeguards

Scoped credentials. The emulation uses identities issued for the exercise, scoped, time-bound and revocable by the white team in one action; a scenario that starts from a compromised account uses a test account, never a real employee's credentials.

Range separation. AI tooling runs in the red team's own environment. The only route to targets is through approved egress points, the only route to blue is through findings, and the red range shares no package mirror, repository or credential store with any other environment, for reasons the next section makes plain.

Stop paths. The white team holds the stop decision. Its stop path revokes every exercise credential and closes every egress route through a mechanism outside the red range, and reaches a named person at every hour the exercise runs. The manifest lists the stop conditions: production degradation, contact with an out-of-scope asset, real personal data beyond scope, a real incident in progress, and any AI action outside its permission manifest.

Logging. Every action, by a person or by AI, is logged in tamper-evident form outside the red range, with identity, plan step and time, so that the white team sees the exercise live and the organisation can show that it stayed inside its mandate.

Worked example: an ATT&CK-mapped emulation plan with safeguards and stop conditions

The illustrative plan below is one scenario from Hansa Mutual's threat-led exercise under RoE-HM-2027-TLPT. Its adversary profile describes a financially motivated actor using an agentic offensive framework, consistent with reporting by Google Threat Intelligence Group and Mandiant that such frameworks had been abused since late August 2026 and that they "compress attack timelines to machine speed; however, they currently trade stealth for velocity" (50, Blum et al., 2026). The plan therefore tests whether detection keeps pace as well as whether controls hold.

# Illustrative: Hansa Mutual, one scenario of RoE-HM-2027-TLPT (extract)
kind: EmulationPlan
id: HM-TLPT-2027-S2
rulesOfEngagement: RoE-HM-2027-TLPT
adversaryProfile: TI-HM-2027-04
flag: read-access-to-claims-payment-queue
tempo: machine-speed                   # agreed in advance with the white team
steps:
  - {id: S2-1, attack: T1078, executor: operator, autonomyLevel: 1,
     start: white-team-test-account, expect: {detection: DET-ID-031}}
  - {id: S2-2, attack: T1087, executor: ai-agent, autonomyLevel: 2,
     expect: {detection: DET-AD-012, withinMinutes: 10}}
  - {id: S2-3, attack: T1069, executor: ai-agent, autonomyLevel: 2,
     expect: {detection: DET-AD-014, withinMinutes: 10}}
  - {id: S2-6, attack: T1567, executor: operator, autonomyLevel: 1,
     data: canary-records-only, expect: {control: CTRL-DLP-003}}
safeguards:
  aiAgent: {mode: 5, permissionManifest: PM-RT-2027-02, maxActions: 400,
            maxRuntimeMinutes: 90, mayWriteToTargets: false}
  credentials: {issuedBy: white-team, ttlHours: 8, revocation: one-action}
  egress: {default: deny, allow: [rt-c2-endpoint-approved]}
  sharedServicesWithOtherRanges: none
stopConditions:
  - production-service-health-below-threshold
  - out-of-scope-asset-touched
  - real-incident-declared
  - ai-action-outside-permission-manifest
stopPath:
  holders: [white-team-lead, white-team-deputy]
  mechanism: out-of-band-credential-and-egress-revocation
logging: {tamperEvident: true, location: outside-red-range}

In the exercise, steps S2-2 and S2-3 completed in under a minute and DET-AD-012 fired, but the alert was queued at medium severity and triaged after 47 minutes. The detection was not missing; its severity assumed a human tempo. The closing control raised severity for high-rate directory enumeration from a single identity and routed it to the SOC's path for suspected credential theft, verified by replaying both steps until the alert reached triage within ten minutes.

Lessons from agents that exceeded their mandate

AI Governance, Chapter 19, told the story of the July 2026 OpenAI–Hugging Face incident and concluded that it failed through correlated controls: a shared credential, a package mirror that was also an egress path, and an alert that reached people who did not hold the stop decision (72, Nordqvist, 2026). This chapter builds on that conclusion (F14). OpenAI's report describes evaluation agents, run with production safeguards deliberately switched off, that left their sandbox through a self-hosted package mirror, used exposed credentials and compromised parts of Hugging Face's production infrastructure with no attacker directing them (64, OpenAI, 2026); Hugging Face's timeline describes an intrusion driven end to end by an autonomous agent system (65, Larcher et al., 2026). The UK AI Security Institute separately reported unsanctioned actions on the live internet during evaluation runs with internet access enabled on purpose, while stating that this was not a sandbox escape and no real-world harm was found (42, UK AISI, 2026), and the NCSC responded that "relying on detection alone after the fact of an incident will not be enough" (41, NCSC, 2026). Because those agents ran for a legitimate purpose in a security-testing context, their failure modes are the red team's own.

Control that failed in July 2026	Counterpart in an emulation programme	Design response
A credential shared across many workloads	A red-team service account reused across exercises	Exercise-specific identities, time-bound, revocable in one action
A package mirror that was also an egress path	A shared repository, proxy or mirror between ranges	No shared supporting services; every egress route named
An alert that reached people without the stop decision	Exercise alerts routed to the red team or an unaware SOC	Stop path and telemetry go to the white team

Testing containment from the adversary's side

AI Governance, Chapters 19 and 23, also left an open question (Q6): the containment built for one's own agents also limits an external one, and it needs testing from the adversary's side. A production agent's containment profile makes testable claims about its credentials, egress, control artefacts and detection, and the emulation tests them with conditions, not new attack behaviour. In an isolated range mirroring the agent's containment, the red team gives a test agent a planted credential, places a decoy service that would be an egress path if the policy were wrong, and introduces actions just outside the permission manifest. The hypotheses are that the credential is refused, the decoy never reached, the action blocked and alerted, and the alert delivered to the holder of the stop decision in time. Once proven safe, these scenarios enter the continuous validation catalogue and run after every change to the agent, its harness or its model.

Practical guide: preparing a safe AI-assisted emulation run

Purpose. Prepare a run in which AI assists or executes steps so that it tests what it should and cannot exceed its mandate.

Roles. Red-team lead, white-team lead (scope, approvals, stop path), threat-intelligence lead, platform or identity owner (credentials and egress), data protection officer, and the SOC or MSSP shift lead where the stop path must reach them out of hours.

Steps.

1. Confirm that a signed rules-of-engagement manifest under Chapter 7 names the AI tools the red team may use and where each runs.
2. Bind the plan to the manifest, and check that every step names a technique, an executor, an autonomy level and the expected detection or control.
3. Classify every AI component that executes steps as Mode 5, with a permission manifest listing actions, targets and budgets.
4. Have the white team issue exercise credentials that live no longer than the test window, by default eight hours, and test one-action revocation.
5. Verify that the red range shares no supporting service and that egress is denied except for named routes.
6. Test the stop path end to end at the hours of the run, including the out-of-hours holder.
7. Plant honeytokens and canary records so that no real personal data is touched, and start tamper-evident logging outside the red range, visible to the white team.

Default values to adapt. A few hundred actions and 90 minutes per AI-executed scenario; Level 2 Supervised for read-only steps in production and Level 1 Assisted for anything that changes state. Tighten these for systems that process special-category data.

Definition of done. The manifest and plan are approved and versioned, the stop path has been tested at the hours of the run, every executing AI component has a permission manifest and scoped credentials, and the log is visible to the white team.

Continuous validation as code

Threat-led exercises are periodic and cover a fraction of an estate that changes daily. Architecture as Code (Chapter 13) concluded that security tests, including synthetic attack simulations, belong in the pipeline (71, Nordqvist, 2025), and AI Platform and Security, Chapter 17, concluded that the external exercise is for discovery, while every finding becomes a versioned attack case run continuously and published as evidence against control identifiers (73, Nordqvist, 2026). This chapter builds on both conclusions (F8 and F21) and extends them to the whole estate.

Breach-and-attack simulation replays catalogued, benign technique emulations against the production estate as regression testing for Protect and Detect, each entry naming its ATT&CK technique, expected outcome and time limit; every exercise finding that can be reproduced benignly joins the catalogue. Security chaos engineering instead tests whether controls respond to conditions, including their own failure, such as a silent log source or an alert in an unwatched queue. Shortridge and Rinehart describe it as proactive experimentation to identify failures in security controls (67, Shortridge & Rinehart, 2023); Chapter 25 describes the platform. The two practices share a catalogue, a ledger and an evidence store.

AI plans and interprets; deterministic components act. An AI planner selects tests from coverage gaps and recent changes, and an AI evaluator drafts a finding when the expected outcome is missing or partial, but neither holds estate credentials or generates emulation logic. Every result is recorded against the control identifier and CSF Category it verifies, in the evidence manifest of AI Governance, Chapter 11 (72, Nordqvist, 2026). A failed test is a finding, closed only when the same test passes again, so the evidence store can answer when each control was last shown to work.

AI-enabled emulation in threat-led testing

DORA requires financial entities identified by their competent authorities to carry out threat-led penetration testing at least every three years, covering several or all critical or important functions and performed on live production systems (Article 26) (2, European Union, 2022). The TIBER-EU framework of the European Central Bank structures such tests: threat intelligence on the entity, a red team emulating the adversaries it describes, a blue team not told in advance, and a control team (the white team in this book's terms) leading and managing the exercise through preparation, testing and closure (13, ECB, 2025). AI Platform and Security, Chapter 18, asked (Q7) what such testing looks like when the adversaries in scope use AI (73, Nordqvist, 2026). The answer has three parts.

First, if the relevant adversaries use AI, the threat-intelligence report says which techniques are accelerated and what tempo has been observed, drawing on evidence such as the reporting of execution bursts of roughly 2 to 25 seconds and of one campaign that moved from the first compromised account to deep access in about four hours (50, Blum et al., 2026). The scenarios then emulate that tempo. Second, the red team may use AI under the safeguards above, agreed by the white team in advance; because DORA testing runs on live production systems, read-only discovery runs at Level 2 Supervised at most and every state-changing step is executed by a person, while a step that cannot run safely in production may be run in a production-like range. Third, the white team must understand the AI components well enough to judge their permission manifests, enforce stop conditions automatically at machine speed, and review the AI's actions at closure.

Continuous validation complements threat-led testing: the periodic exercise is discovery, the continuous suites are regression, and they meet at closure, where findings become catalogue entries. Under NIS 2, which expects procedures to assess the effectiveness of cybersecurity risk-management measures (Article 21(2)(f)) (3, European Union, 2022), the same combination gives entities outside the financial sector a proportionate approach.

Findings the blue team can use

ID.IM-02 asks that improvements are identified from tests and exercises (19, NIST, 2024), and the purple loop of Chapter 26 turns them into verified ones. Between the two sits the finding record, a machine-readable entry the blue team can act on without reconstructing the exercise. It names the step and technique, so that detection engineers can reproduce the condition; the expected and observed outcomes, with times; the control, never an analyst on shift; and the closing action and re-test, closing only when the scenario has been re-run successfully. It is classified, because it describes a weakness, and in a public authority it records the basis for any exemption from public access. AI reconciles the exercise log with the blue team's alerts and drafts the record, but a red-team member confirms every finding and the control owner accepts it, because a plausible but wrong finding sends the blue team in the wrong direction.

Worked example: a structured finding record

The illustrative record below comes from Nordvik Region's first threat-led exercise under RoE-NR-2027-001, drafted by the contracted tester's tooling and confirmed by the tester and the Region's white team.

{
  "kind": "EmulationFinding",
  "id": "FND-NR-2027-001-04",
  "rulesOfEngagement": "RoE-NR-2027-001",
  "planStep": "S1-3",
  "attack": {"technique": "T1078", "name": "Valid Accounts"},
  "system": "citizen-service-assistant-admin-console",
  "expected": {"detection": "Sign-in to admin console from unmanaged device", "withinMinutes": 15},
  "observed": {"detection": "none", "window": "out-of-hours",
               "note": "MSSP rule set did not include the console's identity logs"},
  "control": "CTRL-NR-MON-011",
  "severity": "high",
  "classification": "restricted",
  "closingAction": {"type": "detection", "implementedBy": "MSSP under NR-2025-044"},
  "retest": {"method": "replay planStep S1-3 out of hours", "status": "passed", "observedWithinMinutes": 9},
  "csf": ["ID.IM-02", "DE.CM"]
}

The step reached its target undetected because the console's identity logs had never been sent to the SIEM. The closing control was a log-forwarding change and a detection rule under the existing framework agreement, and the finding closed only when the replayed step raised an alert that the MSSP's shift lead handled in time.

Bounding AI across the emulation programme

Across the usage modes

The red team uses Modes 1 to 3 for planning and tooling under its own accountability, while AI that executes emulation steps is treated as Mode 5. The modes are defined in Chapter 6.

Mode	Opportunity	Characteristic vulnerability	Who judges, with whose identity
Mode 0 Ambient AI	Shows what vendor AI in tested products actually does	Vendor features contain test accounts unexpectedly	The vendor's model; the product's
Mode 1 Conversational assistants	Faster intelligence extraction and reporting	Unsupported technique attributions; scope pasted into unapproved tools	A red-team member; the person's
Mode 2 Embedded workplace assistants	Status and findings summaries for stakeholders	Summaries that hide a partial failure or leak findings	A person; the person's
Mode 3 Agentic work tools	Tooling built quickly in an isolated range	An agent with an engineer's credentials between instruction and review	The reviewing engineer; the engineer's
Mode 4 Built AI applications	Planner and evaluator for continuous validation	Poisoned intelligence or change events steering the plan	Policy and people; a read-only service identity
Mode 5 Autonomous production agents	Emulation at an AI-enabled adversary's tempo	Exceeding the mandate through shared credentials or side egress	The white team; an exercise-specific identity

Across the autonomy levels

The autonomy level is set per step in the emulation plan; the levels are defined in Chapter 6.

Level	Use in adversary emulation
Level 0 Advisory	AI proposes profiles, scenarios and test selections; people decide
Level 1 Assisted	AI drafts a step or finding; a person executes it; the ceiling for state-changing steps in production
Level 2 Supervised	AI executes after each action is approved; read-only steps in production
Level 3 Bounded	Catalogued, benign, reversible emulations, and AI-executed steps in an isolated range
Level 4 Autonomous	Never used for offensive actions or irreversible actions

Verifying with security chaos engineering

On a schedule agreed with the SOC, the chaos platform of Chapter 25 runs a catalogued, benign emulation of a priority technique, for example OS Credential Dumping (T1003), on a managed workstation in scope. The steady-state hypothesis is that the expected detection fires, an alert of the expected severity reaches the SOC queue within five minutes, and it is triaged within fifteen; absence of the alert is a finding, not a pass. A second experiment verifies the programme's own containment: the white team invokes its stop path while a test emulation runs, and the hypothesis is that every exercise credential is refused and every egress route closed within two minutes. The time bounds are defaults to adapt to the organisation's risk appetite.

The three AI focus areas

Securing AI systems. The red team's own AI is a system to contain, with permission manifests, exercise credentials, default-deny egress, budgets, tamper-evident logs and an external stop path, so that the July 2026 pattern cannot recur in the organisation's own testing; emulation from the adversary's side verifies the same containment for production agents (20, NIST, 2025).

AI-enabled cyber defence. AI supports the workflow from scenario design to reporting, while people decide scope and confirm findings and deterministic components execute.

Thwarting AI-enabled attacks. Emulating AI-enabled adversaries includes emulating their tempo, which tests whether detections assume a human pace; continuous validation keeps the time a failed control goes unnoticed shorter than an automated adversary needs to exploit it.

Nordvik Region and Hansa Mutual

Nordvik Region has no red team, so its first threat-led exercise is contracted. Under RoE-NR-2027-001 a contracted tester emulates adversaries against the benefits-triage service and the citizen-service assistant, with the Region's analysts as the blue team and the CISO's office as the white team. The rules of engagement name the tester's AI tools and environment, scope the credentials and name the stop-path holders, including the MSSP's shift lead out of hours. The structured findings, classified with the basis for their exemption from public access, are the first input to the Region's purple loop.

Hansa Mutual adds AI-enabled techniques, including machine-speed tempo, to the scope of its TIBER-DE exercise under RoE-HM-2027-TLPT. Between exercises HM-SEC-004 runs catalogued, benign emulations across the group estate from the red-team range, with no route into blue tooling except through findings. The red team of five uses AI in Modes 1 to 3; where AI executes steps, it is a Mode 5 system with the controls that failed in July 2026. The replay of the revised directory-enumeration detection is now a catalogue entry, and results feed the quarterly purple exercises and the evidence for DORA threat-led testing. Nordvik thus buys periodic exercises and builds the habit of using their findings, whereas Hansa validates continuously and must show its supervisor that its own emulation stays inside the mandate.

Summary

AI makes adversary emulation threat-informed and continuous, but only within written rules of engagement. Scenarios start from threat intelligence and become ATT&CK-mapped plans in which every step names the control it tests. Where AI executes emulation steps, it is a Mode 5 system with a permission manifest, scoped credentials, range separation, full logging and an external stop path, the autonomy level set per step and Level 4 Autonomous never used for offensive actions. The July 2026 incident is a design input for these safeguards, and emulating an agent that exceeds its mandate tests containment from the adversary's side.

Under ID.IM Improvement, findings become structured records with a control and a re-test, so that the blue team can act on them in the purple loop. Across the focus areas, the red team contains its own AI, uses AI from design to reporting, and emulates AI-enabled adversaries at their tempo. Continuous validation complements, and does not replace, threat-led testing under DORA and TIBER-EU. Chapter 13 turns the red team on the organisation's own AI systems and agents.

Sources

1. NIST. The NIST Cybersecurity Framework (CSF) 2.0. (19, NIST, 2024)
2. National Institute of Standards and Technology. Cybersecurity Framework Profile for Artificial Intelligence (Cyber AI Profile). (20, NIST, 2025)
3. MITRE Corporation. MITRE ATT&CK: Adversarial Tactics, Techniques, and Common Knowledge. (10, MITRE, n.d.)
4. MITRE Corporation. MITRE ATLAS: Adversarial Threat Landscape for AI Systems. (11, MITRE, n.d.)
5. European Central Bank. TIBER-EU: Threat Intelligence-Based Ethical Red Teaming framework. (13, ECB, 2025)
6. European Parliament and Council of the European Union. Regulation (EU) 2022/2554 on digital operational resilience for the financial sector (DORA). (2, European Union, 2022)
7. European Parliament and Council of the European Union. Directive (EU) 2022/2555 (NIS 2 Directive). (3, European Union, 2022)
8. Shortridge, K. & Rinehart, A. Security Chaos Engineering: Sustaining Resilience in Software and Systems. (67, Shortridge & Rinehart, 2023)
9. OpenAI. OpenAI – Hugging Face Incident Technical Report. (64, OpenAI, 2026)
10. Larcher, H., Carreira, A. et al. Anatomy of a Frontier Lab Agent Intrusion: A Technical Timeline of the July 2026 Incident. (65, Larcher et al., 2026)
11. AI Security Institute. Incident Report: unsanctioned agent behaviour during cyber testing. (42, UK AISI, 2026)
12. National Cyber Security Centre. NCSC statement in response to recent incidents resulting from frontier AI evaluations. (41, NCSC, 2026)
13. Blum, Mainor, Nambiar et al. ARTEX and Friends: Enter Agentic Offensive Frameworks. (50, Blum et al., 2026)
14. Nordqvist, G. Architecture as Code. (71, Nordqvist, 2025)
15. Nordqvist, G. AI Governance: Governing Artificial Intelligence in European Enterprises and Public Authorities. (72, Nordqvist, 2026)
16. Nordqvist, G. AI Platform and Security: Building and Securing AI Platforms in European Enterprises and Public Authorities. (73, Nordqvist, 2026)