Data and Platform Security with and for AI

Security AI is only as trustworthy as the data it reads and the platform it runs on. A triage assistant handles user names, command lines and, in a public authority with health and social-care duties, fragments of special-category data; a detection-engineering assistant touches the repository that holds the entire detection logic. Each capability concentrates sensitive data in a new place and reads content that an adversary can write, so protecting security data and its platforms is the condition for using AI in defence at all.

This chapter covers two Categories of the Protect Function of NIST CSF 2.0 (19, NIST, 2024). PR.DS Data Security protects the confidentiality, integrity and availability of data at rest (PR.DS-01), in transit (PR.DS-02) and in use (PR.DS-10), and requires backups that are created, protected, maintained and tested (PR.DS-11). PR.PS Platform Security manages platform hardware, software and services, with outcomes for configuration management (PR.PS-01), maintenance and removal of software and hardware (PR.PS-02 and PR.PS-03), log records made available for continuous monitoring (PR.PS-04), prevention of unauthorised software (PR.PS-05) and secure software development (PR.PS-06). The chapter treats both Categories through the three focus areas of the NIST Cyber AI Profile (20, NIST, 2025).

Figure 16.1: Security data passing classification, a gateway and guardrails before reaching security AI, with attacker-controlled input treated as untrusted.

Data security for security data

Security teams are used to protecting the organisation's data; they are less used to protecting their own.

Why security data is sensitive

Security data is sensitive in three ways. It contains personal data, such as user names, locations and conclusions about named colleagues. It describes weaknesses, because findings, red-team reports and exception registers map where the organisation is easiest to attack. And it describes the defender: suppression lists and alerts closed as benign tell an adversary what will not be seen, so a leaked detection repository is worth more to an intruder than most production data.

AI creates new copies of this data in prompts, indexes, caches, logs and weights, and an assistant can disclose content to a user not entitled to the underlying case. OWASP lists sensitive information disclosure as the second of its ten risks for LLM applications, and data and model poisoning as the fourth (14, OWASP, 2025); both apply with particular force to security data.

Classifying security data

The first control is to classify security data by what it reveals, not by where it is stored. Organisations map the classes below to their existing scheme, which under NIS 2 is part of the asset management and access control that Article 21(2)(i) lists among the cybersecurity risk-management measures (3, European Union, 2022).

Class	Main exposure when AI processes it	Default handling
Raw telemetry	Personal data at volume; attacker-written fields	Minimise before the model; approved locations only
Alert and case data	Special-category data; conclusions about people	Retrieval scoped to the case and the analyst's entitlement
Case notes and findings	Weaknesses and named individuals	Highest internal class; no external models
Detection content	Reveals blind spots	Repository access control; signed changes
Threat intelligence	Originator's sharing restrictions	Restrictions carried as metadata into the policy point
Training and evaluation data	Poisoning; memorisation and disclosure	Provenance per record; protected held-out sets

The policy point acts on these classes, for example refusing case-note content bound for an unapproved location. Classification also identifies the backups that matter: detection content, guardrail policies, model artefacts and evaluation sets are security-critical data under PR.DS-11 and must be restorable to a known-good version, because an adversary who alters them silently can blind the defender without touching production.

Protection at rest, in transit and in use

Encryption with organisation-controlled keys, mutual authentication and access control that distinguishes people, services and agents must reach the new copies too: vector indexes, prompt logs, inference caches and fine-tuning datasets. DORA requires financial entities to maintain high standards of availability, authenticity, integrity and confidentiality of data whether at rest, in use or in transit (Article 9) (2, European Union, 2022), and NIS 2 lists policies on cryptography and, where appropriate, encryption (Article 21(2)(h)) (3, European Union, 2022). Neither makes an exception for the security function's own data.

Protection in use is where AI differs most, because during inference the data sits in the memory of the inference service. Security AI therefore runs on capacity the organisation controls or with an operator bound by contract to the chosen jurisdiction (Chapter 4), and the policy point refuses, not merely logs, a call that would process a data class outside its approved location. Minimisation is the cheapest protection in use: a triage assistant needs the verdict, sender domain and attachment type, not the full mail body.

Training data, poisoning and extraction

Security AI learns from the defender's own history, which the adversary partly writes. The NIST taxonomy of adversarial machine learning distinguishes poisoning attacks on training data or model components from evasion attacks at inference time and from privacy attacks that extract information about training data or the model (18, Vassilev et al., 2025); MITRE ATLAS gives the corresponding techniques a shared vocabulary (11, MITRE, n.d.).

Three controls follow: provenance on every training and evaluation record; held-out evaluation sets protected from the feedback loop, so that a poisoned trend in recent verdicts shows up as a quality drop rather than a quietly moved baseline; and disclosure testing before release (Chapter 13). Against extraction through the AI service itself, a planted canary record, a fictitious case whose identifiers exist nowhere else, turns any appearance of those identifiers outside its boundary into a high-confidence signal (Chapter 19).

Guardrails and policy points

This section builds on two conclusions of AI Platform and Security. Prompt injection is a property of the architecture, not a patchable defect, so the controls that matter sit outside the model (AI Platform and Security, Chapter 16) (73, Nordqvist, 2026). And a platform is under control when every model and tool call passes a policy point outside the model, containment holds whatever anyone intends, and evidence is produced by normal operation (AI Platform and Security, Chapter 25) (73, Nordqvist, 2026), with the AI gateway as the single seam (AI Platform and Security, Chapters 4 and 12). Security AI uses the same gateway.

Why security AI is a special case

Most AI applications read some untrusted content; security AI reads little else, since alert descriptions, file names and user-agent strings are chosen by whoever caused the event. Hansa Mutual's campaigns in Chapter 13 found that crafted alert content could steer its investigation agent's proposals. Consequently, guardrails for security AI do not try to recognise malicious instructions, which no filter is known to do reliably (14, OWASP, 2025), but limit what any input can cause.

Layers of a guardrail policy

A guardrail policy has six layers, each enforced at the policy point or by the platform rather than by the model.

1. Input handling. Attacker-controlled fields are passed as delimited, labelled data, never concatenated into instructions; input size is limited, unneeded fields are removed and every span keeps its provenance.
2. Retrieval scope. Retrieval runs with the entitlement of the person on whose behalf the capability acts and is scoped to the case in hand.
3. Tools. The capability receives only the tools its purpose requires, with constrained parameters. At Level 1 Assisted it needs read tools only; write tools belong to capabilities whose autonomy level has been decided under Chapter 6 and whose containment profile is in place, otherwise OWASP's excessive agency risk applies (14, OWASP, 2025).
4. Output checks. Outputs are validated against a schema, checked for identifiers absent from the input case and stripped of links or executable fragments, which addresses OWASP's improper output handling (14, OWASP, 2025).
5. Budgets. Calls per alert and per hour are capped, so that a flood of crafted alerts cannot become a cost or availability problem.
6. Evidence. Every decision of the policy point is logged with the policy version.

Injection classifiers in the input layer reduce noise, but they are not the ceiling. The ceiling is set by the tools, the identity and the output checks, which hold whether or not a manipulation is recognised.

Guardrails as code

Architecture as Code (Chapters 10 and 11) established policy as code with the principle "assure once, comply many", a control written once and mapped to the obligations it serves, and (Chapter 15A) evidence collected by machines, signed and versioned (71, Nordqvist, 2025). Accordingly, a guardrail policy lives in version control, is reviewed like code and is tested against a scenario library before each change. A change to a guardrail is a change to a control.

Practical guide: applying a guardrail policy to a security AI capability

Purpose. Put a security AI capability behind a guardrail policy enforced outside the model, before it reads production security data.

Roles. Capability owner (usually the SOC lead), platform team (gateway and policy point), privacy lead or data protection officer, AI red-teaming lead or contracted tester, CISO (approval).

Steps.

1. Record the capability's usage mode, autonomy level, identity and decision record.
2. Mark every input field as trusted, organisation-written or attacker-controlled; when in doubt, attacker-controlled.
3. Agree the purpose allow-list of fields with the privacy lead.
4. Allow-list tools with constrained parameters; at Level 0 Advisory and Level 1 Assisted, read tools only.
5. Define the output schema and checks, with refusal on violation.
6. Set budgets, starting with four model calls per alert and an hourly ceiling of three times the expected peak, adapted to the risk appetite after a month.
7. Commit the policy, link it to the decision record and require review by someone other than its author.
8. Run the scenario library, including crafted content in each attacker-controlled field, and keep the results as evidence.
9. Add the capability to the chaos engineering catalogue (Chapter 25).

Definition of done. The capability reaches a model only through the policy point; the policy is versioned, reviewed and linked to the decision record; the scenario library passes; refusals reach the owner; and a scheduled experiment verifies the policy.

Worked example: guardrail policy for the SOC triage assistant

The illustrative policy below is Nordvik Region's for NR-SEC-001, the SOC triage assistant introduced in Chapter 18: a Mode 4 service at Level 1 Assisted with a read-only service identity, self-hosted on the Region's reserved capacity because alert data contains personal and health data (AIDR-0301).

# Illustrative: Nordvik Region, guardrail policy for NR-SEC-001
kind: GuardrailPolicy
id: NR-SEC-001-guardrails
version: 1.0.0
capability: NR-SEC-001            # SOC triage assistant
decisionRecord: AIDR-0301
usageMode: 4
autonomyLevel: 1                  # Level 1 Assisted
enforcedAt: policy-point-before-model
hosting: {location: nordvik-reserved-capacity, processingRegion: EU-EEA, refuseCallsOutsideRegion: true}
identity: {serviceIdentity: svc-nr-sec-001, access: read-only}
input:
  attackerControlled: [alert.description, mail.subject, mail.senderName,
                       file.name, process.commandLine, url, userAgent]
  attackerControlledHandling: delimited-labelled-data-never-instructions
  purposeAllowList: triage-fields-v3     # agreed with the privacy lead
  dropFields: [mail.body, attachment.content]
retrieval: {scope: current-case-only, entitlementOf: requesting-analyst}
tools:
  allowed: [siem.query.readonly, case.read]
  writeTools: none
  externalNetwork: deny
output:
  schema: triage-note-v2
  personalIdentifiers: only-those-present-in-case
  links: forbidden
  onViolation: block-log-and-notify-owner
budgets: {maxModelCallsPerAlert: 4, maxModelCallsPerHour: 900}
evidence: {logDecisionsWithPolicyVersion: true}
verification:
  scenarioLibrary: RT-LIB-NR-TRIAGE
  runOn: [policy-change, model-change, weekly]
  chaosExperiment: SCE-PR-016-canary

Dropping mail bodies and attachments removes the largest block of attacker-written text and personal data in one step, and the absence of write tools limits the worst outcome of a manipulated input to a misleading note that an analyst reviews.

Secure configuration with AI

Configuration as the attack surface

An intruder who finds an exposed management interface or logging switched off during a migration needs no software vulnerability at all, and AI helps adversaries find such misconfiguration faster. AI Platform and Security, Chapter 18, concluded that security operations must run faster than a monthly cycle (73, Nordqvist, 2026), and a misconfiguration reviewed quarterly is open to an automated adversary for weeks.

The AI platform's gateway routes, guardrail policies, egress rules and service-identity permissions are part of this surface, and a guardrail switched off is a worse drift than most, because the capability behind it reads attacker-controlled content. Log records from the gateway and the policy point fall under PR.PS-04: they must be produced and made available for continuous monitoring (19, NIST, 2024), and their absence is itself a signal for the SOC (Chapter 18).

Baselines and drift as policy as code

The desired configuration is held as code, a policy evaluates the deployed configuration against it, and deviations are findings with an owner. The illustrative baseline below, which Hansa Mutual's group platform applies to every security AI workload, refuses gateway bypass, non-default-deny egress, processing outside Frankfurt and unsigned model artefacts.

# Illustrative: Hansa Mutual, baseline for security AI workloads
package hansa.platform.secai_baseline

import rego.v1

allowed_regions := {"hansa-dc-frankfurt"}

deny contains msg if {
    input.workload.securityAi
    not input.workload.route.viaGateway
    msg := sprintf("%s: model calls bypass the AI gateway", [input.workload.id])
}

deny contains msg if {
    input.workload.securityAi
    input.workload.egress.default != "deny"
    msg := sprintf("%s: egress is not default-deny", [input.workload.id])
}

deny contains msg if {
    input.workload.securityAi
    not input.workload.processingRegion in allowed_regions
    msg := sprintf("%s: processing outside approved region", [input.workload.id])
}

deny contains msg if {
    some model in input.workload.models
    not model.signatureVerified
    msg := sprintf("%s: unsigned model artefact %s", [input.workload.id, model.name])
}

The same policy blocks non-compliant deployments in the pipeline and detects drift in the running estate, where a change made outside the pipeline, by an engineer, an automation or an intruder, would otherwise go unseen.

Deciding what reverts automatically

Reverting a rule an incident responder widened ten minutes ago can break containment, so the book recommends deciding the response per control class. Drift in a reviewed class, such as a gateway bypass or egress beyond default-deny, is reverted automatically and reported; other drift goes to an owner with a deadline; and drift during a declared incident goes to the incident commander. Each class records who approved its response, making automatic reversion a decision made in advance, as AI Governance recommends for patch decisions (AI Governance, Chapter 24) (72, Nordqvist, 2026).

PR.PS-02 extends the same reasoning to software. NIST frames enterprise patch management as preventive maintenance, planned and routine rather than reactive (29, Souppaya & Scarfone, 2022), and Chapter 11 treats patch velocity as a measured service level. For security AI the scope includes model versions, inference runtimes and agents' skills and connectors, and PR.PS-05 treats unapproved models and unregistered connectors like any other unauthorised software.

What AI adds

At Level 0 Advisory or Level 1 Assisted, AI drafts candidate policies from hardening guides, explains drift from change history, ranks deviations by exposure (Chapter 11) and drafts the correcting pull request. Architecture as Code (Chapter 9C) put the principle in a phrase: celebrate the remediation pull request, not just the discovery (71, Nordqvist, 2025). However, AI neither decides which classes revert automatically nor holds the credentials that perform the reversion; the deterministic policy engine does that, under the decision recorded for each class.

Secure development with AI assistance

PR.PS-06 applies to the security function's own detections, playbooks, guardrail policies, models and pipelines.

Review for soundness, not only for syntax

Architecture as Code (Chapter 27) left an open question for this book: version control records what changed, but not whether the reasoning behind a change was sound (71, Nordqvist, 2025). For security content the question is sharp, because a plausible detection that passes its unit tests can still miss its technique, and a suppression rule can silently remove a true signal.

Three practices answer it in part: every AI-assisted change states its intent (technique, data source, expected false positives); every detection change is tested against recorded true-positive and true-negative events (Chapter 19); and changes that reduce what the defender sees, such as suppression lists and guardrail policies, require a second reviewer who did not prompt the assistant.

Provenance through the pipeline

The commit records in a trailer that an approved assistant contributed, the build signs the artefact, and the deployment verifies the signature. With that chain the organisation can list and re-review everything produced with a given assistant in a given period, which is the impact query that AI Platform and Security, Chapter 21, adds to AI incident management (73, Nordqvist, 2026).

For bought products, the Cyber Resilience Act requires manufacturers of products with digital elements to identify and document components, including by drawing up a software bill of materials, among the vulnerability-handling requirements of Annex I (5, European Union, 2024); most of its obligations apply from December 2027. A deployer can therefore require bills of materials that include model components through the contracts of Chapter 8. NIS 2 lists security in the acquisition, development and maintenance of network and information systems among the risk-management measures (Article 21(2)(e)) (3, European Union, 2022), which covers the organisation's own security tooling.

Integrity of models and pipelines

Because model loading is code execution (AI Platform and Security, Chapter 16) (73, Nordqvist, 2026), a model from a public hub or a supplier is treated like any third-party binary. Models come only through the organisation's registry, formats that cannot execute code on load are preferred, the runtime verifies every artefact's signature as the Rego baseline enforces, and pipelines run with their own narrow identities behind an evaluation gate (AI Platform and Security, Chapters 7 and 8).

The AI Act makes the same point for high-risk AI systems. Article 15 requires them to be resilient against attempts to alter their use, outputs or performance by exploiting vulnerabilities, and names data poisoning, model poisoning, adversarial examples and confidentiality attacks among the AI-specific vulnerabilities to address (1, European Union, 2024). Few security AI capabilities are high-risk, but the list is a sound checklist for all of them.

Development environments are part of the platform

AI Governance, Chapter 19, concluded that the July 2026 incident failed through correlated controls, among them a package mirror that was also an egress path (72, Nordqvist, 2026). Package mirrors, model registries and build runners are thus monitored as platform with default-deny egress. Because a Mode 3 coding agent runs with an engineer's credentials, detection engineers' agents work in isolated ranges without workstation secrets, read only sanitised data and reach the repository only through reviewed pull requests.

Data and platform security across modes, levels and focus areas

Across the usage modes

Data exposure grows with the usage mode, and so does the platform's responsibility (see usage modes).

Mode	Opportunity	Characteristic vulnerability	Who judges, with whose identity
Mode 0 Ambient AI	Vendor AI classifies data and flags misconfiguration	Telemetry processed where and for purposes the contract may not state	The vendor's model; the product's identity
Mode 1 Conversational assistants	Fast drafting of queries and rules	Analysts paste raw telemetry or case notes	The analyst; the analyst's
Mode 2 Embedded workplace assistants	Summaries in ticketing and SOAR tools	Inherited permissions surface unneeded cases	The user; the user's, delegated
Mode 3 Agentic work tools	Agents write and test detections	Workstation secrets within the agent's reach	The reviewing engineer, after the act; the engineer's
Mode 4 Built AI applications	Tailored triage under the organisation's guardrails	Attacker-controlled fields reach the model	A person acting on the output; a read-only service identity
Mode 5 Autonomous production agents	Enrichment and bounded action at machine speed	Attacker-controlled fields reach a model with its own permissions	The policy point and approver; the agent's workload identity

Modes 4 and 5 need the full guardrail policy, and Mode 5 also the containment profile, permission manifest and stop path of AI Platform and Security, Chapter 19 (73, Nordqvist, 2026).

Across the autonomy levels

The levels matter most for configuration remediation (see autonomy levels).

Level	Use in data and platform security
Level 0 Advisory	The AI explains and ranks drift; a person decides and acts
Level 1 Assisted	The AI drafts the correcting pull request; a person reviews and merges it
Level 2 Supervised	The AI triggers a reversion after a person approves each one
Level 3 Bounded	Pre-approved reversions for reviewed control classes; in practice performed by deterministic automation, not the model
Level 4 Autonomous	Not used for security-control configuration or data handling

Verifying with security chaos engineering

Three experiments from Chapter 25's catalogue verify the controls, in test environments with synthetic records only.

• Configuration drift in a test tenant. Condition: an actuator disables policy-decision logging for a test security AI workload and opens its egress. Steady-state hypothesis: the baseline detects and reverts both deviations, and the same change through the pipeline is blocked. Time bound: detection within 10 minutes, reversion within 15.
• Canary record. Condition: a synthetic case with a canary identifier is placed in the triage assistant's case store, and an actuator moves the identifier into an outbound message to a test address and a request to an unapproved model endpoint. Steady-state hypothesis: data-loss detection alerts on the message and the policy point refuses and logs the call. Time bound: alert within 15 minutes; refusal immediate.
• Guardrail regression. Condition: a catalogued benign test alert with a marker phrase in its description is injected into the test SIEM. Steady-state hypothesis: the marker appears only inside the labelled untrusted-data span, the output contains no identifiers absent from the case, and no tool beyond the allow-list is called. Time bound: one triage cycle after every policy or model change.

The three AI focus areas

Securing AI systems. Provenance on training records, protected evaluation sets, signed artefacts and narrow pipeline identities protect data, model and pipeline integrity; guardrails outside the model limit what any input can cause, and configuration as code keeps them in place (20, NIST, 2025).

AI-enabled cyber defence. AI drafts baselines, explains drift and drafts remediation pull requests under review; what to enforce stays with people and deterministic policy.

Thwarting AI-enabled attacks. Minimisation, scoped retrieval, output checks and canary records counter extraction through AI services; provenance and protected evaluation sets counter poisoning (18, Vassilev et al., 2025); and continuous drift detection narrows the window in which an AI-assisted adversary can abuse a misconfiguration.

Nordvik Region and Hansa Mutual

Nordvik Region runs NR-SEC-001 self-hosted on capacity its European hosting provider reserves for it, and the MSSP's processing and log storage stay inside the EU/EEA under framework agreement NR-2025-044. Relying largely on supplier defaults, it applies policy as code first to its SIEM and email-security gateway, with drift reported to an analyst; automatic reversion waits until the suppliers agree to it.

Hansa Mutual enforces data-residency overlays per market, so UK and Swiss telemetry is handled under its own transfer rules and security AI in Frankfurt receives only the fields its purpose requires. Guardrails for HM-SEC-001 and HM-SEC-002 run at the gateway, and configuration is policy as code across the group cloud with automatic reversion for reviewed classes. Nordvik's controls are thus shaped by sovereignty on a small self-hosted footprint, Hansa's by residency across seven jurisdictions on a platform it builds itself.

Summary

Security data is sensitive data, because it reveals personal information, weaknesses and the defender's blind spots. Under PR.DS Data Security it is classified by what it reveals and protected at rest, in transit and in use, with minimisation, provenance on training data and processing placed where sovereignty rules require; detection content and guardrail policies must be restorable to a known-good version.

Under PR.PS Platform Security, the controls that protect AI systems sit outside the model, so that the consequences of a manipulated input do not depend on recognising it. Configuration is policy as code with automatic reversion only for pre-reviewed classes, and secure development makes AI-assisted reasoning testable and treats model loading as code execution.

Exposure grows with the usage mode, and chaos experiments verify the controls through drift, a canary record and a marked test alert. Chapter 17 completes Part IV with the infrastructure resilience that limits attacks at machine speed.

Sources

1. NIST. The NIST Cybersecurity Framework (CSF) 2.0. (19, NIST, 2024)
2. National Institute of Standards and Technology. Cybersecurity Framework Profile for Artificial Intelligence (Cyber AI Profile). (20, NIST, 2025)
3. OWASP Foundation. OWASP Top 10 for Large Language Model Applications, 2025 edition. (14, OWASP, 2025)
4. Vassilev, A., Oprea, A., Fordyce, A., Anderson, H., Davies, X. & Hamin, M. Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations. (18, Vassilev et al., 2025)
5. MITRE Corporation. MITRE ATLAS: Adversarial Threat Landscape for AI Systems. (11, MITRE, n.d.)
6. Souppaya, M. & Scarfone, K. Guide to Enterprise Patch Management Planning: Preventive Maintenance for Technology. (29, Souppaya & Scarfone, 2022)
7. European Parliament and Council of the European Union. Regulation (EU) 2024/1689 (Artificial Intelligence Act). (1, European Union, 2024)
8. European Parliament and Council of the European Union. Regulation (EU) 2022/2554 on digital operational resilience for the financial sector (DORA). (2, European Union, 2022)
9. European Parliament and Council of the European Union. Directive (EU) 2022/2555 (NIS 2 Directive). (3, European Union, 2022)
10. European Parliament and Council of the European Union. Regulation (EU) 2024/2847 (Cyber Resilience Act). (5, European Union, 2024)
11. Nordqvist, G. Architecture as Code. (71, Nordqvist, 2025)
12. Nordqvist, G. AI Governance: Governing Artificial Intelligence in European Enterprises and Public Authorities. (72, Nordqvist, 2026)
13. Nordqvist, G. AI Platform and Security: Building and Securing AI Platforms in European Enterprises and Public Authorities. (73, Nordqvist, 2026)