Security Monitoring and Anomaly Detection

Traditional security monitoring watches bytes, packets, access control events, and file system changes. These signals are structural: a port scan looks different from normal traffic at the network layer. A privilege escalation leaves an audit trail in the operating system. The threat is visible in the metadata.

AI security threats are different in three fundamental ways that require a different monitoring approach.

The signal is in the semantics, not the bytes. A malicious prompt injection and a benign customer support query arrive through the same authenticated API channel with the same headers and the same byte count. The threat is in the meaning of the text, not the structure of the packet. No traditional network security tool can see this difference.

The attacker is often authenticated. A distillation campaign operates through valid API keys. A jailbreak attempt comes from a paying user. An insider running extraction queries has legitimate access. Access control events show nothing unusual because the attacker is authorized to make requests. The attack is in what they request and what they do with the response.

Model drift is a security event. A model whose behaviour has shifted may have been poisoned through a compromised update pipeline, fine-tuned on adversarial data, or jailbroken by a technique that the deployment team has not detected. A 15-percentage-point drop in safety refusal rates over two weeks is not a performance metric. It is a potential security incident.

A production AI system is not one thing. It has a query intake layer, an inference layer, a retrieval layer (if it uses RAG), an agent layer (if it uses autonomous workflows), and a model layer. Each layer has distinct security signals that are invisible to the others. Missing any layer leaves a monitoring blind spot.

The input layer sits at the front of the AI system: every query passes through it before reaching the model. It is the first place where anomalies can be detected, but also the layer where detection is hardest, because malicious queries are semantically crafted to look benign.

The inference layer is where the model produces output. Security signals at this layer are about what comes out, not what went in. AgentIQ runs inline at this layer, classifying every output before it reaches the user and feeding those classifications into the monitoring pipeline.

In a RAG deployment, the retrieval layer is where user queries reach the vector database. This layer has a specific attack surface: an attacker who can observe or manipulate retrieved documents can inject content into the model's context window. Retrieval anomalies are often the first visible signal of an indirect prompt injection attack.

Agentic AI deployments introduce a new category of monitoring signal: the actions an agent takes, not just the text it produces. An agent that calls a payments API, spawns child agents, reads from a database, and sends emails in one session has a much larger security footprint than a chatbot that produces text. The agent layer requires monitoring the actions, not just the outputs.

Model drift is normally treated as a performance issue. In a security context, it is also a security event. A model that was safe at deployment may not be safe after a fine-tuning update, after a new jailbreak technique becomes public, or after a backdoor was inserted during an upstream supply chain compromise.

The distinction matters: a model whose refusal rate on a standard probe set drops from 92 percent to 74 percent over two weeks has drifted by 18 percentage points. That is detectable before the model causes harm if you are running the probes. NIST's evaluation found that a named frontier model responded to 94 percent of malicious requests under common jailbreaking techniques. The model had drifted from its designed safety posture.

Model drift monitoring requires a held-out evaluation set that does not change between measurements. If your evaluation set changes between measurements, you cannot distinguish model drift from evaluation drift.

Distillation attacks extract the reasoning capabilities of a frontier model through large-scale systematic querying. The attacker builds a training dataset of (prompt, reasoning, response) triples from your model, then trains a student model on this dataset. Done at scale, the student model approximates the frontier model's capabilities at a fraction of the development cost.

In February 2026, Anthropic documented over 16 million exchanges generated by three Chinese AI laboratories across roughly 24,000 fake accounts targeting its Claude models. One proxy network managed more than 20,000 simultaneous fraudulent accounts, mixing extraction traffic with legitimate queries to camouflage the operation. OpenAI and Google's Threat Intelligence Group made similar disclosures in the same period.

What distillers actually steal: reasoning capability (chain-of-thought traces teach students how to decompose problems and verify intermediate steps), safety properties (distilled models inherit capability but shed safety alignment), and architectural insight (systematic extraction reveals how the model structures its reasoning across domains).

The monitoring challenge: detection approaches that focus on individual queries all fail. Any transformation that preserves a response as useful to a human also preserves its training signal for a student model. The defense must operate at the population level.

The most important insight in AI security monitoring: most attacks are invisible at the individual query level and visible only at the population level. A single distillation query, a single injection attempt that did not succeed, and a single account with slightly higher than normal query volume are all noise. The signal emerges across thousands of queries, across many accounts, over days or weeks.

Building population-level monitoring requires aggregating signals over time windows (rolling 24-hour, 7-day, and 30-day windows for different signal types), across accounts (grouping by account age cohort, IP range, and query embedding cluster), and across the semantic space of the model (tracking which capability regions of the model have been queried and how uniformly).

This kind of monitoring infrastructure is not built into any standard API gateway. It requires custom telemetry pipeline design: query embeddings must be computed and stored (not the queries themselves), account-level aggregates must be maintained in real-time, and alert thresholds must be calibrated against the genuine user population baseline before anomalies become meaningful.

AI security monitoring requires logging, but AI systems process sensitive data that must not appear in logs. The solution is to log security signals rather than content: derived metadata that tells you what happened without telling you what was said.

This also makes the logs useful for security analysis. A log file full of raw query text is hard to analyse statistically. A log file full of structured fields (hash, embedding cluster, PII flag, hallucination score, injection detected) is directly queryable by a SIEM.

A minimal viable AI security monitoring dashboard covers at least one metric per layer with an alert threshold. The thresholds below are starting points. Calibrate against your actual user population baseline before activating alerts: a threshold that is correct for a consumer chatbot will generate constant false positives for a developer API.

MITRE ATLAS (Adversarial Threat Landscape for Artificial-Intelligence Systems) provides a framework for classifying AI-specific attack techniques. Each technique has a monitoring signal that can be instrumented. Mapping your monitoring coverage to ATLAS tells you which attack techniques you can detect and which you cannot.

AgentIQ runs inline at the inference layer, classifying every model output before it reaches the user or triggers a downstream agent action. Each classification produces a structured event record that feeds directly into the security monitoring pipeline described in this module.

The per-output events from AgentIQ are the foundation of inference-layer monitoring. Without them, monitoring the inference layer requires either logging raw outputs (which creates a privacy problem) or building a separate output scanning pipeline (which adds latency and infrastructure). AgentIQ produces the inference-layer monitoring signal as a side effect of its inline enforcement role.

In aggregate, AgentIQ events answer the questions that the inference layer metrics require. What fraction of outputs contained PII in the last hour? Has the hallucination score distribution shifted this week? Is there an active injection campaign: how many outputs have been flagged as injection-affected in the last 15 minutes? These are all derived from AgentIQ's per-output classification stream.

AgentIQ's chain security validation is specifically relevant for agentic monitoring. In multi-step workflows, it checks whether each step in the agent's reasoning chain is consistent with the delegated task and the AgentID token scope. A chain that attempts to justify an out-of-scope action is flagged before the action reaches the Resource Gateway, providing defence in depth: the chain security check catches the problem at the reasoning layer, and the gateway enforces it at the action layer.

DiscoveR provides the model-layer monitoring function described in Section 07. It runs structured adversarial tests against your deployed model on a schedule and after model updates, comparing results against the previous scan to detect drift.

The core monitoring workflow: run a DiscoveR baseline scan against the model before deployment. Store the per-category pass rates as the baseline. Run the same scan after every model update and on a weekly schedule. Compare new pass rates against the baseline. Any category where the pass rate has dropped is a potential security regression that blocks the update or triggers an investigation.

The correlation_id feature links scans across remediation cycles. If a DiscoveR scan finds a jailbreak vulnerability and the engineering team deploys a fix, the next scan with the same correlation_id compares only the tests that failed in the previous scan. This confirms that the specific vulnerabilities were addressed and not just that overall pass rates stayed constant while new vulnerabilities appeared.

For continuous monitoring between updates, DiscoveR can be run on a schedule against production endpoints. This catches two things that post-update scanning misses: vulnerabilities introduced by prompt changes (not model changes) and drift that accumulates gradually rather than appearing suddenly after an update.