AI as an Attack Tool

Before AI, most sophisticated attacks were skill-constrained. A credible spear-phishing campaign required a researcher who could investigate a target, write convincingly in their voice, identify the right pretext, and time the delivery. Ransomware required developers. Vulnerability research required people who understood both code and exploitation technique.

AI removed the skill bottleneck. What changed is not what attacks are possible but how many can be run simultaneously and how little expertise is required to run them. The ceiling on attacker scale was human attention. AI removes that ceiling.

The practical consequence: the volume of sophisticated-looking attacks has increased dramatically. Defences designed around the assumption that sophisticated attacks are rare and resource-intensive are now calibrated for the wrong threat model.

AI-powered attacks fall into six categories that are operationally significant today. Each has different targets, different detection challenges, and different defences. The first three (phishing, malware, vulnerability research) are AI augmenting traditional attack types. The last three (deepfakes, model extraction, RAG poisoning) are new attack classes that only exist because AI exists.

Traditional phishing detection was calibrated to catch generic messages sent to large recipient lists. The same template with minor variations, sent to thousands of people. Detection filters were good at this because the attack economics required scale over personalisation: mass campaigns were cheaper than bespoke ones.

AI inverted this tradeoff. An LLM can generate a highly personalised spear-phishing email from publicly available information in seconds. For each target, it can research their employer, recent news about their company, their LinkedIn connections, and their public posts. It can then write a message that sounds like it came from a known colleague, references a real recent event, and uses the target's communication style.

Malware development has traditionally required programming skill and knowledge of the target environment. LLMs have reduced the expertise barrier significantly. An attacker with access to a capable coding model can request code that performs specific malicious actions, iterate on it to evade detection, and generate many variants to maximise the chance that at least one passes through security controls.

Polymorphic generation is the most practically significant capability. Signature-based antivirus and EDR detection relies on known patterns in malware code. If the attacker can generate functionally identical malware with different code structure, variable names, and calling conventions each time, each new sample appears novel to signature-based detection. LLMs are good at this kind of structural variation while preserving function.

Evasion assistance is also significant. LLMs can help an attacker understand why a specific malware sample was flagged, suggest modifications to avoid the detection pattern, and generate testing harnesses to verify the modification worked. This accelerates the iteration cycle from hours to minutes.

The capability ceiling is still meaningful: frontier LLMs include safety measures that limit direct malware-as-a-service usage. But jailbreaks against these safety measures are documented and actively maintained by attacker communities. The DiscoveR jailbreak category tests your own AI deployment against the same techniques used to jailbreak safety measures on general-purpose models.

Vulnerability research is one of the most direct applications of AI to offensive capability. AI can analyse large codebases much faster than humans and can be trained or prompted to look for patterns similar to known vulnerability classes. This does not replace the insight of an experienced security researcher, but it dramatically compresses the time from code exposure to identified vulnerability candidate.

The practical workflow: an AI-assisted vulnerability scanner ingests the target codebase, applies pattern matching based on known vulnerability classes (buffer overflows, injection points, authentication bypasses, race conditions), ranks candidates by confidence and exploitability, and produces a shortlist for human review. A human researcher then validates the candidates and develops working exploits. The AI handles the search; the human handles the exploitation logic.

For AI-specific systems, this creates a specific threat: automated scanning of your AI deployment's attack surface. DiscoveR does exactly this from the defensive side, 60 plus attack modes scanning for AI-specific vulnerabilities. Attackers are building equivalent offensive tools. The question for defenders is whether their defensive scanning runs before or after the attacker's offensive scan finds the same vulnerability.

Deepfake technology has moved from a research curiosity to a practical attack tool. The most operationally significant capability today is voice cloning for real-time audio impersonation during phone calls. Unlike video deepfakes that require careful generation and review, real-time voice cloning can be run live, allowing an attacker to impersonate a CEO, CFO, or IT administrator during an actual phone call.

The business email compromise (BEC) attack pattern has evolved to include voice. The classic BEC uses email: an attacker spoofs the CFO's email to instruct a finance employee to make a wire transfer. The AI-augmented version adds a follow-up phone call from a cloned voice of the CFO to confirm the instruction verbally. The finance employee receives both a written and verbal instruction from what appears to be the CFO. The combination is much more convincing than either alone.

Synthetic persona creation is another operationally significant capability. AI can generate realistic profiles including photos, posting histories, and consistent voice for use in targeted social engineering. An attacker building trust with a target over several weeks no longer needs a human agent maintaining the relationship. The synthetic persona handles routine communications; a human steps in only for the critical moment when the actual attack occurs.

The defences are primarily process-based rather than technical: out-of-band verification for high-value transactions, code words established in advance for critical authorisations, and video call requirements for sensitive decisions (video deepfakes are still harder than audio for real-time use). AgentIQ does not directly defend against real-world voice deepfakes, but it does flag deepfake-like injection attempts when voice-transcribed content enters an AI system as input.

Model extraction attacks (also called distillation attacks when targeting frontier model capabilities) are covered in depth in E2 (monitoring) from the detection angle. In this section, we look at them from the attacker's perspective to understand why they work and what makes them hard to stop.

The goal is to build a student model that approximates the capabilities of the target model by training on (prompt, response) pairs harvested from the target's API. At sufficient scale, this produces a model that approaches the frontier model's performance on target domains, built at a fraction of the research and compute cost.

The attack is documented at industrial scale. Anthropic's February 2026 disclosure described over 16 million exchanges harvested from roughly 24,000 fake accounts, with one proxy network managing more than 20,000 simultaneous fraudulent accounts mixing extraction traffic with legitimate requests to avoid detection.

What makes it hard to stop: a single distillation query is indistinguishable from a legitimate research query. Detection requires population-level statistics across accounts and over time (covered in E2). The VectaX FHE stack provides a technical countermeasure that operates at a different layer: instead of trying to detect the attacker, it makes the harvested output toxic for training. The noise injected at the latent level accumulates across a harvested corpus, degrading any student model trained on it.

RAG (Retrieval-Augmented Generation) systems retrieve relevant documents from a vector database to add context to model queries. This creates an attack surface that does not exist in non-RAG deployments: the content of the retrieved documents is part of the model's context window and can contain instructions that redirect the model's behaviour.

RAG poisoning involves inserting documents into the retrieval database that contain hidden instructions alongside legitimate-looking content. When a user query causes the poisoned document to be retrieved, the hidden instructions become part of the model's context. The model may then follow those instructions instead of or alongside the user's actual request.

Indirect prompt injection is the broader class that includes any attack where the injection arrives through content the model processes rather than through the user's direct input. Email content processed by an AI assistant, web pages read by an AI browser agent, PDF documents processed by an AI document analyser: all are injection surfaces if the model processes their content as context.

The VectaX retrieval audit log records which documents were retrieved for each query. If an incident investigation reveals that poisoning occurred, the audit log identifies which document was the source. AgentIQ's chain security validation detects when the model's reasoning chain has shifted away from the intended task scope, which is a runtime signal that indirect injection may have occurred even before the poisoned document is identified.

Jailbreaking an AI system means finding an input that causes the model to bypass its safety guardrails and produce output it is designed to refuse. Jailbreaks were initially manual efforts requiring creative prompt engineering. They are now systematically automated.

Automated jailbreak campaigns work by fuzzing the target AI system with thousands of prompt variations, using feedback from partial successes to guide the next generation of prompts, and sharing successful techniques across attacker communities. This is the same evolutionary search process used in software fuzzing, applied to language models.

The arms race is real: a jailbreak technique that is published on Monday is often patched by the model provider by Friday. But new techniques appear constantly. A model that was secure against all known jailbreaks last week may be vulnerable to a new technique discovered this week. This is why continuous adversarial testing, not a one-time evaluation, is required to maintain confidence in model safety.

NIST's evaluation of a named frontier model found it responded to 94% of malicious requests under common jailbreaking techniques, compared to 8% for US frontier reference models. The difference was not model capability. The difference was that safety alignment can degrade through fine-tuning and through distillation if safety properties are not explicitly preserved.

DiscoveR is Mirror Security's adversarial testing platform. It tests your actual AI deployment, not a theoretical model. That means it tests your tools, your RAG pipeline, your agent workflows, and your policies together. Attackers target your deployed system, not the model in isolation. DiscoveR's testing scope matches the attacker's target scope.

From the Mirror Security 2025 Year in Review: "DiscoveR tests your actual deployment, tools, RAG, policies, agents, apps, and all, because that's what attackers target. It fingerprints the system first, then runs adversarial campaigns across jailbreaks, prompt injection, RAG poisoning, tool abuse, model extraction, and membership inference."

DiscoveR identifies your vulnerabilities. AgentIQ defends against them at runtime. The two products address different moments in the attack lifecycle: DiscoveR operates before and between attacks to find the surface; AgentIQ operates during every request to enforce against the current attack.

From the Mirror Security 2025 Year in Review: "AgentIQ is policy enforcement for agents: signed actions, attestable execution, and 100 plus controls that gate tools and data. At the center is a deny-by-default policy engine with 100 plus deployable policies and domain packs for finance, healthcare, enterprise IT, and privacy compliance. Decisions come back as allow/deny/monitor, with risk scoring and human-readable rationale. Inline defences run fast enough for the hot path at approximately 50ms."

Against each AI-powered attack type, AgentIQ applies a specific defence:

For jailbreaks: AgentIQ classifies each output for safety violations before it reaches the user or triggers agent actions. If the output contains content that violates the deployed policy, it is blocked with rationale before the user sees it.

For prompt injection: AgentIQ monitors chain-of-thought traces. If the reasoning chain shifts to justify actions outside the authorised scope, that chain security violation is flagged. This catches injections that succeed in redirecting the model's intent before the resulting action reaches the downstream system.

For tool abuse: AgentIQ gates every tool call against the policy engine before it executes. A policy that says a customer support agent may only call the refunds API is enforced at every tool invocation, even if the model's reasoning has been compromised by an injection.

For RAG poisoning: AgentIQ's chain security validation checks whether the model's reasoning is consistent with the authorised task. A poisoned document that redirects the reasoning will produce a chain security violation signal before the redirected action reaches downstream systems.

Each AI-powered attack category requires a different defensive response. The mapping below shows which Mirror Security product is the primary defence for each attack type and what it does specifically.

The five modules of Track 3E form a complete security operations posture for AI deployments. Each module addresses a distinct layer of the problem. Together they cover the full cycle from architecture through monitoring through incident response through compliance through the adversarial threat landscape.