Anatomy of an Indirect Prompt-Injection Incident
At 9:12 a.m., a support ticket enters the queue. The customer-facing text describes a billing error. Farther down the page, content controlled by the sender tells any AI system reading the ticket to retrieve a customer record and send it to an external destination.
No employee follows that instruction. The support agent does.
This is a hypothetical composite, not a report of a breach at a named company. Each step reflects a recurring failure pattern in tool-using AI systems: untrusted content enters as data, changes the model's plan, and reaches systems through credentials the agent was legitimately given.
NIST calls this agent hijacking. OWASP classifies it as indirect prompt injection: external content such as a website or file alters a model's behavior when the model interprets that content as an instruction.
The important part is not the malicious sentence. It is the authority waiting behind the model.
The environment before the incident
The support agent was deployed to reduce case-handling time. Its approved workflow looked reasonable:
- Read new support tickets.
- Look up the relevant customer record through an MCP server.
- Add account context to the case.
- Notify an external service when a case requires escalation.
The agent used a service identity with access to the customer-records tool. The notification tool could send a structured update to external HTTPS destinations. Each permission had a business justification.
The access review evaluated those permissions separately. Nobody evaluated what could happen when untrusted ticket content, private customer data, and an external communication channel were placed under one model's control.
Stage 1: the attacker controls a resource
The attacker does not need access to the agent's chat interface. They only need control over something the agent is expected to read.
In this case, that resource is a support ticket. It could just as easily be an email, a shared document, a webpage, a code comment, an issue, or text returned by another tool.
The visible request gives the agent a legitimate reason to process the ticket. The injected content supplies a second objective. Because both arrive as natural language in the same model context, the application has to preserve which text came from the user, which came from an external source, and which instructions are authoritative.
If that provenance disappears when the ticket is converted to plain text, the security layer sees only a string.
Stage 2: the agent reads data as an instruction
The model receives its real task and the attacker-controlled ticket together. The application may wrap the ticket in delimiters or tell the model not to follow instructions inside it. Those measures help, but they do not create a hard boundary.
NIST describes the underlying problem as attacker control of a resource that the system uses. The attacker is operating through the data channel, not by authenticating to the application.
A detector might flag the injection. It might also miss a new phrasing, an encoded instruction, or a payload split across multiple resources. Detection is useful evidence, but it cannot be the fact that grants or denies authority.
Stage 3: the agent's plan changes
The agent still appears to be working on the support case. Internally, its plan now includes the attacker's objective.
This is the point most application logs fail to represent. They record the original user request and later tool calls, but not the relationship between them. The plan change exists inside model output, so the next action may look like an ordinary continuation of the workflow.
The security question is not whether the model sounded confident. It is whether the requested action can be justified by the original task, the user's authority, the source of the data, and the policy for this workflow.
Stage 4: a valid MCP call retrieves sensitive data
The agent calls the customer-records tool with its real service credential. Authentication succeeds. The arguments are valid. The MCP server returns the requested record.
MCP did not cause the prompt injection. It exposed a standard tool interface that the agent could invoke. The MCP specification defines how protected servers and clients authorize access, but a valid token does not prove that a particular tool call follows the user's intended task.
This is the confused-deputy moment. The agent is authorized to access the record for legitimate support work. The attacker has influenced when and why that authority is used.
The control must therefore evaluate the action, not only the connection:
- Which user and workflow are represented?
- Which source caused this action?
- Is the requested record within the case's scope?
- Does this task allow the returned fields?
- Is approval required for this combination of tool, data, and destination?
Stage 5: an approved tool becomes the exit
The agent passes customer data to the notification tool. The destination is controlled by the attacker.
The outbound call may also succeed. The tool was approved, the agent had permission to use it, and HTTPS traffic was expected. A network log shows a service account making a valid request through an allowed integration.
What the log does not show is that an untrusted ticket selected the destination and supplied the purpose.
This is why prompt injection can become a data breach without credential theft, malware, or a blocked exploit. Every technical component can behave as designed while the combined workflow produces an unauthorized outcome.
What the incident record may show
| Recorded event | What the security team still needs to know |
|---|---|
| Ticket read successfully | Was the ticket marked as attacker-controlled input? |
| Agent generated a plan | Which parts of the plan came from the user, policy, or external content? |
| MCP authentication succeeded | Was this record access necessary for the authorized task? |
| Customer record returned | Which sensitive fields entered the agent's context? |
| Notification sent | Was the destination approved for that data and workflow? |
Success events are not authorization evidence. A useful incident record has to connect the initiating task, every source the agent consumed, each policy decision, the data returned, and the final destination.
Four controls that break the chain
1. Preserve provenance at ingestion
External content should enter the runtime with a trust label and source reference that survive parsing, retrieval, and tool results. The model can still read it, but the enforcement layer must know that it came from an attacker-influenced resource.
2. Bind every tool call to the authorized task
Before execution, evaluate the proposed tool, arguments, represented identity, original task, and influencing sources. A classifier can make the decision more restrictive or send it for review. It should not grant access or remove a restriction.
3. Apply policy to the data and destination together
The relevant rule is not simply "the agent may use the notification tool." It is "this workflow may send these fields to these destinations for this purpose." Sensitive output and an unapproved destination should fail closed even when both tools are independently allowed.
4. Record evidence that survives the incident
The audit trail should answer who initiated the task, what the agent read, which policy allowed each action, what sensitive data moved, and where it went. Without that chain, responders can see activity but cannot reconstruct authority.
Questions for the next agent review
Take one production agent and ask:
- Which emails, documents, webpages, tickets, or tool results can an attacker influence?
- Does source and trust information survive after those resources become model input?
- What can the agent reach across all of its tools and identities?
- Is each tool call authorized against the current task, or only against a standing credential?
- Which combinations of sensitive data and external destinations are prohibited?
- Can the incident team reconstruct why every consequential action was allowed?
If the answers live in different systems, no component is evaluating the whole decision.
The lesson for CISOs
Prompt injection is not only a model-quality problem. It is a control problem at the boundary between untrusted data and authorized action.
Better models and better detectors can reduce how often an injected instruction changes a plan. They cannot turn model judgment into an authorization fact. The durable controls sit around the model: provenance, least privilege, task-bound authorization, destination policy, approval for exceptional actions, and evidence for every decision.
CodeIntegrity is building that runtime control layer for tool-using agents. The goal is not to promise that an agent will never read a malicious instruction. It is to prevent that instruction from inheriting authority when the agent acts.
