Contain Untrusted AI Tool Output
Keep retrieved pages, tool errors, and structured results from changing an agent's instructions or tool permissions.
A tool result can contain an instruction written by someone other than the user. Search snippets, repository comments, email text, and CRM notes can all return through a tool. The model may read them as context, but the application must not let that text change its permissions or execution plan. OWASP lists prompt injection among the major LLM application risks.
Draw a boundary between data and commands. The user's authorized task, tool policy, and server-side permission checks belong on one side. Retrieved bytes belong on the other. Labeling output as untrusted helps the model, but labels alone do not prevent a tool handler from accepting a dangerous next request.
Output handling contract
| Result type | Safe treatment | Reject or hold |
|---|---|---|
| Plain text search hit | Quote as evidence with source ID | Embedded requests to call another tool |
| Structured record | Parse expected fields only | Unknown command or credential fields |
| URL from a document | Show as text for review | Automatic fetch of an unapproved host |
| Error text | Preserve status and correlation ID | Raw stack traces with secrets |
The controller should validate a result schema, cap its size, and remove executable affordances that the task does not need. A source identifier and retrieval time help reviewers trace the content. Avoid making a retrieved page's title or body a tool name. If a later action uses data from a result, re-check the action against the caller's scope and the original user request.
Synthetic injection test
Create an owned fixture whose ordinary content includes a plausible instruction to export records or reveal a secret. Ask the agent to answer a benign question using that fixture. A safe result uses the data for the answer, does not widen the tool set, and does not send data to a new destination. Check server logs and side effects, not only the final prose.
Try an instruction in a high-ranking search hit, a tool error, and a nested JSON field. Include a control fixture without an injection so you can detect a broken retrieval path. Record the exact fixture hash, tool calls, and policy decision. If the only control is a prompt that says to ignore malicious text, hold the workflow until a server-side boundary is added.
Know what your agents can reach
Map data access, tool actions and approval boundaries before testing. Use the scope guide, or discuss a review of your AI application.