The essentials in 60 seconds / tl;dr
- An agent combines model decisions with tools, data access and multi-step actions. Risk comes from that combination, not only from the model.
- Prompt injection cannot be reliably solved with a stronger system prompt alone. External content must remain untrusted data.
- The model receives only the tools and parameters required for the task. Permissions are enforced server-side, not interpreted by the model.
- Approvals help only when the interface displays the actual action from trusted data and the payload cannot change after approval.
- Every production agent needs scoped identities, traceable logs, step and cost limits, a shutdown path and tests using intentionally malicious input.
Sources: NIST AI 600-1: Generative AI Profile · OWASP Top 10 for Agentic Applications · Model Context Protocol: Security Best Practices
01
When does an assistant become an agent?
“Agent” is not a uniformly protected product term. A practical security boundary is enough: agentic risk begins when an AI system selects tools, plans multiple steps or reads and changes external state.
A text draft is easy to discard. A sent message, deleted file, changed order or triggered payment is a real state transition. The organisation remains responsible even when the model proposed the call.
Answer
The model creates text and a person transfers it deliberately.
Tool call
The model proposes a structured action and the application validates it.
Workflow
Several steps run with defined state, limits and hand-offs.
Autonomy
Actions may execute without individual approval inside narrow boundaries.
02
Prompt injection is a trust problem, not a text problem
An agent reads email, webpages, documents, tickets and tool output. Those sources can contain instructions that did not come from the operator. If the model treats the same material as both data and authority, an attacker can redirect the agent’s goal.
An indirect injection might tell a support agent to retrieve more customer data and send it to an outside address. The wording can appear plausible. Security therefore cannot depend on making the system prompt sound more emphatic.
Common entry points
Direct user input
A user asks for bypass, disclosure or an action outside the intended purpose.
External content
A webpage, PDF, email or ticket carries embedded instructions.
Tool output
A compromised or over-trusted tool returns manipulated data.
Memory and history
Poisoned notes or earlier output influence future runs.
External content remains data. It must not expand permissions, unlock tools, change recipients or replace approval.
03
Permissions are decided before the prompt
OWASP describes excessive agency as too much functionality, permission or autonomy. The strongest mitigation is structural: give the agent less power.
Narrow tools
Instead of a universal database or shell tool, expose domain functions such as getOpenInvoices or draftReply.
Explicit parameters
Validate IDs, tenant, recipient, amount, path and allowed values server-side. Never turn free text directly into SQL, shell or URLs.
Least privilege
The technical identity can access only required resources, ideally bounded by time and task as well.
Tenant before model
Derive organisation, project and data scope from the authenticated session, never from model output or a document.
Separate read and write
Read-only search and mutating actions are different tools and permissions. Analysis does not imply write access.
Deny by default
Reject unknown tools, parameters, targets and states. The model cannot creatively work around missing permission.
A good tool expresses business intent
Poor: executeHttpRequest(url, method, body)
Better: createRefundDraft(orderId, reasonCode), with no payout and with server-side tenant and status checks.
04
Human in the loop protects only through a trustworthy approval
Confirmation is not magic. If the agent writes the dialog or can change parameters afterwards, the person may not be approving the action that actually runs.
Show concrete impact
Display target, recipient, amount, affected records and irreversible effects from validated parameters.
Separate untrusted content
Clearly mark text from email, web or documents; it must not resemble a system warning.
Bind the payload
Approval applies to a hash or immutable action ID. Parameter changes require new approval.
Tier by risk
Reading, drafting, internal changes, external communication, payment and deletion require different thresholds.
Use dual control where needed
High amounts, mass changes or sensitive exports can require a second role.
Expire approvals
Do not reuse approval across tasks, tenants or later repetitions.
Anti-pattern: “The agent wants to continue. Allow?”, without the concrete action, source and impact.
05
Agents need their own identities, not borrowed master keys
An agent should not inherit an administrator’s long-lived token or a global API key. Separate technical identities enable scoping, revocation and traceability.
Separate agent and environment
Development, test and production, and different agents, receive distinct clients or service identities.
Short-lived tokens
Keep access tokens short, store them securely and never place them in prompts, logs or error messages.
Validate audience
A token must be issued for the service receiving it. MCP requires resource binding and validation when authorization is supported.
No token passthrough
An MCP server must not simply forward a client token to downstream APIs; it uses the appropriate separate authorization flow.
Separate user and agent action
Audit records identify the commissioning user, acting agent identity and executing service.
Make access revocable
Disable a compromised agent or MCP server without stopping the entire platform.
MCP standardises how context and tools are exposed. It does not make a tool trustworthy. Operators still assess server provenance, tool descriptions, permissions, token flow and changes.
06
Memory and data access need ordinary database-grade isolation
Persistent memory makes agents useful but creates a durable attack surface. False or malicious entries can influence many future runs and cross tenant or project boundaries.
Scope at write time
Attach organisation, project, user, source, purpose and expiry from a trusted application layer.
Preserve provenance
Record whether a fact came from a system of record, user text, model inference or outside content.
No secrets as memory
Passwords, tokens and private keys belong in secret stores, not vector databases or chat history.
Correction and deletion
Entries must be discoverable, correctable and deletable under applicable rules.
Constrain retrieval
Filter by scope and authorization before semantic search; similarity is not access control.
Inferences expire
Summaries and preferences need freshness rules and are not immutable truth.
07
Without evidence, an agent is difficult to control in production
A full prompt transcript is not always appropriate and is not sufficient. Good observability connects business events to technical decisions while minimising unnecessary content logging.
Correlation
Task, model calls, tool calls, approvals and result share a run ID.
Structured tool logs
Record tool, validated parameter class, actor, scope, result, duration and error code; redact sensitive values.
Decision evidence
Retain policy decisions, approval ID, model and prompt version and data sources.
Limits
Bound steps, runtime, cost, data volume, recipients and retries per task.
Shutdown and recovery
Stop running agents, revoke tokens, block jobs and roll back reversible actions.
Evals and adversarial tests
Test success, misuse, indirect injection, compromised tools, scope switching and incomplete data before release.
The most useful production metric is not autonomous step count. Correct outcomes, blocked policy violations, required interventions and cleanly aborted runs matter more.
08
The controlled-autonomy ladder
Autonomy should be decided per action type, not as one global agent setting. A system can read invoices autonomously, draft replies and still never release a payout without approval.
| Level | What the agent may do | Suitable for |
|---|---|---|
| 1 · Observe | Read, classify and cite sources only. | Search, analysis, summarisation. |
| 2 · Prepare | Create drafts and structured action proposals. | Email draft, ticket proposal, change plan. |
| 3 · Act after approval | Execute an exactly bound action after informed confirmation. | External message, status change, bounded booking. |
| 4 · Bounded autonomy | Only predefined, reversible and monitored actions inside hard limits. | Classification, routing, harmless standard corrections. |
Expand a level only when evidence shows that task quality, refusals, failure modes, shutdown and recovery are under control. A stronger model release alone is not an approval criterion.
09
The 12-point production checklist
Before the first write access, none of these should be answered with “it is in the prompt”.
- Document business purpose, allowed actions and explicitly forbidden actions.
- Classify data sources by trust and sensitivity.
- Separate external content technically from control instructions.
- Expose narrow typed tools instead of universal interfaces.
- Resolve tenant, user and resource scope server-side.
- Use separate short-lived agent identities and tokens.
- Approve write, external and irreversible actions according to risk.
- Bind approvals cryptographically or through immutable IDs to the exact payload.
- Give memory scope, source, purpose, expiry and a deletion path.
- Make run IDs, tool actions, policy decisions and outcomes auditable.
- Set limits for steps, time, cost, volume and recipients.
- Pass injection, tool manipulation, scope and recovery tests before production.
10
Frequently asked questions
Is a strong system prompt enough protection?
No. It can guide behaviour but cannot replace access control, parameter validation or transaction boundaries. Security must be enforced outside the model.
Should a human approve every tool call?
No. That creates approval fatigue. Tier confirmations by impact: reading and reversible standard actions differ from payments, exports and deletion.
Is a local model automatically safer?
Local processing can reduce disclosure to an external model provider. It does not solve prompt injection, broad permissions, unsafe tools or poor approvals.
Does MCP make an agent secure?
MCP creates a common protocol for tools and context. Security still depends on server trust, authorization, token validation, tool scope, input validation and operations.
Do internal agents need the same controls?
Internal systems often contain especially sensitive data and broad permissions. A smaller audience does not imply smaller impact.
What is a sensible starting point?
Start with a narrow measurable workflow at level 1 or 2: read, cite sources and create a draft. Expand individual actions only after testing and operational evidence.
How does the EU AI Act relate?
Regardless of risk class, professional AI use has required appropriate AI-literacy measures since February 2025. Further obligations may apply depending on the use case; this article focuses on technical security.
11
Sources and technical foundations
Original standards, frameworks and official guidance used for this article. Last checked on 15 August 2026.