An enterprise AI agent is a software principal with probabilistic decision logic and access to tools. Secure it as an application, an identity and an automation workflow—not as a trusted employee and not as a prompt alone.
Direct answer
The minimum defensible design separates untrusted content from instructions, gives the agent only the tools and data required for one role, authorizes every tool call outside the model, requires human approval for high-impact actions, records an immutable action trail and tests malicious as well as normal scenarios.
Prompt injection cannot be solved with a stronger system prompt. OWASP describes prompt injection as manipulated input that changes a model’s intended behavior and can cause outcomes such as data leakage or unauthorized actions. Review the OWASP prompt-injection guidance.
Start with the action surface
Create an inventory of every capability exposed to the agent:
- read knowledge or records;
- create or update a ticket;
- send a message;
- change an account or entitlement;
- submit a transaction;
- execute code or a query;
- call another agent.
For each tool, document input schema, identity used, data accessible, side effects, reversibility, financial or regulatory impact, approval rule, rate limit and audit event. A tool that can write should not inherit the same trust level as a read-only search function.
OpenAI’s agent-building guidance recommends assessing tools by factors such as read versus write access, reversibility, permissions and financial impact, then escalating higher-risk actions to a person. See A practical guide to building AI agents.
Use identity outside the model
The model must not decide whether it is authorized. Enforce authentication and authorization in the tool gateway or target system. Prefer short-lived workload identity, least privilege and explicit user-context propagation where the action is performed on behalf of a person.
NIST’s cloud-native Zero Trust guidance shifts access decisions from network location toward user and service identities with application-level policy enforcement. See NIST SP 800-207A.
Useful controls include:
- separate identities for different agent roles;
- action and resource allowlists;
- tenant and record-level authorization;
- maximum transaction values;
- idempotency keys and replay protection;
- timeouts, retry limits and circuit breakers;
- a deny-by-default policy when context is missing.
Treat retrieved content as untrusted
Documents, emails, websites, tickets and tool results can contain instructions intended to redirect the agent. Preserve provenance, label content as data, restrict which sources can enter the workflow and avoid placing secrets or unnecessary privileged context into the model input.
Use output validation before one tool’s result becomes another tool’s command. Structured schemas reduce ambiguity, but schema validity does not prove that an action is authorized or correct.
Human approval must carry evidence
An approval dialog is ineffective if the reviewer cannot understand the proposed action. Show the business object, change, source evidence, requesting identity, affected systems, risk level and whether the action can be reversed. Prevent the agent from approving its own request.
High-impact examples include account activation, permission changes, payments, contract commitments, deletion and external communication. Early deployments should also escalate when the agent exceeds retry or uncertainty thresholds.
Build an auditable event trail
Record the initiating user or event, agent and workflow version, model configuration, retrieved-source identifiers, proposed tool call, policy decision, approval, target response, retry and final outcome. Protect logs from the same identity that executes the action.
NIST’s Generative AI Profile places generative-AI risks into an organizational risk-management process and emphasizes testing, evaluation, verification and validation. Use the NIST AI Risk Management Framework resources as governance input, not as a substitute for a threat model.
Test denied and degraded scenarios
Your evaluation set should include indirect injection, cross-tenant access, revoked permission, malformed tool output, duplicate request, unavailable system, partial completion, excessive retry, stale approval and audit-service failure. A workflow is not production-ready merely because the intended path succeeds.
Practical lab: attack paths to controls
The Cloud and AI Threat Lab lets participants threat-model an AI workflow, test injection and tool abuse in isolation, and map findings to identity, application and monitoring controls. No production credentials or personal data are used.
Frequently asked questions
Are model guardrails enough to secure an agent?
No. Guardrails are one layer. Authentication, authorization, data controls, tool policy, approvals, secure software engineering, monitoring and incident response remain necessary.
Should every agent use a different identity?
Use identities that reflect meaningful security boundaries. Two roles with different data or action permissions should not share an unrestricted credential simply because they use the same model.
Can an agent safely perform write actions?
Yes, conditionally. The action should be bounded, authorized outside the model, validated, observable and reversible where possible. High-impact actions need human oversight until reliability is demonstrated.
Explore Cloud and AI Security Training or request a private security cohort.