The Guardrails Gap

There is a specific kind of organizational confidence that forms when a team solves the wrong problem very well. In 2023 and 2024, enterprise AI teams got exceptionally good at managing LLM output. They built prompt libraries, toxicity filters, hallucination benchmarks, and red-teaming protocols. They hired responsible AI leads and drafted usage policies. They ran their models through evaluation suites and reported on accuracy rates to their boards. By most internal metrics, they were winning.

They were also solving for the previous war. The risk model underpinning all of that effort assumed one thing: the AI talks, humans act. The worst-case scenario was an embarrassing response — a hallucinated legal citation, an off-brand customer service reply, a factually wrong product recommendation. Bad, certainly. Recoverable, almost always.

That assumption no longer holds. In 2026, the AI doesn't just talk. It acts. It books the meeting, fires the API call, updates the CRM record, executes the SQL query, sends the email, triggers the payment. And the governance posture that most enterprises built — carefully calibrated for output risk — offers almost no protection against action risk. The guardrails gap is not a future problem. It is an active operational exposure, compounding in real time at every organization scaling agentic workloads.

From Output Risk to Action Risk: Why the Distinction Matters

To understand the guardrails gap, you need to be precise about what changed. An LLM deployed as a chatbot sits inside a narrow interaction loop: user sends prompt, model generates text, human reads response, human decides what to do next. Every consequential action still passes through a human checkpoint. The blast radius of a failure is bounded by human attention.

An agentic AI system operates outside that loop. It is given a goal — "reconcile last quarter's vendor invoices," "monitor customer support tickets and escalate anomalies," "draft and schedule all follow-up emails from yesterday's sales calls" — and then executes a sequence of actions autonomously, often across multiple tools, APIs, and data systems, with minimal or no human checkpoints along the way. The human is upstream, at the instruction level. They are not present at the action level.

This shift fundamentally changes the failure mode. Output risk is recoverable because a bad output requires a human to act on it before it causes harm. Action risk is frequently not recoverable because the action is the harm. A misconfigured agentic workflow that bulk-deletes records, sends unauthorized communications, or submits incorrect transactions to an external system does not produce a bad answer that someone can correct — it produces a bad outcome that someone has to undo, explain, and potentially litigate.

Researchers studying LLM agent vulnerabilities have formalized this distinction in the attack surface literature. Direct prompt injection — where a malicious user embeds instructions in their input — is the agentic equivalent of a bad chatbot response. Indirect prompt injection is categorically different: an attacker manipulates external content (a document the agent retrieves, data it reads from a connected system) to hijack the agent's action chain without ever interacting with the system directly.^[4] The agent, operating autonomously, processes the poisoned content and executes instructions it believes came from its operator. No human was watching. No output filter triggered. The damage was downstream.

Multi-agent architectures compound this further. When agentic systems operate in pipelines — one agent orchestrating others, passing instructions and context between them — inter-agent trust becomes a live attack surface. An agent that trusts instructions from another agent in its pipeline has no reliable way to verify whether those instructions have been tampered with, injected, or adversarially modified in transit.^[4] The security perimeter that worked for a single model responding to a single user does not scale to a mesh of autonomous actors operating across enterprise infrastructure.

The Governance Posture Most Organizations Actually Have

Here is what most enterprise AI governance looks like in practice in 2026: a responsible AI policy document, a model evaluation framework focused on accuracy and bias metrics, a prompt engineering team, content moderation filters on customer-facing outputs, and perhaps a nascent AI ethics committee that meets quarterly. Some organizations have added red-teaming for jailbreak resistance. A few have implemented RAI maturity assessments.

What almost none of them have: a systematic inventory of what actions their agentic systems are authorized to take, a runtime enforcement layer that constrains those actions to defined parameters, an audit log that captures the full decision chain of each agent execution, an incident response playbook specifically designed for agentic failures, or a testing protocol that evaluates agent behavior under adversarial conditions in multi-tool, multi-step environments.

McKinsey's 2026 AI Trust survey, drawn from approximately 500 organizations with direct responsibility for AI governance, found this gap significant enough that they added an entirely new dimension to their AI Trust Maturity Model this year: agentic AI governance and controls. The fact that the leading AI governance framework had to add a new category in 2026 is itself diagnostic. The field has not been measuring the right things.^[1]

The Arkose Labs 2026 Agentic AI Security Report, surveying 300 enterprise security, fraud, identity, and AI leaders globally, found that 97% of respondents expect a material AI-agent-driven security or fraud incident within the next 12 months. Nearly half expect one within six months.^[2] Read that again: this is not a survey of skeptics or laggards. These are the people inside organizations actively deploying agentic AI — and they are nearly unanimous in their expectation that something will go wrong, soon, in a way that matters.

The question is not whether enterprises are aware of the risk. They are. The question is why awareness has not translated into governance redesign. The answer, in our view, is structural: the teams responsible for AI governance inherited frameworks designed for a different threat model, and retrofitting those frameworks is harder than acknowledging the problem.

Three Ways Agentic Systems Fail That Output Guardrails Don't Catch

1. Tool Misuse and Scope Creep

Agentic systems are given access to tools — APIs, databases, communication platforms, code execution environments — to complete their tasks. The problem is that tool access is typically defined at provisioning time, based on a best-guess of what the agent will need, and then rarely reviewed as the agent's scope evolves. Agents accumulate tool permissions the way software systems accumulate technical debt: gradually, quietly, until something breaks.

A procurement agent that starts with read access to vendor records and authority to draft purchase orders can, through a combination of expanded task scope and insufficiently scoped permissions, end up with the ability to submit orders, trigger payments, and modify vendor records. Nobody explicitly authorized that capability chain. It emerged through the interaction of individually reasonable decisions made at different times by different people. This is not a hypothetical — it is the standard pattern of how enterprise software systems accumulate unauthorized capability, now accelerated by the autonomy of agentic AI.

2. Autonomous Decision Chains Without Human Checkpoints

The productivity promise of agentic AI is that it completes multi-step tasks without requiring human intervention at each step. The risk is the mirror image of that promise: when an agent makes an incorrect decision early in a chain, every subsequent action amplifies the error. There is no human checkpoint to catch the mistake before it compounds.

Consider a financial reconciliation agent that misclassifies a transaction category in step two of a twelve-step workflow. Every downstream calculation, every report generated, every external filing triggered from that point forward is built on a faulty foundation. By the time the error surfaces — if it surfaces — the remediation cost is not the cost of fixing step two. It is the cost of unwinding everything that followed.

Backdoor attacks in agentic systems exploit exactly this dynamic. A backdoor trigger — embedded in training data or activated by a specific input pattern — causes the agent to behave normally under standard conditions but execute unintended actions when the trigger fires.^[6] Because the agent behaves correctly in testing and in most production conditions, the backdoor can operate undetected for extended periods. The damage accumulates silently across a long decision chain before any human notices the pattern.

3. Identity and Credential Exploitation

Agentic systems authenticate to external services using real credentials — API keys, OAuth tokens, service account credentials. They operate inside enterprise infrastructure with legitimate access, indistinguishable from authorized human users at the system level. This creates an attack vector that traditional security tooling is poorly equipped to detect: an agent acting outside its intended scope looks, from the perspective of the systems it's accessing, exactly like an agent acting within its intended scope.

The Arkose Labs research highlights this specifically: AI agents are already retrieving data, triggering transactions, and interacting across services through legitimate credentials and approved workflows.^[2] When an attacker hijacks an agent's action chain — through indirect prompt injection, inter-agent trust exploitation, or credential compromise — they are operating inside the enterprise with legitimate access. The perimeter is not breached. The agent is the breach.

Why Companies Scaling Hardest Are Most Exposed

There is a painful irony at the center of the guardrails gap: the organizations most committed to capturing the value of agentic AI are, by construction, the most exposed to its risks. Scaling agentic workloads means provisioning more agents, expanding tool access, increasing autonomy, reducing human oversight, and deploying into higher-stakes workflows. Every one of those moves widens the gap between what the agents can do and what the governance framework can safely constrain.

The Observer's analysis of early enterprise agentic deployments in 2026 puts a sharp number on this: an estimated 40% of agentic AI projects will be canceled by end of 2027. The primary driver is not technology failure. It is preparation failure — organizations that began deployment before their data, governance, and operating model infrastructure was ready to support autonomous systems at scale.^[3] The failure mode is not "the AI didn't work." It is "the AI worked, but we didn't have the controls to operate it safely, so we had to shut it down."

This is the specific failure pattern the guardrails gap produces. The technology performs. The governance doesn't. And the cost of that mismatch is not just operational incidents — it is the strategic cost of pulling back, rebuilding confidence, and re-deploying on a slower timeline with more overhead, while competitors who invested in governance infrastructure earlier continue to scale.

Risk Dimension	Output-Era Guardrail	Why It Fails in Agentic Systems	Required Control
Content Quality	Toxicity filters, hallucination evals, output monitoring	Agentic failures produce bad actions, not bad text. Filters don't evaluate SQL queries or API payloads.	Action-level validation; tool output auditing
Scope Control	System prompt constraints, topic restrictions	Agents operate across multi-step chains; prompt-level scope doesn't constrain downstream tool calls	Runtime permission enforcement; least-privilege tool access
Adversarial Input	Direct prompt injection detection	Indirect injection operates via external content the agent processes — invisible to input-layer defenses	Content provenance validation; input sanitization at tool-call boundaries
Identity & Auth	User authentication, session management	Agents hold persistent credentials; compromise is invisible — attacker operates as the agent	Short-lived credentials; just-in-time access; continuous behavioral monitoring
Audit & Recovery	Conversation logs, output archives	Multi-step agentic workflows span many systems; no single log captures the full decision chain	Distributed trace logging; action replay capability; rollback protocols
Model Integrity	Red-teaming for jailbreaks	Backdoor attacks behave normally under standard conditions; red-teaming misses trigger-dependent failures	Behavioral regression testing; production monitoring for distributional shifts

The Architecture of a Governance Posture Built for Action Risk

Most organizations treat AI governance as a compliance exercise: document the policies, run the evaluations, check the boxes. That approach was inadequate for output risk. It is useless for action risk. Governance for agentic systems needs to be architectural — embedded in the design of the systems themselves, not layered on top after deployment.

The highway guardrail analogy from the infrastructure security community is instructive here.^[7] Highway guardrails don't slow traffic down. They increase safe operating speeds by eliminating the risk of catastrophic failures at the edges. Organizations without strong access controls and policy frameworks for their agentic systems don't operate more freely — they operate more cautiously, with more manual review, more limited agent capability, and more overhead that negates the automation benefits they're chasing. Good guardrails are not friction. They are the precondition for moving fast without causing harm.

What does that architecture look like in practice? It has five components that output-era governance frameworks typically lack entirely.

Runtime Action Authorization

Every tool call an agent makes should be evaluated against a policy layer at the moment of execution — not at the time the agent was provisioned. Static permissions granted at deployment time drift out of alignment with actual risk as agents evolve and expand. Runtime authorization means the question "is this agent allowed to do this, right now, given its current context?" is answered dynamically, at the action boundary, every time.

Least-Privilege Tool Access with Time-Bounded Credentials

Agents should hold the minimum permissions necessary to complete their current task — and those permissions should expire. Short-lived, just-in-time credentials eliminate the standing attack surface that persistent agent credentials create. An agent that authenticates to a database for a specific read operation and whose credential expires sixty seconds later is categorically less exposed than an agent holding a persistent service account with broad access.^[7]

Full Decision Chain Auditability

Enterprise audit infrastructure was built for human actions, which happen at human speed across a limited number of systems. Agentic systems operate at machine speed across many systems simultaneously. The audit infrastructure has to match the architecture. Every agent action — every tool call, every API request, every data read and write — needs to be captured in a distributed trace that can reconstruct the complete decision chain after the fact. This is not a nice-to-have for compliance. It is the prerequisite for incident response.^[8]

Behavioral Regression Testing in Production

Agent behavior in production will drift from agent behavior in testing. Models get updated. Configurations change. Data distributions shift. Regression testing frameworks need to detect when previously well-performing behaviors degrade — and production monitoring needs to alert human operators when the distribution of agent actions shifts in ways that warrant investigation.^[8] A single red-team exercise at deployment time is not sufficient. Agentic systems require continuous behavioral evaluation.

Human Escalation Thresholds with Defined Incident Playbooks

Not every action an agent takes warrants human review. But every category of high-stakes action — irreversible operations, external communications, financial transactions above a threshold, modifications to production data — should have a defined escalation path that routes to a human before execution. These thresholds need to be explicit, parameterized, and enforced at the system level, not left to the agent's judgment or the operator's hope.

A Self-Assessment for Technical Leaders

If you answered "no" or "I'm not sure" to more than two of those questions, your governance posture is not calibrated for the systems you're running. That is not a criticism — it is the median condition of enterprises in 2026. The governance frameworks available when most of these teams started their AI programs simply did not address these questions. The issue is recognizing that the old framework is no longer sufficient and acting on that recognition before the incident forces your hand.

What to Do About It: An Action Agenda

We are not in the business of recommending that organizations slow down their agentic AI programs. The competitive pressure is real, the productivity gains are real, and the organizations that get agentic AI right will have meaningful advantages over those that don't. The recommendation is not to move slower. It is to move with the right architecture underneath you.

In the next 30 days: Conduct a permissions audit of every deployed agentic system. Document what tools each agent can call, what data it can read and write, and what external systems it can communicate with. Flag every permission that exceeds the minimum necessary for the agent's current task. This is unglamorous work, but it is the foundation of everything else. You cannot govern what you haven't inventoried.

In the next 60 days: Instrument your agentic workflows for full decision chain logging. If you cannot replay the complete action sequence of any given agent execution from your existing log infrastructure, you cannot do incident response — you can only do incident discovery, which is much more expensive. Prioritize logging for agents operating in high-stakes domains first: financial operations, customer communications, data modifications.

In the next 90 days: Draft an agentic AI incident response playbook. It should answer at minimum: How do we detect an unintended agent action? Who is notified and in what timeframe? What is the process for halting an agent mid-execution? How do we assess the blast radius of a confirmed incident? What is the rollback or remediation protocol for different categories of agent action? This playbook should be owned by a named individual, not a committee.

This quarter: Run your first indirect prompt injection red team exercise against a production agentic system. Have your security team or a third party attempt to manipulate external content the agent processes — documents it reads, data it retrieves, messages it receives from other agents — and evaluate whether those manipulations can redirect the agent's action chain. The results will be instructive. In most cases, they will also be humbling.

This half: Redesign your AI governance framework around action risk rather than output risk. This means updating your maturity model to include the dimensions McKinsey's framework now measures — agentic AI governance and controls — and evaluating your organization honestly against them.^[1] It means moving AI security conversations from the AI team to the security and infrastructure teams who own the systems the agents are acting on. And it means treating agentic AI governance as an architectural discipline, not a policy document exercise.

The enterprises that will capture the full value of agentic AI over the next three years are not the ones moving fastest today. They are the ones building the governance infrastructure now that will let them move fastest safely — the organizations that understand the guardrails gap, close it deliberately, and scale from a foundation that can hold the weight of genuine operational autonomy. The gap is real, it is widening, and the window to close it before an incident forces the conversation is narrowing. The time to act is before the playbook gets tested under fire.