We are entering a new architectural phase and navigating a meaningful shift. AI systems are moving beyond static responses and into systems that can take actions like triggering workflows, calling APIs and making decisions within production environments. This is transformative.
At the same time, this shift is happening faster than most teams can fully operationalize or standardize. Across industry conversations, early implementations and emerging case studies, I believe a pattern is starting to become clear:
Most AI failures are not model failures – they are control failures.
Not necessarily because systems are poorly designed, but because:
- boundaries are still evolving
- failure modes are not fully understood
- recovery paths are often under-defined
As we move toward more autonomous systems, we are effectively taking cautious steps into production – without always knowing how and when things might surface as unintended outcomes.
When they do, the impact is rarely isolated:
- it can affect multiple customers
- it can impact trust and brand perception
- it can translate into real cost
When these systems scale, we don’t just scale capability. We scale uncertainty and potentially, mistakes.
I believe we are collectively building the playbook as we go and this is my attempt to make sense of what that might look like.
Guardrails: More Than Just a Safety Feature
Guardrails are no longer a theoretical concept or something that can be deferred for later. Increasingly, they are becoming a real and necessary part of building agentic AI systems.
What I still observe, though, is that in many implementations, guardrails are treated as an add-on introduced after the core system is already designed or applied post-facto to fill gaps.
Even when guardrails are considered early, they can sometimes become a checkbox exercise that makes us feel the system is “covered”, while important aspects may still be missing.
Part of the challenge is that we are still learning what “complete” actually looks like. As AI systems continue to evolve, new behaviors, edge cases and failure modes emerge – often faster than teams can fully anticipate.
This is where I have found it useful to shift how I think about guardrails. Instead of treating them as isolated checks, it helps to think of guardrails as the control plane of agentic AI.
Just as modern systems separate execution (data plane) from governance and coordination (control plane), agentic AI needs a layer that defines:
- what the system can do
- what it should do
- how it behaves under uncertainty or failure
Without this Control Plane, we’re not really building systems – we’re simply reacting to them.
Three Questions Every Architect Should Ask
To make the idea of a control plane more practical, I have found it useful to step back and ask a few simple questions – often before writing a single prompt.
1. Can it do this? (Capability & Access)
- Does the agent have the right permissions?
- Are tool calls constrained?
- Are access boundaries clearly defined?
Example – Billing Agent
An agent generating invoices should not have unrestricted access to pricing configuration.
2. Should it do this? (Policy & Context)
- Is the action aligned with business rules?
- Does it respect compliance and intent?
- Is context being interpreted correctly?
Example – Support AI
Issuing refunds requires understanding policy thresholds and not just user sentiment.
3. What if it goes wrong? (Resiliency & Recovery)
- Can actions be rolled back?
- Is there an audit trail?
- Is there a clear escalation path?
Example – Workflow Agent
Deleting or modifying customer data should always be recoverable.
If these questions are unclear, the agentic system will eventually surface that ambiguity – usually in production.
A Practical Framework for Control
Building on the idea of Guardrails as a Control Plane, it helps to think of them not as a single gate, but as a distributed system of controls.
One way to reason about this is across a few key areas:
| Category | Focus | Example |
|---|---|---|
| Technical | Validation & thresholds | Prevent hallucinated financial metrics |
| Security | Access & abuse prevention | Mask PII based on user roles |
| Ethical | Bias & responsible behavior | Ensure fair hiring recommendations |
| Operational | Runtime control | Rate limits and kill switches |
| Infrastructure | Platform safety | Sandboxing, isolation and cost boundaries |
| Business | Alignment & compliance | Enforce pricing rules and customer tiers |
Note: These are not independent layers – they interact continuously across the system lifecycle.
Where Guardrails Actually Live
To make this more concrete, it helps to think about where guardrails show up within a system.
They exist across the lifecycle and surface at different points as the system processes inputs, makes decisions and produces outcomes.
In practice, this often looks like:
- Input validation & policy enforcement
- Orchestration decisions & tool execution controls
- Model grounding & memory handling
- Output validation, monitoring and feedback
Each of these points represents a place where control can be applied or missed.
Hard-Earned Realities of Scaling
For engineers and architects building these systems, the gap between theory and production is where most learning happens.
Here are a few patterns I have seen emerge across implementations and industry discussions:
1. The Trap of Human-in-the-Loop (HITL)
HITL is often used as a safety net. In many cases today, it’s a necessary part of deploying AI systems responsibly. At the same time, as systems begin to scale, it’s worth being mindful of how it’s used.
In practice:
- humans can become bottlenecks
- alert fatigue can set in
- approvals can turn into routine “rubber-stamping”
The shift is not to remove HITL, but to use it more intentionally.
Design systems to be safe by default and rely on human intervention primarily for:
- high-risk actions
- policy exceptions
- low-confidence scenarios
If every decision requires human approval, I believe the system isn’t truly autonomous instead it’s closer to a complex UI with an approval layer.
2. The Latency Tax
Safety introduces latency where every validation adds a cost in time.
Rather than forcing everything into synchronous checks, it helps to distribute controls across the lifecycle:
- Pre-execution: Prevent obvious failures
- In-line: Enforce business logic
- Asynchronous: Audit and reconcile
3. Policy-as-Code vs. Prompt Engineering
Prompts are flexible, but brittle. Policies are enforceable. Decoupling rules from the model (using tools like Open Policy Agent (OPA) or similar approaches) allows for version control, auditability and model independence. For example, instead of encoding refund limits inside prompts, define them as policies that can be updated independently as business rules evolve.
In many ways, this becomes a key part of the control plane:
- Prompts guide behaviour
- Policies enforce behaviour
4. Guardrails Break Silently
A guardrail that works with one model may behave differently with another. Different models interpret constraints differently and edge cases surface in unexpected ways. For example, switching models can silently weaken compliance checks by a Contract Review Agent.
The Takeaway: Maintain a guardrail testing suite. Test adversarial cases, edge scenarios and validate across model versions. If guardrails aren’t tested, they’re just assumptions.
Two Often Overlooked Risks
As systems mature, a couple of areas tend to surface as more “silent” failure modes. They don’t always show up immediately but can have significant impact over time.
1. Economic Guardrails
Agents can loop recursively or call expensive APIs repeatedly, leading to what can effectively become a “Financial Denial of Service”.
In practice, this makes it important to introduce controls such as:
- session-level budgets
- token or usage limits
- execution caps
Cost, in this context, becomes a control boundary – not just a metric.
2. Memory & State Management
Agents don’t just act, they remember. Over time, this introduces challenges around PII retention, long-term context storage and unintended persistence of sensitive data.
Mitigation often involves:
- retention policies
- PII filtering
- memory scrubbing workflows
Memory becomes a liability if not managed intentionally.
The Strategic Bottom Line
To build production-grade agentic AI systems, it becomes important to think in terms of controlling:
- What the system does (actions)
- What it spends (economics)
- What it remembers (state)
Guardrails are not just about safety – they are about sustainability and trust.
Here’s a consolidated view of how these guardrails come together:
* This is still evolving but having a structured way to think about it helps in designing systems that scale.
Final Thought
Autonomy is the promise of agentic AI. But autonomy without control isn’t innovation – it’s risk.
As architects, our goal isn’t just to make AI systems work but to make them predictable, controllable and trustworthy over time.
The model is the engine.
Guardrails are the steering, the brakes and the dashboard.
.