AgentGate

AgentGate is a policy-enforcement firewall that sits between an AI Agent and the tools it wants to execute.

Why I Built It

AI agents can read files, query databases, send messages, call APIs, and execute code. That makes them useful. It also means a model can produce a perfectly harmless-looking response while doing something you absolutely did not intend.

I built AgentGate to put a security boundary between what an agent decides to do and what the system actually lets it do.

The dangerous part isn't always what the model says

Most AI guardrails focus on model output: is the response harmful, sensitive, or against policy?

That becomes incomplete once a model has tools.

An agent can say “Generating your report now” while its next tool call deletes a table, reads a credential file, or sends data somewhere it shouldn't.

By the time you inspect the text, the side effect may have already happened.

AgentGate moves the check downstream. It intercepts the actual tool call, including its arguments, immediately before execution.

Agent → tool call → AgentGate → ALLOW / BLOCK → execution

If AgentGate blocks it, the underlying function never runs.

Permission, not prediction

The core design came from a simple inversion.

Instead of trying to enumerate every possible malicious action, AgentGate starts with what the agent is actually supposed to be capable of doing.

agentgate.protect_all()

with agentgate.scope(
    task="Generate Q4 sales report",
    allowed_operations=["read", "aggregate"],
    allowed_resources=["sales_data", "reports"],
):
    run_agent()

Inside that scope, reading sales data makes sense.

Touching users, deleting records, accessing system files, or sending data to an unrelated endpoint does not.

The important part is that AgentGate doesn't need to have seen the exact attack before. The action is outside the declared capability boundary, so it can be rejected by default.

That turned out to be much more useful than building an ever-growing blacklist of bad strings.

Fast when it's obvious. Semantic when it isn't.

Two Tier System | Static + LLM-as-judge

I didn't want every agent action to require another expensive model call.

AgentGate therefore uses two layers.

Tier 1 handles cases that software can reason about deterministically: SQL operations, filesystem paths, HTTP targets, resource boundaries, rate limits, destructive commands, and other concrete signals.

When the answer isn't obvious, Tier 2 escalates the action to a separate LLM acting as a security reviewer. It evaluates the action against the task, allowed scope, blast radius, reversibility, recent agent behavior, and other risk signals.

On the latest evaluation run, only 24.2% of actions required Tier 2 escalation.

And the system is intentionally fail-closed: if an ambiguous action can't be reviewed reliably, it gets blocked rather than silently executed.

One harmless action can become a dangerous sequence

A harder problem appeared once I stopped looking at tool calls individually.

read_file() can be legitimate.

So can send_email().

But repeatedly collecting sensitive data and then sending it to a new destination is a very different behavior.

AgentGate keeps an action history so the reviewer can reason about trajectories, not just isolated calls. That lets it detect patterns like data collection followed by exfiltration, gradual scope creep, or a sequence of individually plausible actions that becomes suspicious as a whole.

On multi-turn adversarial tests, it classified 121 of 126 steps correctly — 96.0% trajectory accuracy.

The first version caught 67%. That wasn't good enough.

I didn't want the project to end with a hand-picked demo showing three attacks getting blocked.

So I built a separate adversarial evaluation harness around it.

The suite grew to 370 evaluations across 180 scenarios, spanning database operations, filesystems, communications, infrastructure actions, subtle boundary violations, and multi-turn attack chains. Tool calls were generated through a real model rather than relying only on fixed malicious strings.

The first run achieved 67.3% recall.

I used the failures — especially false positives and subtle misses — to repeatedly change the policy engine, scope logic, analyzers, and semantic judge.

By Run 007:

99.6% recall · 98.1% precision · 3.6% false-positive rate

100% recall on subtle violations

100% recall on credential access, destructive actions, exfiltration, resource-boundary violations, and escalation chains

The remaining errors are recorded in the evaluation report rather than hidden: one sensitive-data miss and five over-blocked legitimate actions.

That evaluation loop became one of the most important parts of the project.

Built like infrastructure, not a demo

I wanted AgentGate to disappear into an existing agent stack rather than require developers to rebuild around it.

Calling protect_all() can automatically intercept tool execution through the OpenAI SDK and LangChain/LangGraph, including both synchronous and asynchronous paths. Individual Python functions can also be protected with @agentgate.guard.

The security engine sits behind that integration layer, so application code doesn't need to know whether a decision came from a deterministic analyzer or the semantic reviewer.

I also added per-agent rate limiting, configurable scopes, provider-flexible LLM judging, structured block errors, and audit logging that runs outside the execution-critical path and can stream decisions to Supabase.

The library is packaged separately as agentgate-py, so the end result is something installable and reusable rather than security logic buried inside a single demo application.

The boundary I wanted agents to have

AgentGate started with a question:

If we're going to give models increasingly powerful tools, where should the trust boundary live?

I don't think the answer is only in the system prompt, or only in what the model says.

There should be a boundary at the moment intent becomes action.

AgentGate is my attempt to build that boundary.