Every company deploying a public-facing LLM chatbot inherits the full prompt-injection attack surface: jailbreaks, data exfiltration, PII leakage, indirect injection via retrieved documents. Single-model classifiers and regex filters consistently miss novel attacks, the same way single-AV products missed novel malware a decade ago.
This talk presents an open-source defensive architecture that runs five specialist AI agents in parallel against every prompt and response, each looking at a different attack dimension (injection patterns, semantic intent, encoding tricks, output exfiltration, PII exposure) and aggregates their verdicts before the request reaches the upstream LLM. We'll walk through the architecture, show live attacks bypassing commercial single-model guardrails but caught by the multi-agent pipeline, and discuss the latency/cost tradeoffs that make this practical as inline middleware.
A problem that will be bigger and bigger in the future is that public-facing LLM applications (customer support bots, in-product assistants, RAG-backed chat) are now production infrastructure for thousands of companies, yet the security tooling around them is roughly where web app firewalls were in 2005: regex blocklists and single-shot classifiers. Real-world attacks (DAN-style jailbreaks, indirect injection via poisoned documents, encoding bypasses (base64, Unicode homoglyphs, leetspeak), multi-turn priming attacks) defeat these defenses routinely. Commercial guardrails (Lakera, NeMo, LLM-Guard) help but operate on a single-pass pipeline that struggles with novel, composed, or low-signal attacks.
Proposed approach. I've built and open-sourced an agentic guardrail system that treats LLM defense as a multi-agent reasoning problem rather than a classification problem. The architecture runs concurrent specialist agents in a LangGraph fan-out/fan-in pattern:
Injection Agent — looks at instruction-override patterns, role confusion, system prompt extraction attempts
Encoding Agent — detects base64, hex, Unicode obfuscation, and language-switch bypasses
Context Agent — analyzes indirect injection via retrieved documents and tool outputs
Output Protection Agent — scans LLM responses for system prompt leakage, sensitive data, and policy violations
PII Agent — Presidio-backed entity detection on both prompts and responses
A fast-path ONNX classifier handles obvious traffic in <10ms; uncertain cases escalate to the agent panel. Verdicts are aggregated with a tunable severity policy. The whole pipeline sits as an OpenAI-compatible proxy, so it drops in front of any application without code changes.
What's original. The architecture combines three patterns that, to my knowledge, haven't been deployed together in a production LLM guardrail: (1) parallel-agent supersteps borrowed from agentic forensics tooling, (2) prompt-injection feature engineering from earlier academic work on adversarial input detection, and (3) a dual-engine self-learning loop where a fast ML classifier and a slow LLM panel cross-train each other on disagreements. The result catches novel attacks that single-model defenses miss because no individual agent has the full attack surface in its training distribution — the ensemble disagreement is the signal.
What attendees will see.
Live demo: ~6 attacks (jailbreak, encoded injection, indirect injection via RAG, PII exfiltration, multi-turn priming, output leakage) run against (a) a raw LLM, (b) a commercial guardrail, and (c) the multi-agent system — with the verdict reasoning shown in real time
Architecture walkthrough: LangGraph state, agent prompts, aggregation policy
Latency/cost numbers: real measurements on local Ollama (Llama 3.1 8B) vs. cloud (Haiku, GPT-4o-mini)
Failures and lessons learned — what didn't work: agents agreeing on wrong verdicts, prompt-engineering the agents themselves, the cost blow-up before we added the ONNX fast path
How to run it locally with Docker Compose after the talk