LLM guardrails: defending against data leakage, prompt injection, and more
Guardrails are the runtime checks that sit between your model and the world — catching leakage, injection, and unsafe output in real time. Here's what to guard, where guards go, and why guardrails and evals need each other.
TL;DR: Guardrails are runtime checks on an LLM's inputs and outputs — blocking prompt injection, catching PII and secret leakage, filtering harmful content, and constraining tool actions — while the request is happening. They're the real-time complement to evaluation: evals tell you your defenses hold in aggregate, guardrails stop the individual bad request. You need both.
Evaluation measures whether your system is good on average, over time. Guardrails do something different and immediate: they inspect a single request or response as it flows and block the bad one before it reaches a user or a tool. This guide covers what to guard and how guards and evals fit together.
Input guardrails
Guards that run on what comes in, before the model acts:
- Prompt-injection detection — flag content (typed, retrieved, or returned by a tool) that tries to override the system's instructions. Especially critical for RAG and agents, where the attack rides in on a document. (See how to jailbreak LLMs.)
- PII / sensitive-input handling — detect and redact sensitive data before it enters the prompt or gets logged. (See sampling and redacting PII.)
- Topic / scope enforcement — reject requests outside the app's remit before spending a model call on them.
Output guardrails
Guards that run on what comes out, before it's shown or used:
- Data-leakage detection — catch PII, secrets, or another user's data in the response before it ships.
- Harmful-content filtering — block disallowed output the model produced despite its prompt.
- Groundedness / hallucination checks — verify the answer is supported by the retrieved context, so overreliance becomes a caught failure rather than a user's problem. (See RAG evaluation metrics.)
- Format / schema enforcement — ensure structured output is valid before downstream code trusts it.
Action guardrails (for agents)
For agents, the highest-stakes guard is on actions: enforce least privilege so a manipulated step can't take a destructive tool action, require confirmation for irreversible operations, and treat every tool as a trust boundary. (See secure enterprise deployment strategies.)
Guardrails vs. evals — the division of labor
| Guardrails | Evals | |
|---|---|---|
| When | Real time, per request | Batch, over many traces |
| Job | Block the individual bad case | Measure the system's quality/safety |
| Output | Allow / block / redact | A score and a trend |
| Failure caught | This request | Regressions across a release |
They're complements, not substitutes. A guardrail that fires is also a data point — feed blocked requests into your red-teaming suite and evals so your defenses keep learning. And you evaluate your guardrails themselves: measure their false-positive rate (blocking safe requests) and false-negative rate (missing real attacks), because a guardrail you haven't scored is a guess.
How Currai fits
Currai traces the full request — input, retrieved context, each tool call, and the output — so every guardrail decision is visible in context: what fired, on what, and why. You can score guardrail effectiveness (false-positive and false-negative rates) against production traffic, feed blocked cases into your adversarial dataset, and watch leakage and injection-resistance rates across model and prompt changes. See the LLM security guide and the LLM safety guide, or start with Currai free.
Frequently asked questions
What are LLM guardrails?
Runtime checks on an LLM's inputs and outputs (and, for agents, its actions) that block or redact bad cases as a request happens — catching prompt injection, PII and secret leakage, harmful content, and unsafe tool actions in real time.
How are guardrails different from evaluation?
Guardrails act on a single request in real time to block the bad case; evaluation scores the system's quality and safety across many traces over time. Guardrails stop incidents; evals measure whether your defenses hold. You need both.
Do I need guardrails if I have good evals?
Yes. Evals tell you your system is safe on average, but they don't stop the one malicious request happening right now. Guardrails provide the real-time block that batch evaluation can't.
How do I know my guardrails work?
Evaluate them. Measure false-positive rate (safe requests wrongly blocked) and false-negative rate (real attacks missed) against production traffic, and feed what slips through back into your adversarial test set.