An LLM application remains secure only when unsafe requests, responses, and tool actions are stopped before they affect users or connected systems. Since each failure occurs at a different stage, a single guardrail rarely covers the full application.
Teams must verify their security controls before release and continue measuring them as prompts, models, and tool access change. Testing and production monitoring reveal where filters, validators, and permission rules miss threats, block valid requests, or regress.
This guide compares guardrail and security testing tools across five defense layers. Braintrust provides the evaluation and monitoring layer for testing security controls before release and measuring their performance in production.
The five layers of an LLM security stack
LLM security failures can occur at several points between a user request and an application action, so product coverage should be assessed by control layer. Use the table below to map each failure mode to the control responsible for preventing or detecting it.
| Layer | Primary role | Example failure |
|---|---|---|
| 1. Input filtering | Detects prompt injection, jailbreaks, and other prohibited inputs before the model processes them | A retrieved document contains hidden instructions that redirect the model from its assigned task. |
| 2. Output moderation | Blocks unsafe or policy-violating responses before they reach users | A support agent provides medical advice outside its approved scope. |
| 3. Schema validation | Rejects malformed or unexpected structured output before downstream processing | An agent returns invalid JSON and causes the downstream pipeline to fail. |
| 4. Tool permissions | Restricts which tools, actions, and parameters an agent can use | An agent issues a refund above the approved limit or calls a tool outside its assigned scope. |
| 5. Evaluation and monitoring | Measures whether security controls catch prohibited behavior, allow valid requests, and remain effective as the application changes | A prompt or model update weakens jailbreak detection, and the regression reaches production without being identified. |
For a deeper look at how the evaluation layer works, see the guide to what LLM guardrails are and how to implement them with evals.
Best LLM guardrail and security testing tools
Four tools cover the five layers because Guardrails AI combines output moderation and schema validation within one framework.
Layer 1. Lakera Guard for input filtering
Lakera Guard, now offered through Check Point AI Security, screens untrusted content before the application proceeds with an LLM interaction or agent step.
- Speed: The Guard API screens content in the live request path and runs as a hosted service or a self-hosted deployment.
- Detection depth: Policies can detect prompt injection, jailbreaks, sensitive-data exposure, content violations, malicious links, and off-policy agent behavior across prompts, retrieved content, and tool responses.
- Simple integration: The application submits content to the Guard API and receives threat flags and categories, then uses the returned flags to decide whether to block, redact, or record the interaction.
Operating limits: Lakera identifies threats but does not control downstream permissions within connected systems. Teams should measure latency, false positives, and missed attacks using representative traffic.
Layers 2 and 3. Guardrails AI for output moderation and schema validation
Guardrails AI provides configurable validation for Python applications that need to inspect model responses or produce structured output.
- Validator library: Prebuilt and custom validators can check specific requirements such as content safety, topic restrictions, or sensitive data. Configurable failure actions determine how invalid output is handled.
- Schema validation: Developers define the expected response structure with a Pydantic model. Guardrails AI validates the generated JSON and returns the validated result or applies the configured failure action.
- Deployment control: Validation can run within the Python application or through a self-hosted Guardrails server.
Operating limits: Coverage depends on the validators selected and how they are configured. Teams remain responsible for testing whether validators miss violations or reject acceptable responses.
Layer 4. NVIDIA NeMo Guardrails for tool permissions and orchestration
NVIDIA NeMo Guardrails applies code-defined controls to conversations, retrieval, model responses, and external tool use.
- Five rail types: Input, retrieval, dialog, execution, and output rails apply controls at different stages of an LLM interaction.
- Conversation control: Colang flows can guide multi-turn behavior, while execution rails validate tool calls, arguments, and returned results.
- Broad model support: The library works with multiple hosted and self-hosted models, while custom Python actions connect application-specific checks and external services.
Operating limits: Teams must configure, integrate, deploy, and maintain the selected rails. Authentication and account-level permissions must still be enforced by the application and connected service.
Layer 5. Braintrust for guardrails evaluation and monitoring
Braintrust measures whether security controls catch prohibited behavior, allow valid requests, and remain effective as prompts, models, tools, and policies change. It connects pre-release testing with production results, giving teams a consistent process for approving security changes and identifying regressions.
- Security datasets and experiments: Teams can build datasets from prompt injections, jailbreaks, policy violations, unauthorized tool actions, and valid requests that guardrails should allow. Experiments compare control configurations against the same cases, while custom scorers can turn agreed security thresholds into CI pass or fail criteria. Failed production cases can also be added to evaluation datasets, extending future test coverage with attacks and edge cases observed in live traffic.
- Custom scorers as policy checks: Code-based scorers measure deterministic requirements, while LLM-as-a-judge scorers evaluate policies requiring contextual judgment. Trace-level scoring can also inspect tool calls and intermediate agent steps that may not appear in the final response.
- Online scoring: Security criteria defined during development can run as online scoring, evaluating all or a sample of production traces asynchronously without adding latency to live requests.
- Alerts and automations: Teams can monitor score thresholds and log conditions, then send notifications or webhooks when guardrail performance declines. Braintrust alerts support investigation and response across engineering, product, and security teams.
Operating limits: Braintrust provides the evaluation and monitoring layer. Live prompts, responses, and tool calls still require runtime guardrails or application controls that block unsafe behavior during execution.
Also read: Braintrust's security controls and AI compliance and governance guide to understand how evaluation data can be stored, governed, and presented for security and compliance review.
Best LLM guardrails tools compared (2026)
Use the comparison below to identify each tool's role in the security stack and the controls that must be supplied separately.
| Tool | Layer covered | Open source or hosted | Latency profile | What it does not cover |
|---|---|---|---|---|
| Braintrust | Layer 5, evaluation and monitoring | Hosted with a free Starter plan; on-prem deployment is available on Enterprise | Online scoring runs asynchronously outside the request path without affecting application latency | Inline filtering or blocking of prompts, responses, and tool actions |
| Lakera Guard (Check Point AI Guardrails) | Primarily Layer 1, with additional screening for model output and agent activity | Managed SaaS; Enterprise customers can self-host through Docker, Kubernetes, or an air-gapped deployment | Runs in the request path; latency varies with content length, enabled detectors, and deployment region | Schema validation and application-wide evaluation across prompts, models, business criteria, and end-to-end traces |
| Guardrails AI | Layers 2 and 3, output moderation and schema validation | Apache 2.0-licensed Python library; can run within an application or as a self-hosted server | Runs in the request path; latency depends on the selected validators, with model-based checks and reasks adding more processing time | Application-level tool permissions and centralized measurement across releases and production traffic |
| NVIDIA NeMo Guardrails | Primarily Layer 4, with additional rails for input, retrieval, dialog, and output control | Apache 2.0-licensed Python library; supports self-managed servers and Kubernetes deployments | Runs in the request path; latency and model usage depend on the configured rails and underlying safety models | A managed evaluation process spanning experiments, release checks, and asynchronous production scoring |
Why guardrails need evaluation and monitoring
A guardrail can return an allow or block decision without indicating whether it handled the interaction correctly. Prompt-injection detectors may miss disguised attacks, output validators may reject acceptable responses, and tool policies may approve actions under prohibited conditions. Pre-release evaluation measures both missed threats and incorrect blocks across known attacks, borderline cases, and valid requests, so teams have clear requirements for release approval.
Guardrails that perform well during testing may lose accuracy as prompts, models, retrieved content, response formats, agent tools, and attack techniques change. Production monitoring identifies the interactions behind declining scores so teams can investigate each failure, add it to the evaluation dataset, update the affected control, and verify the correction before release.
Braintrust applies consistent security criteria across testing and production. Custom scorers measure guardrail performance before release, online scoring evaluates production traces, and alerts notify engineering and security teams when scores meet defined failure conditions.
Start evaluating guardrail performance with Braintrust's free tier →
How to build your LLM guardrails stack
The right combination of guardrails depends on what the application receives, produces, and has permission to do. Cover the failure modes already present in the product, then expand the stack as outputs become structured or agents gain access to external tools.
Chat and content-generation features → Start with input and output screening for the risks exposed by the feature, supported by evaluation cases that include both prohibited and acceptable requests. If Lakera Guard handles runtime screening, evaluations should measure attacks that pass through and valid requests that are incorrectly blocked.
Applications with structured outputs → Add schema validation before model output reaches downstream systems. Guardrails AI can validate structured responses against a Pydantic model and reask the LLM when the output does not meet the schema. The application should also define how to handle responses that remain invalid after the allowed retries.
Agents that call external tools → Add controls before the agent can issue refunds, send messages, modify records, or perform other consequential actions. NeMo Guardrails execution rails can validate tool calls, arguments, and returned results, while authentication and account-level permissions remain with the application and connected service.
Regulated or high-impact applications → Map all five layers to the application's documented risks and control requirements. An application that accepts untrusted input, produces user-facing output, passes structured data downstream, and takes external actions needs coverage across the complete stack. Evaluation records can support audit evidence, but they do not replace access controls, incident procedures, or compliance documentation.
Lakera Guard, Guardrails AI, and NeMo Guardrails operate within the runtime path, while Braintrust evaluates whether the combined controls meet defined security criteria before release and in production. Each layer requires separate integration, ownership, maintenance, and latency planning.
FAQs: Best LLM guardrails and security testing tools (2026)
What is the difference between LLM guardrails and evals?
A guardrail handles a live interaction by allowing, blocking, modifying, or escalating it. An eval tests representative cases to measure whether the guardrail applies the policy correctly. The same security requirement may inform both, but an individual guardrail decision does not reveal the control's overall miss rate or the number of valid requests it rejects. The guide to LLM guardrails and evals covers this relationship in more detail.
Do I need a separate guardrails product?
Model-provider filters may handle prohibited content, validation libraries can enforce response schemas, and connected services should control account-level permissions. A dedicated runtime guardrail becomes necessary when an application needs specialized prompt-injection detection or consistent policies across multiple models. Braintrust can then evaluate whether the combined controls meet the application's security requirements.
How fast can guardrails catch violations?
Inline guardrails can detect a violation within the live request, before a model response or tool action is completed. Detection time depends on the selected checks, content length, and deployment path, which also determine the latency added to the request. Asynchronous scoring runs after the interaction, so notification speed depends on trace ingestion, scoring, batching, and alert settings. Actions that could cause immediate harm require inline enforcement, while asynchronous detection supports investigation and follow-up.
How do guardrails relate to prompt injection and red teaming?
Red teaming probes an application with adversarial inputs to uncover exploitable behavior before release, while a prompt-injection guardrail screens live inputs and retrieved content for similar attacks. Successful red-team prompts should become regression cases for the detector, and new production attacks should be added to the next red-team assessment. Feeding cases in both directions keeps testing aligned with the attacks encountered during development and production.