Overview
The guardrail is a Go proxy on the default loopback port 4000. It receives LLM traffic, inspects request prompts before the upstream call, inspects completions after the upstream call, and records a ScanVerdict with action, severity, reason, findings, and scanner attribution. In observe mode findings are logged and traffic continues. In action mode a block verdict is returned in-band.
The local guardrail path is built around internal/gateway/guardrail.go::GuardrailInspector. It can run deterministic regex/rule-pack checks, optional Cisco AI Defense remote inspection, optional LLM judge checks, and final OPA policy evaluation through policies/rego/guardrail.rego.
Inspection surfaces
| Surface | Runtime direction | Code path | Notes |
|---|---|---|---|
| Request messages | prompt | GuardrailProxy.handleChatCompletion -> Inspect | Runs before the upstream model call. |
| Non-streaming response text | completion | handleNonStreamingRequest -> Inspect | Runs after the model response is available. |
| Streaming response text | completion | handleStreamingRequest -> InspectMidStream and final Inspect | Mid-stream checks are regex-only; final inspection runs after accumulation. |
| OpenAI-format tool calls | tool_call | inspectToolCalls -> ScanAllRules | High or critical tool-call findings block in action mode. |
| Gateway tool results | completion | EventRouter.inspectToolResult | Only configured sensitive tools are inspected. |
Detection defaults
The code defaults are set in internal/config/defaults.go and mirrored by internal/config/config.go.
| Key | Default | Meaning |
|---|---|---|
guardrail.enabled | false | The proxy does not bind unless enabled. |
guardrail.mode | observe | Verdicts are recorded but not enforced. |
guardrail.scanner_mode | both | Local rules and Cisco AI Defense can both contribute when configured. |
guardrail.detection_strategy | regex_judge | Global strategy when a direction-specific override is absent. |
guardrail.detection_strategy_completion | regex_only | Completion path uses the deterministic path by default. |
guardrail.judge_sweep | true | regex_judge can run a full judge sweep when regex triage found no signal. |
guardrail.stream_buffer_bytes | 1024 | Initial streaming text buffer before first action-mode flush. |
Core pages
| Page | What it covers |
|---|---|
| Architecture | Proxy, inspector, rules, judge, OPA, and runtime update boundaries. |
| Configuration | Actual guardrail.* keys in GuardrailConfig. |
| Rule packs | Built-in YAML packs and generated rule inventory. |
| Writing rules | The rule YAML shape loaded by internal/guardrail/rulepack.go. |
| Suppressions | Pre-judge strips, PII suppressions, and tool suppressions. |
| Sensitive tools | Tool-result inspection settings. |
| Judge vs regex | Strategy trade-offs and the exact decision path. |
| Verdict cache | The process-local LLM judge verdict cache. |
| Multi-turn | ContextTracker bounds and current runtime limits. |
| Notification queue | The 50-item TTL system-message queue used after blocks. |
| Providers | Provider catalog and adapter matrix. |
| Streaming | Initial buffering, mid-stream scans, and block chunks. |
| Tuning | Source-backed knobs for observe, action, strategy, and policy thresholds. |
| Troubleshooting | Failure modes that map to real code paths. |