Semantic model routing
Send each model call to the model that fits it. A built-in rules classifier (or System One, or the vLLM Semantic Router) picks the model, for proxy agents and, through route-only mode, for Claude Code and Codex.
Semantic routing lets the gateway pick a model after it reads a request and before it forwards it. A classifier names the kind of work (for example coding or summarize), and the model you mapped to it answers. Guardrail checks, administrator rules and budgets still apply to the model that is picked, and a routing failure sends the original request unchanged.
See it run
Choose a classifier and a way in
Classifier (routing.classifier) | What it is | Needs |
|---|---|---|
rules | Keyword rules in routing.signals and routing.decisions, evaluated inside the gateway. defenseclaw routing setup selects it. | Nothing else. Linux, macOS and Windows. |
systemone | A decision model (System One or Gemini Decision 1) picks among your categories. | A decision-model endpoint and token. |
vllm_sr | The vLLM Semantic Router container. It is what a 1.0.9 config without routing.classifier still uses. | Docker, or a remote router; not available on native Windows. |
| Way in | Agents | How |
|---|---|---|
| Proxy mode | OpenClaw and ZeptoClaw, and any agent already sending model calls through the guardrail proxy. | Routing applies to the calls on the guardrail path. |
| Route-only mode | Claude Code and Codex. | defenseclaw routing enable --connector claudecode (or codex) sends only the model calls through the gateway under /r/<connector>/, for routing and usage. Hooks keep deciding tool calls, and no guardrail scan runs on the model call. defenseclaw routing disable --connector NAME restores the agent settings. |
If the classifier is down, times out or names a model that is not configured, routing falls back to the original provider (routing.on_classifier_error).
Prerequisites
- Model backends in
routing.models.base_urlis the origin, for examplehttps://api.openai.com, without/v1/chat/completions. - Provider keys are in the environment or
~/.defenseclaw/.env, never inconfig.yaml. - Only for
vllm_sr: Docker running Linux containers (macOS or Linux), or a remote router. On native Windowsdefenseclaw setup routingsays so and refuses--enable;--disablestill turns off a routing block copied from another machine.
Built-in rules classifier (1.0.10)
The rules classifier runs inside the gateway: no Docker container, and it works on Linux, macOS and Windows. defenseclaw routing setup selects it; a config that does not set routing.classifier keeps the 1.0.9 vLLM Semantic Router. It matches the keywords of routing.signals as case-insensitive substrings in the newest request text and sends the request to the model of the highest-priority matching entry of routing.decisions. A request that matches nothing goes to routing.default_alias.
defenseclaw routing setup --yes \
--model fast=openai/gpt-4.1-mini --model deep=anthropic/claude-sonnet-5-5 \
--route coding=deep --route reasoning=deep --route summarize=fast
defenseclaw-gateway restart
defenseclaw routing test "refactor this function"--route CATEGORY=ALIAS (categories coding, reasoning and summarize) writes one keyword signal and one decision per category into config.yaml, with starting keyword lists you can edit. Running it again re-points the model and keeps your edits to the keywords. With --classifier systemone, --route also maps each category to its alias in routing.categories, which is what System One chooses from (the keyword decisions are the rules fallback), and keeps categories you set yourself. The final Test: line names the alias the gateway reports after it has loaded the change: setup asks again for up to 12 s while the gateway still answers with the previous configuration, and says so, rather than printing the old alias, when it has not reloaded by then. Without --route, setup says that no route rules exist: every request goes to the default model. defenseclaw doctor shows the same warning on the Semantic routing row (no route rules (routing.decisions is empty)), and defenseclaw routing status and routing models list show what is configured.
Keyword matching sees the newest user message only. It cannot tell personal data in prose from other text: use routing.sensitive and a profile with local-only models for regulated groups.
A route-only request whose caller cannot be verified (a wrong or replayed route credential, route_credential_invalid; an account other than the one the gateway runs as, not_gateway_owner; an account that is not enrolled) is answered 401 and written to the audit log (action routing.listener_refused, target routing.listener, details connector=NAME reason=CODE), not only to the gateway log. It is its own action: guardrail block counts and dashboards do not include it. One row is written per connector and reason every 10 s, with repeated=N for the refusals it stands for; the connection-limit answer (429) is not audited, and no credential is recorded.
Read what routing did with defenseclaw routing status, routing explain REQUEST_ID, routing test PROMPT (a dry run, no model is called) and routing usage (needs observability.tokenomics).
Configure the vLLM Semantic Router
This is the 1.0.9 flow: DefenseClaw runs a pinned vLLM Semantic Router container and asks it to classify each request.
-
Add a routing block to
~/.defenseclaw/config.yaml.nameis the alias the router returns;decisionspick one by keyword.~/.defenseclaw/config.yaml routing: enabled: true version: "0.3.0" port: 8080 algorithm: static models: - name: fast provider: openai model: gpt-4o-mini base_url: https://api.openai.com api_key_env: OPENAI_API_KEY - name: deep provider: openai model: gpt-4.1 base_url: https://api.openai.com api_key_env: OPENAI_API_KEY signals: keywords: - name: code-request keywords: ["debug", "implement", "refactor"] operator: OR decisions: - name: code priority: 100 conditions: - type: keyword name: code-request model_refs: [deep] - name: default priority: 1 model_refs: [fast] -
Store the backend key.
defenseclaw keys set OPENAI_API_KEY -
Enable the router and restart the gateway.
defenseclaw setup routing --enable defenseclaw-gateway restartTo use a router you run yourself, set
routing.remote.endpoint(HTTPS for non-loopback endpoints) instead of starting the container.
Route on the conversation
By default the router reads only the newest message the user typed. In an agent loop most requests end with a tool result, so the router sees the last thing the user said and nothing of the loop. routing.context lets the rules and systemone classifiers read a little more. It is off by default.
routing:
classifier: rules
context:
recent_user_turns: 2 # older user messages, 0 to 8
include_first_user_turn: true # the message that started the conversation
max_bytes: 8192 # older messages together, at most 65536
per_turn_bytes: 2048 # one older message, at most max_bytes
include_tool_names: true # names of the tools the loop called- Read: user messages. Left out: tool results, assistant replies, system prompts, and user-role text that starts with a marker agents write:
<system-reminder>,<environment_context>,<user_instructions>, command, hook and IDE tags (<bash-stdout>,<local-command-stdout>,<command-name>,<user-prompt-submit-hook>,<ide_selection>and similar),# AGENTS.md instructions,# CLAUDE.md,Contents of, compaction summaries (Claude Code and Codex), skill loads,<user_shell_command>, the OpenClaw context message, itsSender (untrusted metadata):envelope and its heartbeat and session-startup prompts. Each content block is checked on its own, so a typed prompt in the same message as agent blocks is still read. For a custom slash command the router reads what you typed after the command name (<command-args>) and skips the next block of that message, the expanded command file. Text an agent writes without such a marker can still be read. Tool names are read without their arguments or results; MCP tool names are chosen by the tool server. - The newest message keeps its 8 KiB limit;
per_turn_bytesandmax_bytesbound the older messages. - Keyword signals match the newest message and the context, newest first, one message per paragraph, with no recency weighting: an older message can match a higher-priority rule than the newest one. With
include_tool_namesthey also see a line such astools: Bash,Read, and the request conditiontool_loopmatches when the conversation called a tool.routing.sensitive.signalsmatch the newest message only. systemonegets the same context, at most 8 KiB.vllm_srreads the newest message only, so validation refusesrouting.contextwith it.- When a limit cuts the context, the decision carries the warning
context_truncated:bytes(itemspast 32 tool names), shown bydefenseclaw routing explain. The newest message is never dropped, and a cut never removes a signal the newest message matched.
Verify the running state
defenseclaw setup routing --status
docker ps --filter label=com.defenseclaw.component=semantic-routerThe docker ps check applies to vllm_sr only: rules and systemone run inside the gateway and have no container, and --status names the classifier and the state the gateway reports.
A routed response carries these headers:
X-Semantic-Router: routed
X-Semantic-Router-Reason: source=… rule=… model=…Only the guardrail proxy sends the reason header, and only for chat completions. Route-only mode sends X-Semantic-Router: routed, or X-Semantic-Router: fallback when the routed model failed before its first byte and the original request was sent.
Each routed request also records one defenseclaw.semantic_routing.decision count and a defenseclaw.semantic_routing.latency measurement. The labels are a closed set: result (applied, fallback or denied), error.code (none, timeout, upstream_status, decode_failure, unknown_alias, credential_failure, configuration_failure, filtered, breaker_open, low_confidence or no_candidate) and decision (override or default), plus connector, route_source, route_category, filtered_by, listener, deny_reason and budget_id (see Tokenomics). Both listeners record the count. Prompt text, credentials, endpoints and the header reason string are never labels.
Sensitive requests stay local
With routing.sensitive set (aliases plus severity_at_least and/or signals), a request the guardrail severity or a listed signal marks as sensitive is sent only to a routing.sensitive.aliases model. It is never sent to the model the agent asked for, or to another alias, on any install.
- What is checked. The newest user turn (its first 8 KiB); tool results and earlier turns are not read. Once a turn of a conversation is sensitive, the following turns of that conversation stay local too, because they carry it in their history; each of them renews the mark, which ends 12 hours after the last such turn. Marks are kept in memory: a gateway restart clears them. A conversation is the agent session id (the DefenseClaw session header, or the Claude Code or Codex session header); a request without one is judged alone. The check is best effort: keywords and the regex guardrail miss personal data written in prose.
- When no local model can take it (its key is missing, it does not speak the request API, an administrator rule does not allow it, it is unreachable on the route-only listener, or the request is a signed Bedrock body or an API DefenseClaw does not re-route, such as
count_tokens), DefenseClaw refuses the request withrouting_sensitive_local_unavailableand a message that names the cause, writes an audit row (actionrouting.sensitive_refused, targetrouting.sensitive, whichever of these causes it was; any other administrator-rule or budget refusal of a request isrouting.admin_denied, targetrouting.admin; neither counts as a guardrail block), anddefenseclaw routing explain <id>showsdenied. - Size bounds. With
routing.sensitiveset, the route-only listener refuses a body over 64 MiB (HTTP 413) and answers HTTP 503 withRetry-Afterwhile its shared request buffer is full, instead of sending the request unchecked. - Keyless local models. A model with its own loopback
base_urland noapi_key_envorcredential(Ollama, vLLM on this host) is called with no credential, also when the agent signs in with a login or a Bedrock bearer key. A sensitive request tries its local models this way at any address. - When the local model refuses the request. A local model that does not accept a request the agent sends (for example Ollama and the
additional_toolsinput item Codex sends when its model catalog entry usesuse_responses_lite) is not replaced by the cloud model: the agent gets HTTP 400 with a DefenseClaw message that namesrouting.sensitive, the local alias and the model's own reason. Use a local model that accepts the request, or a Codex catalog entry withoutuse_responses_litefor the model.
What you see
Setup in the terminal UI lists Semantic routing in the Guardrail and scanning group. Its card shows whether routing is on and which classifier it uses.

The wizard runs the same command you can type, and asks you to confirm it first.

The Routing panel lists the models, rules and budgets that routing can use. Its keys add a model, delete one, turn routing on or off, test a prompt, and open usage and prices.

The routing explain command shows why one call went where it went: the model the client asked for, the outcome, the category, the source and the reasons.

Going back to a release before 1.0.10
A release before 1.0.10 has no route-only listener, so the Claude Code and Codex model settings route-only changed go back first. defenseclaw rollback (and defenseclaw upgrade --version on Windows) does it in the installer, after its prompt, with the gateway stopped. On Linux and macOS, defenseclaw upgrade --version 1.0.9 asks first (or takes --yes), stops the gateway and puts them back; if they cannot be put back, or the older installer fails, the gateway of this release starts again and writes them again. Restart the agent afterwards. When you run an older release's installer yourself (install.sh --version 1.0.9), run defenseclaw routing disable --connector <name> for each route-only connector first.
Ask the decision model too
The regex floor finds structured data (card and ID numbers, keys, tokens) but misses personal data written in prose: diagnoses, HR cases, unreleased numbers. Set routing.sensitive.check: auto (or decision_model or judge) to also ask the decision model one yes/no question per request; a positive from either keeps the request local. On the 555-prompt router evaluation set, the regex floor alone kept 45% of the sensitive prompts local; the floor plus a check by a Nova Lite-class model kept 99% (1 of 80 leaked on the default profile), and sent 0.4 to 0.6% of the other prompts local. The check adds about 0.4 s (p50) and 0.5 s (p95) to every routed request, because the route waits for it. A check that fails or times out prefers the local models (on_unavailable: local) and never blocks: when no local model can take the request, it keeps its normal route. Privacy: the check sends the request text (on the route-only and passthrough listeners the newest user turn, on the chat listener the newest turns up to max_bytes, never system prompts) to the decision model or, with judge or auto and no decision model, to the guardrail judge on every routed request; choose a local or in-tenant endpoint for it. The numbers above are for the decision-model check; the judge check has no numbers. A profile whose allowed_aliases are local models only leaks nothing in any mode: use that for regulated groups. On a managed host the administrator sets and locks routing.sensitive in the bundle.
Disable routing
defenseclaw setup routing --disable
defenseclaw-gateway restartRequests use their original provider again. Connector and guardrail config stay.
Troubleshooting
| Symptom | Check |
|---|---|
| No container appears | Only the vllm_sr classifier starts one: run docker info, then restart the gateway. |
| No routing headers | With the vLLM router: the call must be on the guardrail path (OpenClaw or ZeptoClaw in proxy mode) and end in /chat/completions. Claude Code and Codex route only after defenseclaw routing enable --connector NAME; check defenseclaw routing status and defenseclaw doctor. |
| Requests of one alias are answered by the original model | The alias fails before its first byte (its endpoint is down or answers 404, 503 or 529, or gateway signing or its credential is not ready), so route-only sends the original request instead and records fallback, not applied, with the warning routed_alias_failed:ALIAS:CAUSE. defenseclaw routing status and defenseclaw doctor (the Routed alias row) name each such alias until it serves a call again; fix its base_url and key with defenseclaw routing models list, or remove the route. |
| Every request goes to the default model | With the rules classifier, routing.decisions is empty or no keyword matched: run defenseclaw routing setup --route coding=ALIAS, or edit routing.signals and routing.decisions. defenseclaw routing test PROMPT shows the category and why. |
unknown model alias in the gateway log | The router's alias must match a routing.models[].name. |
| Authentication errors from a backend | Check the key with defenseclaw keys list. |
| The router fails to start after you edit the YAML | Check the port, that model names are unique, that every decision's model_refs names a model, and the Docker logs of the labeled container. |
The classifier receives message roles and content. Keys, headers, tools and user ids are not sent. Port 8080 is the loopback classifier API, not a chat endpoint; clients keep using their normal DefenseClaw connector endpoint. The managed container is the digest-pinned v0.3.0 image, with a read-only config mount, a read-only root filesystem, all Linux capabilities dropped and no-new-privileges.
Next steps
- Guard an ACP editor
- Routing rules and budgets for administrator rules, budgets and what users can read
- Tokenomics for the cost and usage of routed calls
- OpenClaw and ZeptoClaw
Model routing
Send each model call to the model that fits it, limit which models a group may use, cap spend, and see what every call cost. Where to start, and which page covers what.
Routing rules and budgets
Limit which models each group may use and cap model spend per user, group or connector, on one device or across the fleet with the opt-in budget store.