Semantic model routing

Send each model call to the model that fits it. A built-in rules classifier (or System One, or the vLLM Semantic Router) picks the model, for proxy agents and, through route-only mode, for Claude Code and Codex.

Semantic routing lets the gateway pick a model after it reads a request and before it forwards it. A classifier names the kind of work (for example coding or summarize), and the model you mapped to it answers. Guardrail checks, administrator rules and budgets still apply to the model that is picked, and a routing failure sends the original request unchanged.

See it run

Semantic routing on one laptop, recorded live: turn routing on, restart the gateway to apply it, the Routing panel of the terminal UI, route-only mode for Codex, a cheap prompt to the small model, a hard prompt to the strong model, a sensitive prompt kept on the local model, a rule added while Codex runs, the doctor check, and usage and cost per model. Names and addresses are replaced with examples, prices are example prices, and waits are shortened.

Choose a classifier and a way in

Classifier (routing.classifier)What it isNeeds
rulesKeyword rules in routing.signals and routing.decisions, evaluated inside the gateway. defenseclaw routing setup selects it.Nothing else. Linux, macOS and Windows.
systemoneA decision model (System One or Gemini Decision 1) picks among your categories.A decision-model endpoint and token.
vllm_srThe vLLM Semantic Router container. It is what a 1.0.9 config without routing.classifier still uses.Docker, or a remote router; not available on native Windows.
Way inAgentsHow
Proxy modeOpenClaw and ZeptoClaw, and any agent already sending model calls through the guardrail proxy.Routing applies to the calls on the guardrail path.
Route-only modeClaude Code and Codex.defenseclaw routing enable --connector claudecode (or codex) sends only the model calls through the gateway under /r/<connector>/, for routing and usage. Hooks keep deciding tool calls, and no guardrail scan runs on the model call. defenseclaw routing disable --connector NAME restores the agent settings.

If the classifier is down, times out or names a model that is not configured, routing falls back to the original provider (routing.on_classifier_error).

Prerequisites

  • Model backends in routing.models. base_url is the origin, for example https://api.openai.com, without /v1/chat/completions.
  • Provider keys are in the environment or ~/.defenseclaw/.env, never in config.yaml.
  • Only for vllm_sr: Docker running Linux containers (macOS or Linux), or a remote router. On native Windows defenseclaw setup routing says so and refuses --enable; --disable still turns off a routing block copied from another machine.

Built-in rules classifier (1.0.10)

The rules classifier runs inside the gateway: no Docker container, and it works on Linux, macOS and Windows. defenseclaw routing setup selects it; a config that does not set routing.classifier keeps the 1.0.9 vLLM Semantic Router. It matches the keywords of routing.signals as case-insensitive substrings in the newest request text and sends the request to the model of the highest-priority matching entry of routing.decisions. A request that matches nothing goes to routing.default_alias.

defenseclaw routing setup --yes \
  --model fast=openai/gpt-4.1-mini --model deep=anthropic/claude-sonnet-5-5 \
  --route coding=deep --route reasoning=deep --route summarize=fast
defenseclaw-gateway restart
defenseclaw routing test "refactor this function"

--route CATEGORY=ALIAS (categories coding, reasoning and summarize) writes one keyword signal and one decision per category into config.yaml, with starting keyword lists you can edit. Running it again re-points the model and keeps your edits to the keywords. With --classifier systemone, --route also maps each category to its alias in routing.categories, which is what System One chooses from (the keyword decisions are the rules fallback), and keeps categories you set yourself. The final Test: line names the alias the gateway reports after it has loaded the change: setup asks again for up to 12 s while the gateway still answers with the previous configuration, and says so, rather than printing the old alias, when it has not reloaded by then. Without --route, setup says that no route rules exist: every request goes to the default model. defenseclaw doctor shows the same warning on the Semantic routing row (no route rules (routing.decisions is empty)), and defenseclaw routing status and routing models list show what is configured.

Keyword matching sees the newest user message only. It cannot tell personal data in prose from other text: use routing.sensitive and a profile with local-only models for regulated groups.

A route-only request whose caller cannot be verified (a wrong or replayed route credential, route_credential_invalid; an account other than the one the gateway runs as, not_gateway_owner; an account that is not enrolled) is answered 401 and written to the audit log (action routing.listener_refused, target routing.listener, details connector=NAME reason=CODE), not only to the gateway log. It is its own action: guardrail block counts and dashboards do not include it. One row is written per connector and reason every 10 s, with repeated=N for the refusals it stands for; the connection-limit answer (429) is not audited, and no credential is recorded.

Read what routing did with defenseclaw routing status, routing explain REQUEST_ID, routing test PROMPT (a dry run, no model is called) and routing usage (needs observability.tokenomics).

Configure the vLLM Semantic Router

This is the 1.0.9 flow: DefenseClaw runs a pinned vLLM Semantic Router container and asks it to classify each request.

  1. Add a routing block to ~/.defenseclaw/config.yaml. name is the alias the router returns; decisions pick one by keyword.

    ~/.defenseclaw/config.yaml
    routing:
      enabled: true
      version: "0.3.0"
      port: 8080
      algorithm: static
      models:
        - name: fast
          provider: openai
          model: gpt-4o-mini
          base_url: https://api.openai.com
          api_key_env: OPENAI_API_KEY
        - name: deep
          provider: openai
          model: gpt-4.1
          base_url: https://api.openai.com
          api_key_env: OPENAI_API_KEY
      signals:
        keywords:
          - name: code-request
            keywords: ["debug", "implement", "refactor"]
            operator: OR
      decisions:
        - name: code
          priority: 100
          conditions:
            - type: keyword
              name: code-request
          model_refs: [deep]
        - name: default
          priority: 1
          model_refs: [fast]
  2. Store the backend key.

    defenseclaw keys set OPENAI_API_KEY
  3. Enable the router and restart the gateway.

    defenseclaw setup routing --enable
    defenseclaw-gateway restart

    To use a router you run yourself, set routing.remote.endpoint (HTTPS for non-loopback endpoints) instead of starting the container.

Route on the conversation

By default the router reads only the newest message the user typed. In an agent loop most requests end with a tool result, so the router sees the last thing the user said and nothing of the loop. routing.context lets the rules and systemone classifiers read a little more. It is off by default.

~/.defenseclaw/config.yaml
routing:
  classifier: rules
  context:
    recent_user_turns: 2          # older user messages, 0 to 8
    include_first_user_turn: true # the message that started the conversation
    max_bytes: 8192               # older messages together, at most 65536
    per_turn_bytes: 2048          # one older message, at most max_bytes
    include_tool_names: true      # names of the tools the loop called
  • Read: user messages. Left out: tool results, assistant replies, system prompts, and user-role text that starts with a marker agents write: <system-reminder>, <environment_context>, <user_instructions>, command, hook and IDE tags (<bash-stdout>, <local-command-stdout>, <command-name>, <user-prompt-submit-hook>, <ide_selection> and similar), # AGENTS.md instructions, # CLAUDE.md, Contents of , compaction summaries (Claude Code and Codex), skill loads, <user_shell_command>, the OpenClaw context message, its Sender (untrusted metadata): envelope and its heartbeat and session-startup prompts. Each content block is checked on its own, so a typed prompt in the same message as agent blocks is still read. For a custom slash command the router reads what you typed after the command name (<command-args>) and skips the next block of that message, the expanded command file. Text an agent writes without such a marker can still be read. Tool names are read without their arguments or results; MCP tool names are chosen by the tool server.
  • The newest message keeps its 8 KiB limit; per_turn_bytes and max_bytes bound the older messages.
  • Keyword signals match the newest message and the context, newest first, one message per paragraph, with no recency weighting: an older message can match a higher-priority rule than the newest one. With include_tool_names they also see a line such as tools: Bash,Read, and the request condition tool_loop matches when the conversation called a tool. routing.sensitive.signals match the newest message only.
  • systemone gets the same context, at most 8 KiB. vllm_sr reads the newest message only, so validation refuses routing.context with it.
  • When a limit cuts the context, the decision carries the warning context_truncated:bytes (items past 32 tool names), shown by defenseclaw routing explain. The newest message is never dropped, and a cut never removes a signal the newest message matched.

Verify the running state

defenseclaw setup routing --status
docker ps --filter label=com.defenseclaw.component=semantic-router

The docker ps check applies to vllm_sr only: rules and systemone run inside the gateway and have no container, and --status names the classifier and the state the gateway reports.

A routed response carries these headers:

X-Semantic-Router: routed
X-Semantic-Router-Reason: source=… rule=… model=…

Only the guardrail proxy sends the reason header, and only for chat completions. Route-only mode sends X-Semantic-Router: routed, or X-Semantic-Router: fallback when the routed model failed before its first byte and the original request was sent.

Each routed request also records one defenseclaw.semantic_routing.decision count and a defenseclaw.semantic_routing.latency measurement. The labels are a closed set: result (applied, fallback or denied), error.code (none, timeout, upstream_status, decode_failure, unknown_alias, credential_failure, configuration_failure, filtered, breaker_open, low_confidence or no_candidate) and decision (override or default), plus connector, route_source, route_category, filtered_by, listener, deny_reason and budget_id (see Tokenomics). Both listeners record the count. Prompt text, credentials, endpoints and the header reason string are never labels.

Sensitive requests stay local

With routing.sensitive set (aliases plus severity_at_least and/or signals), a request the guardrail severity or a listed signal marks as sensitive is sent only to a routing.sensitive.aliases model. It is never sent to the model the agent asked for, or to another alias, on any install.

  • What is checked. The newest user turn (its first 8 KiB); tool results and earlier turns are not read. Once a turn of a conversation is sensitive, the following turns of that conversation stay local too, because they carry it in their history; each of them renews the mark, which ends 12 hours after the last such turn. Marks are kept in memory: a gateway restart clears them. A conversation is the agent session id (the DefenseClaw session header, or the Claude Code or Codex session header); a request without one is judged alone. The check is best effort: keywords and the regex guardrail miss personal data written in prose.
  • When no local model can take it (its key is missing, it does not speak the request API, an administrator rule does not allow it, it is unreachable on the route-only listener, or the request is a signed Bedrock body or an API DefenseClaw does not re-route, such as count_tokens), DefenseClaw refuses the request with routing_sensitive_local_unavailable and a message that names the cause, writes an audit row (action routing.sensitive_refused, target routing.sensitive, whichever of these causes it was; any other administrator-rule or budget refusal of a request is routing.admin_denied, target routing.admin; neither counts as a guardrail block), and defenseclaw routing explain <id> shows denied.
  • Size bounds. With routing.sensitive set, the route-only listener refuses a body over 64 MiB (HTTP 413) and answers HTTP 503 with Retry-After while its shared request buffer is full, instead of sending the request unchecked.
  • Keyless local models. A model with its own loopback base_url and no api_key_env or credential (Ollama, vLLM on this host) is called with no credential, also when the agent signs in with a login or a Bedrock bearer key. A sensitive request tries its local models this way at any address.
  • When the local model refuses the request. A local model that does not accept a request the agent sends (for example Ollama and the additional_tools input item Codex sends when its model catalog entry uses use_responses_lite) is not replaced by the cloud model: the agent gets HTTP 400 with a DefenseClaw message that names routing.sensitive, the local alias and the model's own reason. Use a local model that accepts the request, or a Codex catalog entry without use_responses_lite for the model.

What you see

Setup in the terminal UI lists Semantic routing in the Guardrail and scanning group. Its card shows whether routing is on and which classifier it uses.

Setup in defenseclaw tui: the Guardrail and scanning group with Semantic routing selected, showing that routing is on with the rules classifier

The wizard runs the same command you can type, and asks you to confirm it first.

The Semantic routing wizard in defenseclaw tui: the classifier field set to rules and the command it will run, defenseclaw routing setup with yes and classifier rules

The Routing panel lists the models, rules and budgets that routing can use. Its keys add a model, delete one, turn routing on or off, test a prompt, and open usage and prices.

The Routing panel of defenseclaw tui: three models named fast, deep and local, with fast as the default, and one daily budget

The routing explain command shows why one call went where it went: the model the client asked for, the outcome, the category, the source and the reasons.

The routing explain output for a sensitive prompt: requested model us.openai.gpt-5.6-luna, outcome applied to the local alias, category sensitive, source rules, reasons filtered_by:sensitive and sensitive:signal:positive

Going back to a release before 1.0.10

A release before 1.0.10 has no route-only listener, so the Claude Code and Codex model settings route-only changed go back first. defenseclaw rollback (and defenseclaw upgrade --version on Windows) does it in the installer, after its prompt, with the gateway stopped. On Linux and macOS, defenseclaw upgrade --version 1.0.9 asks first (or takes --yes), stops the gateway and puts them back; if they cannot be put back, or the older installer fails, the gateway of this release starts again and writes them again. Restart the agent afterwards. When you run an older release's installer yourself (install.sh --version 1.0.9), run defenseclaw routing disable --connector <name> for each route-only connector first.

Ask the decision model too

The regex floor finds structured data (card and ID numbers, keys, tokens) but misses personal data written in prose: diagnoses, HR cases, unreleased numbers. Set routing.sensitive.check: auto (or decision_model or judge) to also ask the decision model one yes/no question per request; a positive from either keeps the request local. On the 555-prompt router evaluation set, the regex floor alone kept 45% of the sensitive prompts local; the floor plus a check by a Nova Lite-class model kept 99% (1 of 80 leaked on the default profile), and sent 0.4 to 0.6% of the other prompts local. The check adds about 0.4 s (p50) and 0.5 s (p95) to every routed request, because the route waits for it. A check that fails or times out prefers the local models (on_unavailable: local) and never blocks: when no local model can take the request, it keeps its normal route. Privacy: the check sends the request text (on the route-only and passthrough listeners the newest user turn, on the chat listener the newest turns up to max_bytes, never system prompts) to the decision model or, with judge or auto and no decision model, to the guardrail judge on every routed request; choose a local or in-tenant endpoint for it. The numbers above are for the decision-model check; the judge check has no numbers. A profile whose allowed_aliases are local models only leaks nothing in any mode: use that for regulated groups. On a managed host the administrator sets and locks routing.sensitive in the bundle.

Disable routing

defenseclaw setup routing --disable
defenseclaw-gateway restart

Requests use their original provider again. Connector and guardrail config stay.

Troubleshooting

SymptomCheck
No container appearsOnly the vllm_sr classifier starts one: run docker info, then restart the gateway.
No routing headersWith the vLLM router: the call must be on the guardrail path (OpenClaw or ZeptoClaw in proxy mode) and end in /chat/completions. Claude Code and Codex route only after defenseclaw routing enable --connector NAME; check defenseclaw routing status and defenseclaw doctor.
Requests of one alias are answered by the original modelThe alias fails before its first byte (its endpoint is down or answers 404, 503 or 529, or gateway signing or its credential is not ready), so route-only sends the original request instead and records fallback, not applied, with the warning routed_alias_failed:ALIAS:CAUSE. defenseclaw routing status and defenseclaw doctor (the Routed alias row) name each such alias until it serves a call again; fix its base_url and key with defenseclaw routing models list, or remove the route.
Every request goes to the default modelWith the rules classifier, routing.decisions is empty or no keyword matched: run defenseclaw routing setup --route coding=ALIAS, or edit routing.signals and routing.decisions. defenseclaw routing test PROMPT shows the category and why.
unknown model alias in the gateway logThe router's alias must match a routing.models[].name.
Authentication errors from a backendCheck the key with defenseclaw keys list.
The router fails to start after you edit the YAMLCheck the port, that model names are unique, that every decision's model_refs names a model, and the Docker logs of the labeled container.

The classifier receives message roles and content. Keys, headers, tools and user ids are not sent. Port 8080 is the loopback classifier API, not a chat endpoint; clients keep using their normal DefenseClaw connector endpoint. The managed container is the digest-pinned v0.3.0 image, with a read-only config mount, a read-only root filesystem, all Linux capabilities dropped and no-new-privileges.

Next steps