Semantic model routing
Run vLLM Semantic Router as a managed Docker sidecar and route supported proxy traffic between OpenAI-compatible model backends.
Semantic routing lets the gateway pick a model backend after it inspects a prompt and before it forwards the request. DefenseClaw runs a pinned vLLM Semantic Router container, asks it to classify each request, and forwards through the normal guardrail path. It applies to OpenClaw and ZeptoClaw requests sent as OpenAI-compatible */chat/completions. Hook agents keep their own provider path.
If the router is down, times out or returns an unknown model name, routing falls back to the original provider. Guardrail checks still run.
Prerequisites
- OpenClaw or ZeptoClaw is configured in proxy mode. Native Windows is not supported: there
defenseclaw setup routingsays so and refuses--enable, and--disablestill turns off a routing block copied from another machine. - For the managed router, Docker runs Linux containers on macOS or Linux. A remote router needs no Docker.
- Each backend has an OpenAI-compatible Chat Completions endpoint.
base_urlis the origin, for examplehttps://api.openai.com, without/v1/chat/completions. - Provider keys are in the environment or
~/.defenseclaw/.env, never inconfig.yaml.
Configure routing
-
Add a routing block to
~/.defenseclaw/config.yaml.nameis the alias the router returns;decisionspick one by keyword.~/.defenseclaw/config.yaml routing: enabled: true version: "0.3.0" port: 8080 algorithm: static models: - name: fast provider: openai model: gpt-4o-mini base_url: https://api.openai.com api_key_env: OPENAI_API_KEY - name: deep provider: openai model: gpt-4.1 base_url: https://api.openai.com api_key_env: OPENAI_API_KEY signals: keywords: - name: code-request keywords: ["debug", "implement", "refactor"] operator: OR decisions: - name: code priority: 100 conditions: - type: keyword name: code-request model_refs: [deep] - name: default priority: 1 model_refs: [fast] -
Store the backend key.
defenseclaw keys set OPENAI_API_KEY -
Enable the router and restart the gateway.
defenseclaw setup routing --enable defenseclaw-gateway restartTo use a router you run yourself, set
routing.remote.endpoint(HTTPS for non-loopback endpoints) instead of starting the container.
Verify the running state
defenseclaw setup routing --status
docker ps --filter label=com.defenseclaw.component=semantic-routerA routed response carries these headers:
X-Semantic-Router: routed
X-Semantic-Router-Reason: decision=… model=… confidence=…Each routed request also records one defenseclaw.semantic_routing.decision count and a defenseclaw.semantic_routing.latency measurement. The labels are a closed set: result (applied or fallback), error.code (none, timeout, upstream_status, decode_failure, unknown_alias, credential_failure or configuration_failure) and decision (override or default). Prompt text, credentials, endpoints and the header reason string are never labels.
Disable routing
defenseclaw setup routing --disable
defenseclaw-gateway restartRequests use their original provider again. Connector and guardrail config stay.
Troubleshooting
| Symptom | Check |
|---|---|
| No container appears | Run docker info, then restart the gateway. |
| No routing headers | The agent must be OpenClaw or ZeptoClaw and the path must end in /chat/completions. |
unknown model alias in the gateway log | The router's alias must match a routing.models[].name. |
| Authentication errors from a backend | Check the key with defenseclaw keys list. |
| The router fails to start after you edit the YAML | Check the port, that model names are unique, that every decision's model_refs names a model, and the Docker logs of the labeled container. |
The classifier receives message roles and content. Keys, headers, tools and user ids are not sent. Port 8080 is the loopback classifier API, not a chat endpoint; clients keep using their normal DefenseClaw connector endpoint. The managed container is the digest-pinned v0.3.0 image, with a read-only config mount, a read-only root filesystem, all Linux capabilities dropped and no-new-privileges.
Next steps
Protected files
The gateway watches SSH authorized keys, shell startup files and user autostart entries. A change made while an agent tool call runs raises a HIGH alert that names the agent session. Restore is an opt-in setting.
Unified LLM key
Wire up DEFENSECLAW_LLM_KEY — the single environment variable that powers the LLM judge, the MCP / skill / plugin scanners, and any custom LLM call DefenseClaw makes through Bifrost.