Semantic model routing

Semantic routing lets the DefenseClaw gateway choose a configured model backend after prompt inspection and before the upstream request. DefenseClaw starts a version-pinned vLLM Semantic Router container, asks it to classify each supported request, resolves the returned model alias locally, and then forwards through the normal guardrail proxy path.

Compatibility at a glance

Routing fails open to the connector's original upstream: if the router is unavailable, times out, or returns an unknown model alias, DefenseClaw logs the routing failure and continues with the original provider selection. Guardrail enforcement does not fail open with it.

Prerequisites

• OpenClaw or ZeptoClaw is configured in proxy mode.
• For a managed router, Docker Desktop or Docker Engine is installed and running Linux containers. A remote router does not require local Docker.
• Every routed backend exposes an OpenAI-compatible Chat Completions endpoint.
• Each base_url is an origin such as https://api.openai.com, without /v1/chat/completions appended.
• Provider credentials are stored in process environment variables or ~/.defenseclaw/.env; never put key values in config.yaml.

Host support

DefenseClaw is hook-only on native Windows. The only connectors eligible for semantic routing—OpenClaw and ZeptoClaw—both require the guardrail proxy and are unsupported there. Semantic routing is a macOS and Linux feature, even when the classifier runs remotely.

The managed sidecar image runs on amd64 and arm64 hosts, including Apple silicon Macs with Docker Desktop.

Configure routing

Add the routing block to ~/.defenseclaw/config.yaml. Model name values are the aliases returned by the semantic router; model, base_url, and api_key_env are resolved inside DefenseClaw and are not sent to the classifier as credentials.

routing:
  enabled: true
  version: "0.3.0"
  port: 8080
  algorithm: static

  models:
    - name: fast
      provider: openai
      model: gpt-4o-mini
      base_url: https://api.openai.com
      api_key_env: OPENAI_API_KEY

    - name: deep
      provider: openai
      model: gpt-4.1
      base_url: https://api.openai.com
      api_key_env: OPENAI_API_KEY

  signals:
    keywords:
      - name: code-request
        keywords: ["debug", "implement", "refactor"]
        operator: OR

  decisions:
    - name: code
      priority: 100
      conditions:
        - type: keyword
          name: code-request
      model_refs: [deep]

    - name: default
      priority: 1
      model_refs: [fast]

Then enable the managed sidecar:

defenseclaw setup routing --enable
defenseclaw-gateway restart

The setup command checks Docker, saves the routing state, and tells the gateway to start the pinned router image on its next launch. The generated router configuration lives under ~/.defenseclaw/semantic-router/; it is runtime state, not a file you need to maintain.

Use a remote router

To use a router you run yourself, set routing.remote.endpoint. The gateway then sends classifier requests there and does not start a local container:

routing:
  enabled: true
  remote:
    endpoint: https://router.example.com
    timeout_ms: 1500     # 0 uses the default; the maximum is 5000
  models:
    # ... same models, signals and decisions as above

Non-loopback endpoints must use HTTPS.

Store routed-model keys with DefenseClaw

Every non-empty routing.models[].api_key_env is part of DefenseClaw's normal key registry. When routing is enabled, keys list marks it as required; keyless local models such as Ollama do not create a credential requirement.

defenseclaw keys list --missing-only
defenseclaw keys set OPENAI_API_KEY

The gateway resolves the selected backend's key only after classification. Secret values and real provider URLs are not written into the generated router catalog or sent in the classifier request. See Keys and credentials for storage precedence and automation-safe commands.

Verify the running state

defenseclaw setup routing --status
docker ps --filter label=com.defenseclaw.component=semantic-router

Send a normal request through OpenClaw or ZeptoClaw. A routed response includes:

X-Semantic-Router: routed
X-Semantic-Router-Reason: decision=… model=… confidence=…

Each guarded request also emits one request-correlated v8 routing decision (defenseclaw.semantic_routing.decision plus latency) on the same trace as the model operation. Labels are a closed set: applied or fallback, a failure code (none, timeout, upstream_status, decode_failure, unknown_alias, credential_failure, configuration_failure), and whether an override was applied. Prompt text, credentials, endpoints, and the header reason string are not metric labels.

No separate public chat endpoint is exposed on the router port. Port 8080 is the loopback classifier API used by the gateway; clients should continue using their normal DefenseClaw connector endpoint. The container uses a read-only config mount, a read-only root filesystem, no Linux capabilities, and a digest-pinned multi-platform v0.3.0 image.

Disable routing

defenseclaw setup routing --disable
defenseclaw-gateway restart

Disabling routing leaves connector and guardrail configuration intact. Requests use their original provider again after the restart.

Troubleshooting

Next steps

• Unified LLM key for the provider keys routed models use.
• OpenClaw and ZeptoClaw, the connectors that support routing.