Semantic model routing

Run vLLM Semantic Router as a managed Docker sidecar and route supported proxy traffic between OpenAI-compatible model backends.

Semantic routing lets the gateway pick a model backend after it inspects a prompt and before it forwards the request. DefenseClaw runs a pinned vLLM Semantic Router container, asks it to classify each request, and forwards through the normal guardrail path. It applies to OpenClaw and ZeptoClaw requests sent as OpenAI-compatible */chat/completions. Hook agents keep their own provider path.

If the router is down, times out or returns an unknown model name, routing falls back to the original provider. Guardrail checks still run.

Prerequisites

  • OpenClaw or ZeptoClaw is configured in proxy mode. Native Windows is not supported: there defenseclaw setup routing says so and refuses --enable, and --disable still turns off a routing block copied from another machine.
  • For the managed router, Docker runs Linux containers on macOS or Linux. A remote router needs no Docker.
  • Each backend has an OpenAI-compatible Chat Completions endpoint. base_url is the origin, for example https://api.openai.com, without /v1/chat/completions.
  • Provider keys are in the environment or ~/.defenseclaw/.env, never in config.yaml.

Configure routing

  1. Add a routing block to ~/.defenseclaw/config.yaml. name is the alias the router returns; decisions pick one by keyword.

    ~/.defenseclaw/config.yaml
    routing:
      enabled: true
      version: "0.3.0"
      port: 8080
      algorithm: static
      models:
        - name: fast
          provider: openai
          model: gpt-4o-mini
          base_url: https://api.openai.com
          api_key_env: OPENAI_API_KEY
        - name: deep
          provider: openai
          model: gpt-4.1
          base_url: https://api.openai.com
          api_key_env: OPENAI_API_KEY
      signals:
        keywords:
          - name: code-request
            keywords: ["debug", "implement", "refactor"]
            operator: OR
      decisions:
        - name: code
          priority: 100
          conditions:
            - type: keyword
              name: code-request
          model_refs: [deep]
        - name: default
          priority: 1
          model_refs: [fast]
  2. Store the backend key.

    defenseclaw keys set OPENAI_API_KEY
  3. Enable the router and restart the gateway.

    defenseclaw setup routing --enable
    defenseclaw-gateway restart

    To use a router you run yourself, set routing.remote.endpoint (HTTPS for non-loopback endpoints) instead of starting the container.

Verify the running state

defenseclaw setup routing --status
docker ps --filter label=com.defenseclaw.component=semantic-router

A routed response carries these headers:

X-Semantic-Router: routed
X-Semantic-Router-Reason: decision=… model=… confidence=…

Each routed request also records one defenseclaw.semantic_routing.decision count and a defenseclaw.semantic_routing.latency measurement. The labels are a closed set: result (applied or fallback), error.code (none, timeout, upstream_status, decode_failure, unknown_alias, credential_failure or configuration_failure) and decision (override or default). Prompt text, credentials, endpoints and the header reason string are never labels.

Disable routing

defenseclaw setup routing --disable
defenseclaw-gateway restart

Requests use their original provider again. Connector and guardrail config stay.

Troubleshooting

SymptomCheck
No container appearsRun docker info, then restart the gateway.
No routing headersThe agent must be OpenClaw or ZeptoClaw and the path must end in /chat/completions.
unknown model alias in the gateway logThe router's alias must match a routing.models[].name.
Authentication errors from a backendCheck the key with defenseclaw keys list.
The router fails to start after you edit the YAMLCheck the port, that model names are unique, that every decision's model_refs names a model, and the Docker logs of the labeled container.

The classifier receives message roles and content. Keys, headers, tools and user ids are not sent. Port 8080 is the loopback classifier API, not a chat endpoint; clients keep using their normal DefenseClaw connector endpoint. The managed container is the digest-pinned v0.3.0 image, with a read-only config mount, a read-only root filesystem, all Linux capabilities dropped and no-new-privileges.

Next steps