Sandbox telemetry
The v8 records DefenseClaw writes for OpenShell sandboxes — phase changes, egress decisions with bytes and duration, asks, policy changes, shadow AI and other findings, processes, integration health — and the attributes that tie them to one sandbox.
Every OpenShell sandbox writes canonical v8 records, the same
kind as the rest of DefenseClaw's observability. They
go to the local SQLite history and to each destination whose routes select
them, redacted by that destination's profile. This page lists what is recorded,
how a record names its sandbox, and which records tell you something is wrong.
The Sandboxes board
and the Splunk Observability dashboard are built
from these records; the full attribute registry is
docs/OPENSHELL_SANDBOX_EVENTS.md.
What is recorded
| What happened | Record family | Bucket | Notes |
|---|---|---|---|
| A phase change | log.sandbox.lifecycle | agent.lifecycle | The phases are creating, provisioning, starting, ready, stopping, stopped, completed, error, deleting, unknown and deleted. A record names the trigger (create, start, stop, delete, watch, reconcile), the previous phase and the exit code. |
| A snapshot, undo, mask, review, upload, pull, or the quarantine of a nested repository | log.sandbox.workspace | enforcement.action | Counts files and lines and lists up to 64 project-relative paths. A state change, and any record that flags files able to run code, is always written to local SQLite. |
| An egress decision, from the DefenseClaw egress proxy or from OpenShell's own network boundary | log.egress.allowed, log.egress.blocked | network.egress | Source dc-egress-proxy or openshell. A blocked decision is always written to local SQLite. |
| The end of a connection the proxy allowed | log.egress.completed, log.egress.failed | network.egress | Bytes up and down and the duration. failed is a connection the proxy cut short (cancelled, with the bytes), or one whose DNS, connect, TLS or upstream failed (failed, a 502) or timed out (timed_out, a 504). |
| An ask, and your answer | log.approval.requested, log.approval.resolved | compliance.activity | A resolution says approved, denied, expired or cancelled, who decided (you, triage or policy) and the scope (this sandbox or always). |
| A policy change: applied, a rule added or removed, an unblock or block | log.policy.updated | compliance.activity | Always written to local SQLite. Carries the OpenShell policy revision and the policy hash. |
A finding (sandbox.<kind>, see below) | log.finding.observed | security.finding | |
| A process OpenShell launched or ended, an SSH session, a model call | log.sandbox.process, log.sandbox.ssh, log.sandbox.inference | tool.activity, compliance.activity, model.io | From OpenShell's own reports. Process and SSH records are paced per sandbox (a burst of 200, then 20 a second). An inference record names the provider, model, status and latency, never the content. |
| A process joined or left the process tree | log.sandbox.process_tree | agent.lifecycle | Only for a sandbox whose tree is on (observe.process_tree, or sandbox run --process-tree). Carries the parent and the ancestry. |
| The integration's health | log.subsystem.lifecycle, .ready, .degraded, .restored | platform.health | Subsystem openshell. A transition is always written to local SQLite. |
| An AI component found inside a sandbox | ai_component.* | ai.discovery | Carries defenseclaw.sandbox.id and defenseclaw.sandbox.name. See AI discovery. |
| A hook decision | log.compat.hook_decision | Carries the sandbox id and name of the binding that authenticated the hook, so a verdict joins to its sandbox. |
The audit actions behind them are sandbox-lifecycle, sandbox-workspace,
sandbox-egress, sandbox-approval, sandbox-policy, sandbox-health,
sandbox-finding, sandbox-activity and sandbox-process; they appear in
defenseclaw audit export and the TUI's Audit view.
In Loki the family without its log. prefix is defenseclaw_event_name (for
example egress.blocked, finding.observed, sandbox.process_tree).
Which sandbox a record belongs to
Every sandbox record carries these attributes when they are known. An empty value is left out, never guessed.
| Attribute | Meaning |
|---|---|
defenseclaw.sandbox.name | The sandbox's name. Required on every record except the integration's own health. |
defenseclaw.sandbox.id | OpenShell's id, once the gateway accepted the sandbox. |
defenseclaw.sandbox.driver | The compute driver: docker (the Linux default) or vm (MicroVM: every Mac, and a Linux host with KVM that chose it). |
defenseclaw.sandbox.driver_reason | Managed hosts: why the sandbox runs on its driver. configured (the organization set it), kvm_available (driver: auto and the user's KVM probe passed), or, when the Linux docker fallback replaced the MicroVM driver, the probe's reason: kvm_missing, kvm_access, cpu_virtualization, vm_driver_missing or vm_prerequisite_missing. See The docker fallback. |
defenseclaw.sandbox.profile, .pack | The network profile and the policy pack. |
defenseclaw.sandbox.workdir.mode | mount or copy. |
defenseclaw.sandbox.binding.id | The ingress binding the sandbox's hooks and egress proxy credential authenticate with (sb_ and hex; never the token). |
user.id, defenseclaw.user.name | The host account that launched the sandbox. |
gen_ai.conversation.id | The harness session the sandbox's hooks last named, on egress, approval, process, SSH and inference records. |
defenseclaw.connector.source | The sandbox's harness connector. |
Sandbox names are never metric labels
The sandbox name, id and binding are attributes of the log records only. The
metrics carry the harness connector, the egress source and the decision, so a
sandbox that is created and deleted a hundred times does not make a hundred
series. Per-sandbox views read the logs, which is why the Collector copies
defenseclaw.sandbox.name into Loki structured metadata.
Managed computers
On a managed host (the enterprise packages, with identity-based sandbox policy) the managed gateway also stamps each sandbox record with what it knows of the sandbox's governance. These attributes are left out on an unmanaged sandbox, and none of them is a metric label.
| Attribute | Meaning |
|---|---|
defenseclaw.sandbox.registration.id | The managed gateway registration the sandbox's hooks and policy requests are bound to (reg-; never the binding token). |
defenseclaw.guardrail.profile.name, .digest, .match, .matched_group | The guardrail profile that governs the sandbox, the digest of its configuration, and why it matched. The same attributes the hook decisions carry. |
defenseclaw.sandbox.policy.digest | The digest of the effective sandbox policy: the organization floor combined with the profile's sandbox section. |
defenseclaw.sandbox.policy.generation | The managed gateway's configuration generation at the last resolve or re-match. |
defenseclaw.sandbox.policy.provisional | true while the sandbox runs under default_profile because the directory lookup had not resolved. |
defenseclaw.openshell.version, defenseclaw.openshell.install_mode | The OpenShell release the daemon verified, and how it was provided: prerequisite or setup. |
defenseclaw.subject.key | An hmac-sha256: pseudonym of the account, so one person can be followed across records without the principal. The principal itself is recorded only when ai_discovery.include_user_principal is on. |
The records the managed computer adds:
| What happened | Record | Notes |
|---|---|---|
| A registration's profile and sandbox policy were resolved | log.sandbox.policy.resolved | Reason offline_floor for a provisional one. |
| A live registration was matched again | log.sandbox.policy.rematched | The reason is generation, facts or lookup_resolved, with the previous profile and policy digest. Always written. |
| A network rule that nothing accounts for was removed, its sandbox stopped, or only reported | log.sandbox.policy.foreign_rule | The action is remove, stop or alert. |
| The managed gateway decided a person's ask, unblock or block | log.sandbox.ask.decided | Says whether the decision was kept. Always written. |
| A required agent was started on the host | log.sandbox.host_run_refused | Always written. |
| This computer cannot provide a sandbox for a required agent | log.sandbox.host_run_unavailable | The reason is one of os_unsupported, no_docker_group, no_docker_access, no_kvm, no_kvm_group, no_kvm_access, prerequisite_missing and similar; the refusal asks the person to quote it. An alert-only case has action allow. |
| A sandbox run refused because another account is using the sandboxes of a single-account computer | log.sandbox.host_run_unavailable | The reason is account_busy and the action refuse. The record carries the refused person's own profile and subject key and names no other account. Repeated attempts are recorded at most once every 5 seconds. |
| The managed policy refused a destination | log.sandbox.access_refused | Names the rule and server.address; the rule is the caller's own match only. Always written. |
| A sandbox or an agent container DefenseClaw did not create | log.sandbox.foreign_sandbox, log.sandbox.foreign_container | A sandbox is stopped, deleted when it cannot stop, or reported; a container is only reported, with its image in defenseclaw.sandbox.container.image. Both carry the account's own profile match and the detected agent connector, including when the account has no registered sandbox. |
An OpenShell setting differs from enterprise.openshell | log.sandbox.settings_drift | defenseclaw.openshell.settings.field names the key, never the value. Pauses the user's sandboxes. Always written. |
A refusal, a removal or stop, a re-match and an ask decision are mandatory: they reach the audit floor whatever a route's collection settings say. The Grafana Sandboxes board has a managed fleet section built from these records, with matching Splunk saved searches and Splunk Observability charts and a sandbox errors detector.
Egress records
An egress record names the destination (server.address, server.port,
defenseclaw.network.target_ref), the dialed address, the decision (allow or
block), a stable decision code, a short reason and the source. OpenShell's
records also name the program that connected (defenseclaw.sandbox.process.executable
and .pid); both are what the sandboxed program claims about itself, not
something DefenseClaw verified.
| Decision code | When |
|---|---|
SANDBOX_EGRESS_ALLOWED | The end of an allowed proxy connection (log.egress.completed). |
SANDBOX_EGRESS_OPENSHELL_DENIED | OpenShell's own boundary refused the connection. |
SANDBOX_EGRESS_HOST_PORT_ASK | The refusal of a --host-port port that raised an ask. |
SANDBOX_EGRESS_UPSTREAM_FAILED | The proxy could not reach the site: DNS, connect, TLS or timeout. |
SANDBOX_EGRESS_TERMINATED | The proxy or OpenShell ended an allowed connection: the large-upload block, the idle timeout, a refused TLS name, a recheck, or a policy reload. |
SANDBOX_EGRESS_LOOKUP_REFUSED, SANDBOX_EGRESS_HARNESS_FETCH | A refused name lookup, or a request the harness makes on its own and does without. Audited at INFO and not an alert. |
SANDBOX_EGRESS_ plus the category (PACK_BLOCK, REPO_POLICY_BLOCK, FIREWALL_BLOCK, OPERATOR_BLOCK, EGRESS_OFF, ...) | A refusal by the merged block list, naming whose entry it was. |
The model provider's hosts are opened to the harness's own program directly,
around the egress proxy, so the proxy records no byte counts for a model call.
OpenShell's report of the call is a log.sandbox.inference record.
Findings
A finding's category is sandbox.<kind>:
| Kind | Meaning | Severity |
|---|---|---|
shadow_ai | An AI API the sandbox reached or tried to reach that is neither its model provider nor its harness's vendor. At most once per provider and severity per session. | MEDIUM when reached, LOW when refused |
hook_silence | The harness of a user-tier sandbox worked without a hook request reaching DefenseClaw for hooks.silence_after. In balanced and strict the sandbox is stopped. | HIGH |
hook_tamper | A tool ran without a DefenseClaw verdict: a PostToolUse whose PreToolUse was denied or never arrived. | HIGH |
large_upload | A large upload to a host the sandbox had not contacted before. | MEDIUM when only reported, HIGH when the large-upload block cut it |
nested_repo | A repository or a gitlink appeared inside a live-mounted project during a session. | HIGH, CRITICAL when the quarantine failed |
ocsf_finding | A finding OpenShell raised. Its repeats within 10 minutes of a session are one record. | as OpenShell reports |
Health
Health records use the subsystem openshell. A degraded record carries a stable
error code in lower case (defenseclaw.schema.error_code) and a summary that
says what to do.
| Code | Meaning | What to do |
|---|---|---|
openshell_unavailable | The OpenShell gateway is not running, is not this account's, or does not answer. | defenseclaw sandbox doctor |
openshell_egress_auth_failed | The egress proxy refused requests that presented an invalid proxy credential (malformed, unknown, revoked or wrong). A request with none is the normal first step of the handshake and is not counted. One record opens a streak, the daemon log counts the later refusals at most once a minute, and a minute without one ends it with a restored record. | A sandbox may hold a stale or revoked credential (an environment from before a restart), or another program on this machine is trying the proxy. The summary names the last destination. defenseclaw sandbox doctor. |
openshell_telemetry_failed | The recorder refused a sandbox record. The first refusal of a streak is recorded and logged; every one is counted on defenseclaw sandbox status (the red Telemetry row) and in GET /api/v1/sandbox/status (telemetry_failures, telemetry_error). | defenseclaw doctor and the observability destinations. |
openshell_watch_failed | Events were lost on a sandbox's stream. A cursor out of range (the gateway restarted, as an upgrade does) is MEDIUM and no alert; one the gateway could not have issued is HIGH. | defenseclaw sandbox status NAME |
Reconstruct a session
To work out what an agent did in a sandbox, from the records alone:
-
On the machine, the same facts are one command each:
defenseclaw sandbox status NAME(counts, hook traffic, the last blocked call),defenseclaw sandbox activity --sandbox NAME(the feed, in order),defenseclaw sandbox destinations NAME(every host, what it is, how much moved, which program connected),defenseclaw sandbox ps NAME --tree(the processes, when the tree was on),defenseclaw sandbox discover NAME(what the agent installed), anddefenseclaw alertsfor the findings. Destinations and the last discovery stay after the sandbox stops;pssays when the tree was last sampled. Deleting the sandbox removes them. -
defenseclaw audit export --since 1hwrites the audit records, hook decisions included, for the same window. -
In Grafana, set Sandbox on the Sandboxes board. To read the records in order, open Explore on Loki and filter on the sandbox name, then on one family:
{service_name="defenseclaw"} | defenseclaw_sandbox_name="myapp-claude" {service_name="defenseclaw"} | defenseclaw_sandbox_name="myapp-claude" | defenseclaw_event_name="egress.blocked"Another destination (Splunk, a file) holds the same records; filter on
defenseclaw.sandbox.namethere. In Splunk, the audit action names the kind of record. This search counts a two-hour session's records by action (the search termspl-uais that sandbox's name):
-
Treat a program's name and arguments as the agent's own words. The tree samples every 5 seconds, so a process shorter than that is missing from it; a connection's program is what OpenShell reports, and its parents appear only for a process the tree saw. The model provider's traffic does not go through the egress proxy, so it has no byte counts, only OpenShell's
log.sandbox.inferencerecords.
Metrics
The labels below are the ones the local Prometheus shows. In OTLP they are
defenseclaw.connector.source, defenseclaw.metric.source and
defenseclaw.metric.decision.
| Metric | Labels | Notes |
|---|---|---|
defenseclaw.sandbox.active (gauge) | connector | Sandboxes provisioning, starting, ready or stopping, per harness connector. |
defenseclaw.sandbox.transitions (counter) | connector, phase from and to | One per phase change. |
defenseclaw.egress.events (counter) | connector, source (openshell, dc-egress-proxy), decision | One per decision; none for the end of a connection. A refusal that is audited only does not count. |
What leaves the host
- The command line of a process is content class: each destination's redaction profile decides whether it leaves the host. Before that, DefenseClaw replaces the values of arguments that name secrets and keeps at most 1,024 bytes.
- Destinations, file names, finding targets and the program names in a record come from the sandboxed agent. A value that does not fit its registered shape is cut or left out; it never stops the record from being written.
- The egress proxy never records URL paths or query strings. OpenShell's HTTP events can add the path of a plain-HTTP request.
- Local SQLite keeps every collected record; the mandatory ones marked above are kept even when a route turns the bucket's collection off.
See also
Agent360 dashboard
Drill from any observed agent into lifecycle, executions, model and tool activity, tokens, cost, security decisions, hook enforcement/recovery paths, traces, and agent topology.
Galileo
Send DefenseClaw GenAI traces to Galileo Cloud or self-hosted Galileo while keeping local observability enabled.