Safety model
The safety model
What GUS defends against, what it doesn't, and where the boundary sits. If a claim here ever disagrees with the code or the drills, the code and drills win.
The core assumption
The model is not trusted. Not because models are malicious, but because everything a model reads is untrusted input: a fetched page, a file, an MCP server response, or a pasted chat message can carry instructions written by someone else (prompt injection), and the model's own memory of the rules can be silently trimmed by context compaction. So in GUS, no safety property is allowed to depend on the model behaving. Each one is enforced by a component the model cannot reach.
What's at stake
- Credentials (API keys, OAuth tokens) — held by the Vault Broker.
- Money (model spend) — metered by the Governor.
- Your data and machines — workspace files, accounts the agent acts on.
- The integrity of the record itself — ledger, receipts, approvals. A security product whose logs lie is worse than none.
Governor modes: enforcing & permissive
SELinux had the right idea: an enforcement system people can relax while they learn it gets adopted; one they can only suffer gets uninstalled. GUS has two modes.
Enforcing (default)
Shield verdicts bind. Classifier parks and denies bind. This is the posture every install ships in.
Permissive
Everything is still scanned by Agent-Shield and classified by the policy layer — but a would-be block (Shield), park (approval), or deny (classifier) proceeds instead. Each let-through is recorded with what enforcing mode would have done: the gate log carries mode: "permissive" and wouldBe: "block" with the real verdict and its receipt, and classifier let-throughs land in permissive.jsonl. The panel shows today's count and the recent list; /mode in chat reports the same. It is the firewall-disable troubleshooting loop — except the thing you turned off keeps the receipts, so hardening again is an informed decision, not a guess.
Flip it in the panel (Status → Governor mode), the GUS.app menu, or by chat command (/mode permissive / /mode enforcing — operator-only; delegates are refused, like /stop and /update). One state file, read at every decision point, so a flip is live on the next action with no restart.
The floors no mode can cross
- Vault reads stay denied. A secret entering model context is an irreversible disclosure, not a reversible action. The control plane floors it before the mode is consulted.
- The spend caps still stop at the ceiling, and the kill switch still halts everything. Those are Governor mechanics — runaway insurance — not content policy. The mode never reaches them.
- Scanning and receipting never turn off. A mode with no receipts is just an unaccountable agent, and that product is not GUS.
- Some action classes are never delegable, in any mode: vault reads, deletes, installs, sends and posts, git push, sudo/kill-class shell, and keychain access. See approval delegation.
Threats and what stops them
| Threat | Mechanism |
|---|---|
| Prompt injection in fetched content or tool results tries to exfiltrate a credential | Egress allowlist + injection bindings: the credential can only ever travel, injected by the proxy, to its one bound destination; fetched content is Shield-scanned and vault-scrubbed before it reaches the model |
| The agent asks for a secret directly, or scrapes state for one | There is no vault read API; the state volume is not mounted in the sandbox; the sandbox root filesystem is read-only with all capabilities dropped |
| A pasted secret in chat gets forwarded, stored, or logged | Secret-guard screens channel ingress and egress; a hit is refused, redacted in every store, and the user is warned to rotate |
| The agent forgets "ask first" (compaction, long context) and acts destructively | Classification and approval live in the control plane's policy store, not in the model's context; approvals are hash-bound to exact arguments and single-use |
| A runaway loop burns the budget | Governor caps (per-task / day / month) enforced by reservation before each call; the refusal is mechanical, not a request to the model |
| The operator says stop and the agent keeps going | The kill switch is a persisted control-plane flag; every authorization checks it; in-flight spend freezes at the last settled call |
| A malicious or compromised skill / MCP server | Install-time scanner gate (secrets, SSRF surfaces, injection directives, dependency risks); a do not install verdict has no override; runtime tool calls and results still cross the Shield gate |
| A bad or tampered update replaces a working agent | Signed manifests (Ed25519) verified before staging; A/B slots; canary replay of the operator's own automations before promotion; automatic rollback; append-only data stores are never rolled back |
| DNS rebinding: an allowlisted name resolves to an internal address at connect time | Resolve once per hop, pin the socket to a resolved address, verify the socket's actual peer against the resolved set; a mismatch discards the response unscreened |
| The control plane crashes mid-job and work is silently lost or double-run | Durable job store; the boot sweep settles orphaned queued/running jobs failed-loudly with receipts; Governor orphan reservations are refunded at boot so caps never wedge |
| The Shield is down when a scan is required | The operator's recorded policy decides, stated at onboarding: fail-closed (default for tool calls) blocks the surface and mints no fake receipt; fail-open proceeds flagged, never pretending a scan happened |
Out of scope — stated plainly
- Detection is not perfect. The Shield gate guarantees a check happened and left a receipt; it does not guarantee every malicious input is caught. Agent-Shield publishes its own benchmark limits; read a receipt as "no known-bad patterns detected," never as "proven safe." The architectural promises (caps, vault, approvals, updates) do not depend on the scanner being perfect — they depend on it being present, enforced, and receipted.
- A compromised host is a compromised install. If the operator's machine or Docker daemon is owned, the control plane's files — including the vault's encrypted store and its host key — are within reach. GUS's boundary is between the agent and the operator's control system, not against an adversary with host root. Standard self-host hygiene applies: disk encryption, a dedicated box or VM.
- The operator is trusted. Approvals, caps, and policy are the operator's to set. GUS makes the consequences mechanical; it does not second-guess the person who owns it. Social-engineering the operator into pressing Approve is out of scope for any approval system, including this one.
- Channel platforms. Telegram carries approvals and metadata only, because bot chats are not end-to-end encrypted — but a compromised Telegram account is a compromised approval surface. Protect it accordingly (Telegram 2FA). The kill switch also exists locally (panel, file flag), independent of Telegram.
- Skill execution. Third-party skill code is not executed in v1 (installed and scanned only), so skill-code sandboxing beyond the install gate is not yet a claim. When execution lands, it lands inside the same sandbox, behind the same gate — or it doesn't land.