Voice

Voice — Call GUS

Talk to GUS in a live call and hear the answer as it's spoken. Voice is a channel, not a bypass: a call is an ordinary job, with the same scans, caps, approvals, and receipts as typed text.

How voice works

Built in (the default, since v0.9.6). Speech-to-text is the Apple Speech framework (SFSpeechRecognizer, on-device where supported); speech-out is AVSpeechSynthesizer with macOS system voices. A call needs no speech server at all. (An earlier design sketched whisper.cpp and a bundled Kokoro — those were proposed, never shipped.)

Endpoints (optional, Settings → Voice). Either half — transcription or speech — can instead point at any OpenAI-compatible endpoint you run. A typical self-hosted setup: a faster-whisper server (for example Speaches) at http://your-server:8000/v1 with model Systran/faster-whisper-base for transcription, and Kokoro at http://your-server:8880/v1 with model kokoro for speech. At call start the endpoint is probed; if it doesn't answer, the call falls back to built-in voice for that call and says so on the call strip — your setting is never rewritten.

The call itself. The microphone is on only while a call is open. The call opens with a spoken greeting, so you hear GUS before you say a word. Your transcript is treated exactly like a typed message from that moment on. Approvals by voice are deliberately narrow: you hear the exact approval summary read back, and only an exact "confirm" or "yes, do it" runs it — a plain "yes" runs nothing. "Stop" or "GUS stop" trips the kill switch and ends the call. A call ends when you hang up, after three minutes of silence, or at sixty minutes, and every turn's stage timings (heard / thought / spoke) are kept in the call record.

Replies are spoken sentence by sentence, so GUS starts talking before the whole reply exists. Talk over a reply and the speech cuts off and listens (barge-in — it stops the talking, never a running job). Since v0.9.11, the recognizer is also closed while GUS itself is speaking, plus a short tail after — GUS never transcribes its own voice into a new turn.

The voice Shield scan

Voice has its own scan switch in Settings → Voice, on by default:

• On: spoken requests get Agent-Shield's fast scan before they become jobs (since v0.9.10) — deterministic patterns, typically a second or two per turn. Typed messages on every channel get the full semantic review; voice trades that second look for conversational speed. The verdict, the signed receipt, and every block are the scan's own either way.
• Off: spoken turns become jobs without the injection scan — faster, and meant for testing and troubleshooting. Be clear-eyed about off: nothing checks your spoken words for prompt injection before they reach the agent. Every skipped turn is recorded as scan-skipped-by-operator in the call record, so the record never pretends a scan happened.

Either way, the secret-guard still screens voice, and the Governor's caps, approvals, and vault rules for actions are untouched. You don't have to open Settings to know which mode a call is in: since v0.9.11 the call strip shows a chip — "Shield: fast scan" or a red "Shield: OFF" — read from the live session, next to the Mute button.

One Call button, one mute

There is one Call button — in the chat composer. The Mute button is big and labeled, sits in the call strip while a call is open, and turns red while you are muted. Muting stops the microphone feed instantly: nothing is transcribed, posted, or allowed to interrupt a reply, and any half-finished sentence is discarded. Unmuting resumes listening where it left off.

Testing voice before a call (v0.9.9)

• Test Connection on each endpoint works on the values in the form right now — nothing has to be saved first — and reports whether the endpoint answered, how long it took, and whether your model is in the endpoint's model list.
• Test mic and speaker checks your microphone level while you talk, plays a test sentence through the speech engine you selected, and — with built-in transcription — runs a short dictation trial with a speed verdict for your Mac: "real time on this Mac", "borderline", or "too slow on this Mac", the last with a nudge to try an endpoint instead. An older Mac gets steered in Settings, before a call, instead of finding out mid-call.
• If Dictation, Siri, or the speech-recognition permission is off, the app tells you before the call starts, with buttons that open the exact Settings panes — macOS owns those switches, so the app takes you to them rather than pretending it flipped them.

If GUS hears you but never speaks back

Check in order:

1. Is the speech-to-text model actually downloaded? curl http://your-server:8000/v1/models returning {"data":[],"object":"list"} means the server is up with an empty shelf. Pull the model: curl -X POST http://your-server:8000/v1/models/Systran/faster-whisper-base.
2. Prove the stack without GUS: say "hello GUS, can you hear me" -o /tmp/hello.wav, then POST it to /v1/audio/transcriptions. Text back means the speech server is innocent.
3. Check the voice-turns log (~/.gus/state/voice-turns.jsonl): turns with no transcript and no job mean audio never became a sentence inside GUS.
4. Speech-out check: curl http://your-server:8880/v1/models must list kokoro.

If calls are slow or choppy

• Read the per-turn stage timings in the call record (scan / model calls / queue, recorded since v0.9.8). Measure before blaming the machine: in testing, the old full semantic scan was the single biggest chunk of a turn's wait — which is why voice now uses the fast scan (above).
• Empty transcripts usually mean the recognizer never delivered a finished sentence. On older Intel Macs, self-hosted endpoints are often the better path; on Apple Silicon the built-in engine is the first-class option. The Settings speed verdict (above) tells you which side of that line your Mac is on.
• If endpoint speech fails mid-reply, the reply still reaches you as text, with the failure stated plainly.

Errors tell the truth (v0.9.11)

When a model call fails, the message names the backend and model actually configured — never a wrong vendor. And a model response that comes back empty is retried once automatically before it can fail a turn, so a single hiccup doesn't kill a call.

Full setting-by-setting detail for voice lives in Settings, explained.