15 Commits
Author SHA1 Message Date
dsql 2fa3abab63 v0.2.0: context injection + system daemon-control namespace
context injection — named reference blurbs from contexts.toml injected ahead of a
dictated instruction, read-before-send (never auto-submits):
- new contexts.py mirrors config.py: [contexts] name = "blurb"; missing file = empty
  set; names validated as simple words, looked up on a despaced/lowercased key so
  "web hooks"/"web-hooks"/"webhooks" all resolve the same block.
- grammar: context|prepare <name> <instruction> -> Action("context", (name, dictation)).
  same-utterance dictation (everything after <name> is literal, incl. "send"); bare
  context <name> injects just the blurb. one-shot targeting composes:
  [target <name>] [context <ctx>] [filler] <dictation>.
- daemon assembles blurb + (Shift+Enter soft newline | flattened separator) + dictation
  via the existing send_literal/type path, tracks the uncommitted-input buffer, and
  WAITS. config-gated by behavior.context_multiline / context_separator. unknown context
  name announces and injects nothing.

system daemon-control namespace — lands the pass-through vs control split the router was
structured for. reserved leading "system" routes to _do_system (never injects to
claude): system status (mode/target/model/contexts) and system reload [config|contexts].

live reload — voice reload + CLI claudedo reload (SIGHUP) re-read config.toml +
contexts.toml without reinitializing the loaded whisper model. customs now lists loaded
contexts. install.sh installs the contexts.toml template copy-if-absent (else .new).

keys.NEWLINE (S-Enter) added for the soft-newline assembly. wake list unchanged.

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-26 18:08:08 -04:00
dsql f177b46a4b docs: fix stale README config defaults (wake 0.65, vad 700/15)
the lower [vad]/threshold bullets still said 0.6 / 800ms / max 10; sync to the real
defaults (wake_fuzzy_threshold 0.65, silence_ms 700, max_seconds 15). CLAUDE.md and
COMPACT.md (git-ignored) corrected on disk too (model small.en, same numbers).

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-26 04:07:27 -04:00
dsql 252385fb67 feat: highlight wake phrases in magenta (startup banner + wake note)
add a magenta color; paint wake phrases magenta in the startup 'wake:' list and in
the loose-match '(wake: <phrase>)' note (the rest of that green heard line stays
green around the magenta phrase). makes the wake vocabulary visually distinct from
green heard-text and brightblue command words.

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-26 04:02:15 -04:00
dsql 97591eb24d feat: version voice command + matched-wake note on loose matches
add 'version' (prints claudedo <ver> to console; in vocab + menu). when a command's
wake phrase matched loosely (the transcript didn't contain it literally), the green
heard line appends '(wake: <phrase>)' so e.g. 'okay clouds' -> 'okay claude' is
visible. grammar.parse() now returns the matched phrase on ParsedCommand.wake (via a
new strip_wake_match; strip_wake kept as a thin wrapper).

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-26 03:59:52 -04:00
dsql 5f05a01423 feat: v0.1.4 — HELP menu, 15s cap, wake 0.65, small.en default + docs sync
commands menu now prints under a single [HELP] header with bare indented rows
(brightblue usage) instead of 15 repeated [SYSTEM] tags. raise [vad].max_seconds
10 -> 15 for long dictation. wake_fuzzy_threshold 0.6 -> 0.65 (slightly fewer false
wakes; note short spellings 'ok/okay claude' still admit some). carries the prior
small.en default, [vad].silence_ms 700, lighter (brightblue) command color, lean
injection lines, .en model variants in the validator. README/CLAUDE.md synced.

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-26 03:52:19 -04:00
dsql e84ef91e7b tune: small.en default, vad 700ms, lighter command color, lean inject lines
default model -> small.en (english-only small; better english accuracy, same ~1s
latency; .en variants added to the validator). raise [vad].silence_ms 500 -> 700
(500 cut off too early). command words now brightblue (lighter/cyan-ish) instead of
dark blue. drop the redundant target from injection lines — the [session] prefix
already names it, so e.g. '[claude-testing] typed ...' not '... sticky claude-testing
-> typed ...'.

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-26 03:41:46 -04:00
dsql 2cbbabfaa1 feat: unbounded backspace + blue command words in console
backspace now sends exactly n BSpace with no boundary cap (buffer floored at 0 so a
later erase stays correct); erase remains bound to the uncommitted-input buffer. add
a blue color and Console.paint(); paint the command word blue on SYSTEM lines
(list/set/unset/mode -> ...) so the action stands out.

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-26 03:11:42 -04:00
dsql 4357b14fad perf: default back to small model; show per-command STT latency
medium added ~3s/command lag (measured ~1.2s small vs ~3s medium on a 7950X3D), so
default model -> small; lean on initial_prompt + lenient wake for the coined word.
every heard line now shows STT latency as (<ms>/<audio>s) — always on, not just
print_heard — so a model change's cost is visible. snappier vad (silence_ms 500)
from the prior commit stands.

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-26 02:57:52 -04:00
dsql 8e20b7eb0b feat: commands/customs menu, green heard-echo, snappier VAD
add voice 'commands' (alias help/menu) printing the command menu and 'customs'
(alias custom) stubbed for v0.2.0. echo every recognized command as a green
'heard "..." -> ACTION' line before acting, so you see what landed; the result line
then reports target + keystrokes. lower [vad].silence_ms default 800 -> 500 for a
snappier endpoint after you stop talking.

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-26 02:32:28 -04:00
dsql 4abdfd56bc feat: start skips the mic check by default; --check to opt in
invert the pre-listen mic check — default is no check (just start listening); pass
'claudedo start --check' to run it. replaces the old --skip-audio-check flag.

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-26 02:25:20 -04:00
dsql e6dadab143 feat: debug/echo command — print the spoken phrase to the console
'<wake> debug <text>' (alias echo) echoes what you said to the console as
[VOICE] debug: "..." and injects nothing — a no-target test command for checking
wake + STT transcription. added to the STT vocab so it's biased for.

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-26 02:22:37 -04:00
dsql 5064f912a4 fix: install.sh installs config.toml to ~/.config/claudedo
the daemon's config lookup falls through to ./config.toml only, so without a copy in
the standard dir it was repo-cwd-only. install config.toml to ~/.config/claudedo/ —
copy if absent, else write config.toml.new beside the user's edited copy (never
clobber). also gitignore COMPACT.md (handoff doc kept on disk, untracked).

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-26 01:53:25 -04:00
dsql a51c2fbdd4 feat: v0.1.3 STT tuning — medium model, initial_prompt bias, split thresholds, VAD config
default stt.model -> medium (biggest accuracy gain for the coined wake word;
small/large-v3 documented alternatives). seed faster-whisper with an initial_prompt
derived from the configured wake phrases + command vocabulary (grammar.vocabulary /
initial_prompt, one source — command synonyms now live in named _*_VERBS tuples).

split the single fuzzy threshold into wake_fuzzy_threshold (0.6, lenient — a false
wake is cheap) and command_fuzzy_threshold (0.8, tight — a false command fires the
wrong action); grammar.parse() takes both. add a [vad] config section (silence_ms,
max_seconds) for the existing Alexa-style record-until-pause endpointing, which
captures a command whole and lets the trailing pause separate it from following
chatter (that chatter is a separate capture the wake gate discards). bump to 0.1.3.

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-26 01:41:48 -04:00
dsql bd6597352a feat: 'add [a] space' / 'insert <n> spaces' phrasing; drop 'claude due' wake
map 'add a space'/'add space'/'insert two spaces' to the space command (count read
from either side of the noun). remove 'claude due' from the default wake list (it
double-rendered with 'claude do' and wasn't wanted). docs synced.

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-26 01:27:16 -04:00
dsql 08bbe3ce58 docs: sync README with v0.1.2 (wake list, editing cmds, auto_target, console)
reconcile README with the shipped code and with CLAUDE.md: full 6-phrase wake list
(claudedo/claude do/claude due/hey claude/ok claude/okay claude) with the Whisper
rationale; space/backspace/erase in the grammar + flow; colored prefixed console
output description; fix the auto_target contradiction (default false = require
set/target, not auto-pick); drop the stale 'backgroundable'.

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-26 01:22:00 -04:00
15 changed files with 754 additions and 126 deletions
+1
View File
@@ -1,4 +1,5 @@
CLAUDE.md
COMPACT.md
__pycache__/
*.pyc
+104 -31
View File
@@ -11,7 +11,7 @@ hands-free while another window (a game) is focused.
It exists because Claude Code's native `/voice` is hardcoded-blocked in WSL (it
assumes WSL has no audio). Modern WSL2 + WSLg *does* have working mic input via
PulseAudio/RDP. `claudedo` captures the mic itself, transcribes on-device, and drives
Claude Code over tmux — fully local, private, backgroundable.
Claude Code over tmux — fully local and private. You run it in a terminal you watch.
## How it works
@@ -20,9 +20,11 @@ mic (WSLg/PulseAudio RDPSource)
-> sounddevice capture
-> faster-whisper (local STT, on-device)
-> wake gate: utterance must start with a wake phrase, else DISCARD locally
-> grammar match (yes/no/one..four/approve/deny/send/type/mode/set/target/cancel)
-> resolve target session (~/.claude-active)
-> grammar match (yes/no/one..four/approve/deny/send/type/space/backspace/erase/
mode/set/target/unset/list/context/reload/system/cancel)
-> resolve target session (one-shot > sticky ~/.claude-active > auto/none)
-> tmux send-keys -t <session> "<keys>"
-> log the action to the watched terminal ([session]/[SYSTEM]/[VOICE], colored)
```
**Privacy by construction.** STT runs on-device. In listen mode, any speech that
@@ -62,18 +64,22 @@ claudedo test-audio
## Usage
**Run it in a terminal you watch — that's the product.** You launch `claudedo
start`, it does a quick mic check, then drops into a visible listen loop that prints
`heard → matched → sent` for every utterance. That terminal is your
recognition/action console; you attach to the `claude-<name>` session in another pane
to watch the keystrokes land. There is no backgrounding/daemon mode — the whole point
is the console you read.
start` and it drops into a visible listen loop (pass `--check` to run a mic check
first). Each utterance prints a timestamped, colored line — `HH:MM:SS [claude-libs]
heard "…" →
typed 'fix'` (green for injected, red for drops, `[SYSTEM]`/`[VOICE]` for state and
recognition). That terminal is your recognition/action console; you attach to the
`claude-<name>` session in another pane to watch the keystrokes land. It runs in the
foreground by design — the console is the point — though `claudedo stop` can signal a
stray instance.
```bash
claudedo start # mic-check, then the visible listen loop (listen mode default)
claudedo start # the visible listen loop (listen mode default; no mic check)
claudedo start --check # run a mic check before listening
claudedo start --mode ptt # push-to-talk instead (desk-only — see Modes)
claudedo start --skip-audio-check # skip the pre-listen mic check
claudedo status # running? mode? target session?
claudedo stop # stop a running daemon
claudedo reload # reload config.toml + contexts.toml in a running daemon
claudedo set <name> # set the sticky target -> claude-<name> (alias: switch)
claudedo unset # clear the sticky target
claudedo list # list running claude-* sessions
@@ -97,9 +103,14 @@ Switch at runtime by voice: "claudedo mode listen" / "claudedo mode ptt".
## Command grammar
Wake phrases (listen mode), fuzzy-matched: **"claudedo"**, **"hey claude"**.
"claudedo" is a coined word, so the matcher is lenient (accepts "claude do",
"clauddo", "cloud do", …). In PTT mode the wake phrase is optional.
Wake phrases (listen mode), fuzzy-matched. The default list is **"claudedo"**,
**"claude do"**, **"hey claude"**, **"ok claude"**, **"okay claude"** — Whisper has
no token for the coined word "claudedo" and renders it as real words ("claude do"),
so that spelling is listed explicitly. Matching is lenient (case/space-insensitive).
Add the spellings you actually see (turn on `print_heard` to find them). In PTT mode
the wake phrase is optional. When a command's wake phrase matched loosely (e.g. you
said "okay clouds"), the heard line notes which phrase it assumed —
`heard "okay clouds list" -> LIST (wake: okay claude)`.
| Say | Does |
|---|---|
@@ -108,21 +119,31 @@ Wake phrases (listen mode), fuzzy-matched: **"claudedo"**, **"hey claude"**.
| `approve` / `deny` | allow / deny a permission prompt |
| `send` / `enter` | submit (Enter) |
| `type <phrase>` | insert literal text, **no** submit (read-before-send; say "send") |
| `space [<n>]` | insert n spaces (default 1) |
| `space [<n>]` (also `add [a] space`, `insert <n> spaces`) | insert n spaces (default 1) |
| `backspace [<n>]` (alias `delete`) | delete n chars (default 1), capped at the last submit boundary |
| `erase` (alias `clear`/`wipe`) | delete everything typed since the last submit/boundary |
| `debug <text>` (alias `echo`) | just print what you said to the console (test wake/STT; injects nothing) |
| `mode ptt` / `mode listen` | switch input mode |
| `set <name>` (alias `sticky`/`switch`) | set the **sticky** target → `claude-<name>` (persists) |
| `target <name> <command>` | **one-shot** override: run that command on `claude-<name>` for this utterance only; sticky default unchanged |
| `unset` (alias `unsticky`) | clear the sticky target |
| `list` | list running `claude-*` sessions to the daemon console |
| `context <name> <instruction>` (alias `prepare`) | inject a `contexts.toml` blurb as a preamble + the dictated instruction, then **wait** (no submit — say "send") |
| `reload` | re-read `config.toml` + `contexts.toml` live (no daemon restart, model stays loaded) |
| `system status` | print mode / target / model / context count to the console (daemon-control; never injects) |
| `system reload [config\|contexts]` | reload one or both config files |
| `commands` (alias `help`/`menu`) | print the voice-command menu to the console |
| `customs` (alias `custom`) | list the loaded context names |
| `version` | print the claudedo version to the console |
| `cancel` / `escape` | back out of a prompt |
Optional filler (`select` / `use` / `choose`) may precede any command and is ignored:
`select yes` and `use yes` behave like `yes`. (`select 1` is still the select command.)
When no sticky target is set, a bare command auto-targets the **only** running
`claude-*` session; if several are running it does nothing and asks you to `set` one.
When no sticky target is set, a bare command does nothing and asks you to `set` one
(the default). Set `auto_target = true` to instead auto-use the single running
`claude-*` session when there's exactly one; with several running it always does
nothing and asks you to `set` one.
Number words are normalized to digits before matching ("one"/"won" → 1).
@@ -135,9 +156,9 @@ A `target <name>` voice command is a **one-shot** that does NOT touch the sticky
default — it routes a single command and the next bare command reverts to sticky.
Resolution order (one place — `target.resolve()`): one-shot if present →
sticky if set and the session exists → else the only running `claude-*` session →
else (zero or several) do nothing and say so. It never guesses, and never injects
into a nonexistent session.
sticky if set and the session exists → else, only if `auto_target = true`, the single
running `claude-*` session → else (default, or zero/several sessions) do nothing and
say so. It never guesses, and never injects into a nonexistent session.
Every name maps to `claude-<name>` through one helper (`target.session_name()`), and
the cc kit mirrors it exactly — so `cc libs` (shell) and `set libs` (voice) refer
@@ -156,6 +177,40 @@ cck <name> # kill claude-<name>
cckl # kill all claude-* sessions
```
## Contexts (named reference blurbs)
`contexts.toml` holds named reference snippets you can inject ahead of a dictated
instruction with the **`context <name> <instruction>`** voice command (alias
`prepare`). It lives next to `config.toml`
(`$CLAUDEDO_CONTEXTS``~/.config/claudedo/contexts.toml``./contexts.toml`); a
missing file just means no contexts (the feature is opt-in).
```toml
[contexts]
webhooks = "discord webhooks — test: <url> (safe to spam), live: <url> (real, careful)"
testing = "use the test/staging resources only, never touch prod"
```
Saying `context webhooks send a test message` injects the `webhooks` blurb as a
preamble, then the dictated instruction, and **waits** — nothing is auto-submitted. You
say `send` to submit (**read-before-send**; Claude's own permission prompt is the
backstop for anything consequential). A bare `context webhooks` injects just the blurb.
One context per command (no stacking yet); an unknown name announces and injects
nothing.
Names are **spoken and fuzzy-matched**, so keep them simple and distinct — they're
looked up on a despaced/lowercased key, so `web hooks` / `web-hooks` / `webhooks` all
resolve the same block. Assembly is config-gated: `behavior.context_multiline` (default
`true`) puts the blurb and instruction on separate lines via a Shift+Enter soft newline;
set it `false` to flatten onto one line with `context_separator` (default `" — "`) if
Shift+Enter is unreliable in your terminal.
Edit `contexts.toml`, then say **`reload`** (or run `claudedo reload`) — it re-reads
`config.toml` and `contexts.toml` live without restarting the daemon or reloading the
Whisper model. The **`system`** namespace gives daemon-control by voice without touching
Claude: `system status` (mode / target / model / context count) and `system reload
[config|contexts]`.
## The confirmed Claude Code keymap
The keystrokes in [`keys.py`](src/claudedo/keys.py) were confirmed **empirically**
@@ -174,19 +229,37 @@ If Claude Code changes its prompt UI, re-confirm against a live session and upda
## Config
Everything tunable lives in [`config.toml`](config.toml): wake phrases, mode + PTT
key, Whisper model/language/device, audio segmentation thresholds, and `[behavior]`
(`type_autosend`, `filler_words`, `auto_target`, `print_heard`). The default model is
`small`; bump to `medium` if the coined wake word is recognized poorly. `claudedo -c
<path> ...` points at a specific config; otherwise it searches `$CLAUDEDO_CONFIG`,
`~/.config/claudedo/config.toml`, then `./config.toml`.
key, Whisper model/language/device, `[vad]` endpointing, and `[behavior]`
(`type_autosend`, fuzzy thresholds, `filler_words`, `auto_target`, `print_heard`).
The default model is **`small.en`** (the English-only small model — ~1s/command on a
strong CPU, more accurate on English than multilingual `small` at the same speed);
`medium`/`medium.en` are more accurate but ~3× slower (noticeable lag), `base.en` is
snappier/less accurate, `large-v3` most accurate/slowest. Every `heard` line shows the
STT latency as `(<ms>/<audio>s)` so you can see what a model change costs. VAD
endpointing ends a capture after `[vad].silence_ms` (700) of trailing silence, capped
at `max_seconds` (15). `claudedo -c <path> ...` points at a specific config; otherwise
it searches
`$CLAUDEDO_CONFIG`, `~/.config/claudedo/config.toml`, then `./config.toml`.
- **`auto_target`** (default `false`): with no sticky target set and exactly one
`claude-*` session running, `false` makes a bare command do nothing and ask you to
`set` one; `true` auto-targets that single session.
- **`print_heard`** (default `false`, debug): prints non-wake transcripts to the
console so you can see how Whisper renders your wake word. Turn it on to debug
detection, then off. Whisper has no token for "claudedo" — it commonly emits
"claude do" or "claude due", both of which are in the default wake list.
- **STT biasing.** The transcriber is seeded with an `initial_prompt` built from the
configured wake phrases + command vocabulary (one source — `grammar.vocabulary()`),
so Whisper is conditioned to expect "claudedo" and the command words.
- **Split fuzzy thresholds.** `wake_fuzzy_threshold` (default `0.65`, lenient) vs
`command_fuzzy_threshold` (default `0.8`, tight). The asymmetry is deliberate: a
false *wake* is cheap (it wakes, finds no command, does nothing), but a false
*command* fires the wrong action. Prefer expanding command synonyms over loosening
the command threshold.
- **`[vad]` endpointing.** Capture starts on speech and ends after `silence_ms`
(default 700) of trailing silence — Alexa-style record-until-pause — capped at
`max_seconds` (default 15). The pause both ends a command and separates it from
following chatter (the chatter is a separate capture the wake gate discards).
- **`auto_target`** (default `false`): with no sticky target and one session running,
`false` does nothing and asks you to `set`; `true` auto-uses that session.
- **`print_heard`** (default `false`, debug): prints non-wake transcripts so you can
see how Whisper renders your wake word, then tune the wake list/threshold.
- **`context_multiline`** (default `true`) / **`context_separator`** (default `" — "`):
how the `context` command assembles the blurb and instruction — a Shift+Enter soft
newline between them, or (when `false`) flattened onto one line with the separator.
## Requirements
+34 -12
View File
@@ -5,7 +5,7 @@
# wake phrases for listen mode. fuzzy-matched: case/space-insensitive, lenient on
# the coined word "claudedo" (whisper renders it inconsistently). number words are
# normalized to digits before command matching.
phrases = ["claudedo", "claude do", "claude due", "hey claude", "ok claude", "okay claude"]
phrases = ["claudedo", "claude do", "hey claude", "ok claude", "okay claude"]
[input]
# "listen" (default): continuous capture; only acts on utterances that start with a
@@ -21,10 +21,12 @@ mode = "listen"
ptt_key = "space"
[stt]
# faster-whisper model size. "small" is a good accuracy/latency balance for the
# short command grammar (~sub-second per chunk on a strong cpu). if the coined wake
# word "claudedo" is recognized poorly, bump to "medium" (slower per chunk).
model = "small"
# faster-whisper model size. "small.en" is the default — the English-only small model
# (~1s/command on a strong cpu, more accurate on english than multilingual "small" at
# the same speed). "medium"/"medium.en" are more accurate but ~3x slower (noticeable
# lag); "large-v3" is most accurate and slowest. drop to "base.en" for max snappiness
# (less accurate). bump only if recognition is poor.
model = "small.en"
language = "en"
# mic device: "auto", or a sounddevice device index (integer) / substring of a
# device name. run `claudedo test-audio` to list devices.
@@ -36,21 +38,31 @@ compute = "auto"
# capture parameters. 16 kHz mono is what whisper expects.
samplerate = 16000
channels = 1
# listen-mode silence segmentation: an utterance ends after this many seconds below
# the rms threshold. keeps latency low without streaming.
# rms energy below this counts as silence (the VAD onset/endpoint floor).
silence_threshold = 0.012
silence_duration = 0.8
# ignore utterances shorter than this (clicks, coughs).
min_utterance = 0.3
# hard cap on a single utterance so a stuck stream can't grow unbounded.
max_utterance = 15.0
[vad]
# Alexa-style record-until-pause endpointing (listen mode). capture starts on speech
# onset and ends after this much trailing silence — the natural end of an utterance.
# a real pause both ends the command AND separates it from following chatter (the
# chatter becomes a separate capture that the wake gate then discards).
silence_ms = 700
# hard cap so continuous noise can't record forever (also the ceiling for a long
# dictated `type` phrase).
max_seconds = 15.0
[behavior]
# dictation never auto-submits: "type <phrase>" inserts literal text only; you say
# "send" separately to submit (read-before-send).
type_autosend = false
# fuzzy match ratio (0..1) required to accept a wake phrase / command token.
match_threshold = 0.8
# fuzzy match ratios (0..1). the asymmetry is deliberate: a false WAKE is cheap (it
# wakes, finds no command, does nothing), so wake is lenient; a false COMMAND fires
# the WRONG action, so commands stay tight. lower = more lenient = more matches.
# prefer expanding command synonyms over loosening command_fuzzy_threshold.
wake_fuzzy_threshold = 0.65
command_fuzzy_threshold = 0.8
# optional filler words that may precede a command and are ignored for matching:
# "select yes" / "use yes" behave like "yes". (a filler word followed by a digit is
# the select command, e.g. "select 1", and is not dropped.)
@@ -66,3 +78,13 @@ auto_target = false
# how Whisper renders your wake word, then turn it OFF. default false: non-wake speech
# is discarded without ever printing the transcript.
print_heard = false
# how the `context <name> <dictation>` command assembles the blurb + instruction.
# true (default): blurb, a soft newline (Shift+Enter — needs the extended-keys tmux
# settings install.sh appends), then the instruction. if Shift+Enter is at all flaky
# in your terminal (it submits or does nothing), set false to flatten onto one line
# with context_separator between blurb and instruction — the blank line is cosmetic,
# not worth a submit risk. either way the assembled text is NEVER auto-submitted.
context_multiline = true
# separator inserted between blurb and instruction when context_multiline = false.
context_separator = " — "
+18
View File
@@ -0,0 +1,18 @@
# claudedo contexts — named reference blurbs you can inject ahead of a dictated
# instruction with the `context <name> <instruction>` voice command (alias `prepare`).
#
# the named blurb is injected as a preamble, then your dictated instruction, and the
# daemon WAITS — nothing is auto-submitted. you say "send" to submit (read-before-send;
# claude's own permission prompt is the backstop for anything consequential).
#
# names are SPOKEN and fuzzy-matched, so keep them simple, distinct, single words
# (a-z, 0-9; spaces/hyphens/underscores are stripped for matching, so "web hooks",
# "web-hooks" and "webhooks" all resolve the same block). values are free-form text.
#
# edit this file, then say "reload" (or run `claudedo reload`) — no daemon restart,
# the whisper model is not reloaded.
[contexts]
webhooks = "discord webhooks — test: <url> (safe to spam), live: <url> (real, careful)"
testing = "use the test/staging resources only, never touch prod"
discord = "discord.py 2.x; bot token in .env as BOT_TOKEN; guild id 12345"
+24
View File
@@ -94,6 +94,30 @@ mkdir -p "$CONF_DIR"
install -m 0644 "$REPO_DIR/shell/cc.sh" "$CONF_DIR/cc.sh"
echo " wrote $CONF_DIR/cc.sh"
# install config.toml to the standard location so the daemon finds it from any dir.
# never clobber an edited user config: copy only if absent, else drop a .new to diff.
if [ ! -f "$CONF_DIR/config.toml" ]; then
install -m 0644 "$REPO_DIR/config.toml" "$CONF_DIR/config.toml"
echo " wrote $CONF_DIR/config.toml"
elif ! cmp -s "$REPO_DIR/config.toml" "$CONF_DIR/config.toml"; then
install -m 0644 "$REPO_DIR/config.toml" "$CONF_DIR/config.toml.new"
echo " kept your $CONF_DIR/config.toml; new default written to config.toml.new (diff to merge)"
else
echo " $CONF_DIR/config.toml already current"
fi
# install the contexts.toml template (named blurbs for the `context` voice command).
# same policy: copy only if absent, else drop a .new — never clobber edited contexts.
if [ ! -f "$CONF_DIR/contexts.toml" ]; then
install -m 0644 "$REPO_DIR/contexts.toml" "$CONF_DIR/contexts.toml"
echo " wrote $CONF_DIR/contexts.toml"
elif ! cmp -s "$REPO_DIR/contexts.toml" "$CONF_DIR/contexts.toml"; then
install -m 0644 "$REPO_DIR/contexts.toml" "$CONF_DIR/contexts.toml.new"
echo " kept your $CONF_DIR/contexts.toml; new default written to contexts.toml.new (diff to merge)"
else
echo " $CONF_DIR/contexts.toml already current"
fi
# wire EVERY rc that exists (the user may have both zsh and bash).
wired_any=0
for RC in "$HOME/.zshrc" "$HOME/.bashrc"; do
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "claudedo"
version = "0.1.2"
version = "0.2.0"
description = "voice-control daemon for claude code (local STT -> tmux send-keys)"
readme = "README.md"
requires-python = ">=3.10"
+1 -1
View File
@@ -1,3 +1,3 @@
"""claudedo — voice-control daemon for claude code (local STT -> tmux send-keys)"""
__version__ = "0.1.2"
__version__ = "0.2.0"
+14 -4
View File
@@ -33,12 +33,12 @@ def cmd_start(args: argparse.Namespace) -> int:
config = _load_or_die(args.config)
if args.mode:
config.mode = args.mode
if not args.skip_audio_check:
if args.check:
print("checking mic before listening (speak briefly) ...")
peak = _probe_mic(config, seconds=2.0, verbose=False)
if peak is None or peak < 0.02:
print("mic check failed — no usable input.", file=sys.stderr)
print("run `claudedo test-audio` to debug; or `claudedo start --skip-audio-check`",
print("run `claudedo test-audio` to debug, or `claudedo start` to skip the check",
file=sys.stderr)
return 1
print(f"mic OK (peak {peak:.3f}).")
@@ -97,6 +97,14 @@ def cmd_stop(_args: argparse.Namespace) -> int:
return 1
def cmd_reload(_args: argparse.Namespace) -> int:
if daemon.reload_running():
print("signalled claudedo to reload config + contexts")
return 0
print("claudedo is not running")
return 1
def cmd_status(_args: argparse.Namespace) -> int:
pid = daemon.read_pid()
if pid is None:
@@ -217,11 +225,13 @@ def build_parser() -> argparse.ArgumentParser:
sp = sub.add_parser("start", help="run the daemon (foreground)")
sp.add_argument("--mode", choices=("listen", "ptt"), help="override input mode")
sp.add_argument("--skip-audio-check", action="store_true",
help="skip the pre-listen mic check")
sp.add_argument("--check", action="store_true",
help="run a mic check before listening (off by default)")
sp.set_defaults(func=cmd_start)
sub.add_parser("stop", help="stop a running daemon").set_defaults(func=cmd_stop)
sub.add_parser("reload", help="reload config + contexts in a running daemon"
).set_defaults(func=cmd_reload)
sub.add_parser("status", help="show daemon status").set_defaults(func=cmd_status)
sub.add_parser("test-audio", help="verify the mic capture path").set_defaults(func=cmd_test_audio)
sub.add_parser("install", help="re-run the bootstrap (install.sh)").set_defaults(func=cmd_install)
+24 -10
View File
@@ -17,7 +17,10 @@ except ModuleNotFoundError:
log = logging.getLogger(__name__)
_VALID_MODES = ("listen", "ptt")
_VALID_MODELS = ("tiny", "base", "small", "medium", "large-v2", "large-v3")
_VALID_MODELS = (
"tiny", "base", "small", "medium", "large-v1", "large-v2", "large-v3",
"tiny.en", "base.en", "small.en", "medium.en",
)
DEFAULT_CONFIG_PATHS = (
Path(os.environ.get("CLAUDEDO_CONFIG", "")) if os.environ.get("CLAUDEDO_CONFIG") else None,
@@ -44,14 +47,17 @@ class Config:
samplerate: int
channels: int
silence_threshold: float
silence_duration: float
vad_silence_ms: int
vad_max_seconds: float
min_utterance: float
max_utterance: float
type_autosend: bool
match_threshold: float
wake_fuzzy_threshold: float
command_fuzzy_threshold: float
filler_words: tuple[str, ...]
auto_target: bool
print_heard: bool
context_multiline: bool
context_separator: str
source_path: Path | None = field(default=None)
@@ -98,7 +104,7 @@ def load_config(explicit: str | os.PathLike | None = None) -> Config:
if mode not in _VALID_MODES:
raise ConfigError(f"[input].mode must be one of {_VALID_MODES}, got {mode!r}")
model = _require(raw, "stt", "model", (str,), "small")
model = _require(raw, "stt", "model", (str,), "small.en")
if model not in _VALID_MODELS:
log.warning("unknown stt model %r — passing through to faster-whisper", model)
@@ -113,19 +119,27 @@ def load_config(explicit: str | os.PathLike | None = None) -> Config:
samplerate=int(_require(raw, "audio", "samplerate", (int,), 16000)),
channels=int(_require(raw, "audio", "channels", (int,), 1)),
silence_threshold=float(_require(raw, "audio", "silence_threshold", (int, float), 0.012)),
silence_duration=float(_require(raw, "audio", "silence_duration", (int, float), 0.8)),
vad_silence_ms=int(_require(raw, "vad", "silence_ms", (int,), 700)),
vad_max_seconds=float(_require(raw, "vad", "max_seconds", (int, float), 15.0)),
min_utterance=float(_require(raw, "audio", "min_utterance", (int, float), 0.3)),
max_utterance=float(_require(raw, "audio", "max_utterance", (int, float), 15.0)),
type_autosend=bool(_require(raw, "behavior", "type_autosend", (bool,), False)),
match_threshold=float(_require(raw, "behavior", "match_threshold", (int, float), 0.8)),
wake_fuzzy_threshold=float(_require(raw, "behavior", "wake_fuzzy_threshold", (int, float), 0.65)),
command_fuzzy_threshold=float(_require(raw, "behavior", "command_fuzzy_threshold",
(int, float), 0.8)),
filler_words=tuple(_require(raw, "behavior", "filler_words", (list,),
["select", "use", "choose"])),
auto_target=bool(_require(raw, "behavior", "auto_target", (bool,), False)),
print_heard=bool(_require(raw, "behavior", "print_heard", (bool,), False)),
context_multiline=bool(_require(raw, "behavior", "context_multiline", (bool,), True)),
context_separator=str(_require(raw, "behavior", "context_separator", (str,), "")),
source_path=path,
)
if not 0.0 < cfg.match_threshold <= 1.0:
raise ConfigError("[behavior].match_threshold must be in (0, 1]")
for label, val in (("wake_fuzzy_threshold", cfg.wake_fuzzy_threshold),
("command_fuzzy_threshold", cfg.command_fuzzy_threshold)):
if not 0.0 < val <= 1.0:
raise ConfigError(f"[behavior].{label} must be in (0, 1]")
if cfg.vad_silence_ms <= 0 or cfg.vad_max_seconds <= 0:
raise ConfigError("[vad].silence_ms and max_seconds must be positive")
if cfg.samplerate <= 0 or cfg.channels <= 0:
raise ConfigError("[audio].samplerate and channels must be positive")
return cfg
+14
View File
@@ -18,12 +18,16 @@ _COLORS = {
"red": "\033[31m",
"yellow": "\033[33m",
"cyan": "\033[36m",
"blue": "\033[34m",
"brightblue": "\033[94m",
"magenta": "\033[35m",
"dim": "\033[2m",
"bold": "\033[1m",
}
SYSTEM = "SYSTEM"
VOICE = "VOICE"
HELP = "HELP"
class Console:
@@ -45,7 +49,17 @@ class Console:
return text
return f"{_COLORS[color]}{text}{RESET}"
def paint(self, text: str, color: str | None) -> str:
"""public colorizer for pre-coloring a fragment of a message (e.g. a command
word) before passing it to emit() with color=None"""
return self._paint(text, color)
def emit(self, prefix: str, message: str, color: str | None = None) -> None:
"""print one line: ``HH:MM:SS [prefix] message`` (message optionally colored)"""
line = f"{self._stamp()} {self._paint(f'[{prefix}]', 'dim')} {self._paint(message, color)}"
print(line, file=self.stream, flush=True)
def line(self, message: str, color: str | None = None) -> None:
"""print a bare continuation line (no timestamp/prefix) — for multi-row blocks
like the help menu, indented under a preceding header"""
print(self._paint(message, color), file=self.stream, flush=True)
+108
View File
@@ -0,0 +1,108 @@
"""load named context blocks from contexts.toml into a typed lookup.
contexts are user-edited reference blurbs (claude.md-style snippets) keyed by simple
spoken names. the ``context``/``prepare`` voice command injects a named blurb ahead of
a dictated instruction (read-before-send: never auto-submitted). mirrors config.py's
load/validate pattern; a missing file is an empty set, not an error.
"""
from __future__ import annotations
import logging
import os
import re
from dataclasses import dataclass, field
from pathlib import Path
try:
import tomllib as _toml
except ModuleNotFoundError:
import tomli as _toml
log = logging.getLogger(__name__)
_NAME_RE = re.compile(r"^[a-z0-9][a-z0-9 _-]*$")
DEFAULT_CONTEXTS_PATHS = (
Path(os.environ.get("CLAUDEDO_CONTEXTS", "")) if os.environ.get("CLAUDEDO_CONTEXTS") else None,
Path.home() / ".config" / "claudedo" / "contexts.toml",
Path.cwd() / "contexts.toml",
)
class ContextsError(Exception):
"""raised on an unparseable or invalid contexts.toml"""
@dataclass
class Contexts:
"""validated named context blocks (name -> blurb), normalized for spoken lookup"""
blocks: dict[str, str] = field(default_factory=dict)
source_path: Path | None = field(default=None)
def __len__(self) -> int:
return len(self.blocks)
def names(self) -> list[str]:
"""the context names, sorted (for status / listing)"""
return sorted(self.blocks)
def get(self, name: str) -> str | None:
"""look up a blurb by its normalized (lowercased, despaced) name, or None.
names are matched on a lowercase, space/underscore/hyphen-stripped key so a
spoken "web hooks" resolves the configured ``webhooks``/``web-hooks`` block.
"""
return self.blocks.get(_key(name))
def _key(name: str) -> str:
return re.sub(r"[ _-]+", "", name.strip().lower())
def find_contexts_path(explicit: str | os.PathLike | None = None) -> Path | None:
"""resolve the contexts.toml path, or None if no file exists (not an error)"""
candidates: list[Path] = []
if explicit:
candidates.append(Path(explicit))
candidates.extend(p for p in DEFAULT_CONTEXTS_PATHS if p)
for path in candidates:
if path.is_file():
return path
return None
def load_contexts(explicit: str | os.PathLike | None = None) -> Contexts:
"""load contexts.toml from the first existing default path (or an explicit one).
a missing file yields an empty Contexts (the feature is opt-in). names must be
simple words (matchable) and values must be non-empty strings; a bad entry raises
ContextsError so the user sees a clear message rather than a silent drop.
"""
path = find_contexts_path(explicit)
if path is None:
return Contexts(blocks={}, source_path=None)
try:
with open(path, "rb") as fh:
raw = _toml.load(fh)
except _toml.TOMLDecodeError as exc:
raise ContextsError(f"could not parse {path}: {exc}") from exc
table = raw.get("contexts", {})
if not isinstance(table, dict):
raise ContextsError("[contexts] must be a table of name = \"blurb\" entries")
blocks: dict[str, str] = {}
for name, value in table.items():
if not isinstance(name, str) or not _NAME_RE.match(name.lower()):
raise ContextsError(f"context name {name!r} must be simple words (a-z, 0-9, space/-/_)")
if not isinstance(value, str) or not value.strip():
raise ContextsError(f"context {name!r} must be a non-empty string")
key = _key(name)
if key in blocks:
raise ContextsError(f"context {name!r} collides with another name on the spoken key {key!r}")
blocks[key] = value.strip()
return Contexts(blocks=blocks, source_path=path)
+209 -32
View File
@@ -16,9 +16,10 @@ import sys
import time
from pathlib import Path
from . import audio, grammar, inject, target
from .config import Config
from .console import SYSTEM, VOICE, Console
from . import __version__, audio, grammar, inject, keys, target
from .config import Config, ConfigError, load_config
from .console import HELP, SYSTEM, VOICE, Console
from .contexts import Contexts, ContextsError, load_contexts
from .stt import Transcriber
log = logging.getLogger(__name__)
@@ -76,6 +77,16 @@ def stop_running() -> bool:
return True
def reload_running() -> bool:
"""signal a running daemon (SIGHUP) to reload config + contexts. returns whether
one was found. no-op on platforms without SIGHUP."""
pid = read_pid()
if pid is None or not hasattr(signal, "SIGHUP"):
return False
os.kill(pid, signal.SIGHUP)
return True
class _PTTKey:
"""desk-only push-to-talk: 'held' while the configured key is down in the
daemon's own terminal. there is deliberately NO global hotkey — a system-wide
@@ -112,20 +123,34 @@ class Daemon:
self.config = config
self.mode = config.mode
self._stop = False
self._reload_pending = False
self._transcriber: Transcriber | None = None
self._device: int | None = None
self._ptt = _PTTKey()
self._pending: dict[str, int] = {}
self._console = Console()
self._contexts = Contexts()
self._last_stt_ms = 0.0
self._last_audio_s = 0.0
def _install_signals(self) -> None:
signal.signal(signal.SIGTERM, self._on_signal)
signal.signal(signal.SIGINT, self._on_signal)
if hasattr(signal, "SIGHUP"):
signal.signal(signal.SIGHUP, self._on_reload_signal)
def _on_signal(self, _signum, _frame) -> None:
log.info("stop requested")
self._stop = True
def _on_reload_signal(self, _signum, _frame) -> None:
"""SIGHUP from `claudedo reload` -> reload both config files on the next tick.
the actual reload runs in the loop (not the handler) so it never races a
capture/transcribe; the handler only sets the flag.
"""
self._reload_pending = True
def stopped(self) -> bool:
return self._stop
@@ -136,12 +161,23 @@ class Daemon:
model=cfg.stt_model, language=cfg.stt_language,
device=cfg.stt_compute if cfg.stt_compute in ("cpu", "cuda") else "auto",
compute_type="auto",
initial_prompt=grammar.initial_prompt(cfg.wake_phrases),
)
self._load_contexts()
if audio.warm_up(cfg.samplerate, cfg.channels, self._device):
log.info("mic warmed up (source live)")
else:
log.warning("mic warm-up saw only silence — check mic permission / RDPSource")
def _load_contexts(self) -> None:
"""(re)load contexts.toml, leaving the loaded model untouched. a parse error is
logged and leaves the previous set in place rather than crashing the loop."""
try:
self._contexts = load_contexts()
except ContextsError as exc:
log.warning("contexts.toml invalid, keeping previous set: %s", exc)
self._console.emit(SYSTEM, f"contexts.toml error (kept previous): {exc}", "red")
def _capture(self):
cfg = self.config
if self.mode == "ptt":
@@ -151,46 +187,89 @@ class Daemon:
return audio.record_while(
cfg.samplerate, cfg.channels, self._device,
held=lambda: not self._ptt.wait_press(self.stopped),
max_utterance=cfg.max_utterance, min_utterance=cfg.min_utterance,
max_utterance=cfg.vad_max_seconds, min_utterance=cfg.min_utterance,
)
return audio.record_until_silence(
cfg.samplerate, cfg.channels, self._device,
silence_threshold=cfg.silence_threshold, silence_duration=cfg.silence_duration,
min_utterance=cfg.min_utterance, max_utterance=cfg.max_utterance,
silence_threshold=cfg.silence_threshold, silence_duration=cfg.vad_silence_ms / 1000.0,
min_utterance=cfg.min_utterance, max_utterance=cfg.vad_max_seconds,
stop=self.stopped,
)
def _handle(self, transcript: str) -> None:
cfg = self.config
require_wake = self.mode == "listen"
parsed = grammar.parse(transcript, cfg.wake_phrases, cfg.match_threshold, require_wake,
filler=cfg.filler_words)
parsed = grammar.parse(transcript, cfg.wake_phrases, cfg.wake_fuzzy_threshold,
cfg.command_fuzzy_threshold, require_wake, filler=cfg.filler_words)
if parsed is None or parsed.action is None:
self._console.emit(VOICE, f'heard "{transcript}" -> no command matched', "yellow")
self._console.emit(VOICE, f'heard "{transcript}" -> no command matched {self._timing()}',
"yellow")
return
action = parsed.action
# a command was recognized — echo what we heard (green) before acting. note the
# matched wake phrase (magenta) when the transcript didn't literally contain it
# (so a loose match like "okay clouds" -> "okay claude" is visible).
head = self._console.paint(f'heard "{transcript}" -> {self._describe(action)}', "green")
note = ""
if parsed.wake and parsed.wake.replace(" ", "") not in transcript.lower().replace(" ", ""):
note = (self._console.paint(" (wake: ", "green")
+ self._console.paint(parsed.wake, "magenta")
+ self._console.paint(")", "green"))
tail = self._console.paint(f" {self._timing()}", "green")
self._console.emit(VOICE, f"{head}{note}{tail}")
def blue(s):
return self._console.paint(s, "brightblue")
if action.name == "mode":
new_mode = str(action.arg)
if new_mode != self.mode:
self.mode = new_mode
self._console.emit(SYSTEM, f"mode -> {new_mode}", "cyan")
self._console.emit(SYSTEM, f"{blue('mode')} -> {new_mode}")
self._refresh_state()
return
if action.name == "set":
session = target.set_target(str(action.arg))
self._pending.pop(session, None)
self._console.emit(SYSTEM, f"set sticky -> {session}", "cyan")
self._console.emit(SYSTEM, f"{blue('set sticky')} -> {session}")
self._refresh_state()
return
if action.name == "unset":
target.unset_target()
self._console.emit(SYSTEM, "unset (cleared)", "cyan")
self._console.emit(SYSTEM, f"{blue('unset')} (cleared)")
self._refresh_state()
return
if action.name == "list":
sessions = target.list_sessions()
self._console.emit(SYSTEM, "list -> " + (", ".join(sessions) if sessions else "(none running)"))
self._console.emit(SYSTEM, f"{blue('list')} -> "
+ (", ".join(sessions) if sessions else "(none running)"))
return
if action.name == "commands":
self._console.emit(HELP, "voice commands:")
for usage, desc in grammar.command_menu():
self._console.line(f" {self._console.paint(f'{usage:<26}', 'brightblue')} {desc}")
return
if action.name == "customs":
names = self._contexts.names()
listed = ", ".join(names) if names else "(none — edit contexts.toml)"
self._console.emit(SYSTEM, f"contexts: {listed}")
return
if action.name == "version":
self._console.emit(SYSTEM, f"claudedo {__version__}")
return
if action.name == "debug":
self._console.emit(VOICE, f'debug: "{action.arg}"', "yellow")
return
if action.name == "reload":
self._do_reload(str(action.arg))
return
if action.name == "system":
self._do_system(action.arg)
return
if action.name == "context":
name = str(action.arg[0])
if self._contexts.get(name) is None:
self._console.emit(VOICE, f"no context named '{name}' -> did nothing", "red")
return
session, reason = target.resolve(parsed.one_shot, auto_target=cfg.auto_target)
@@ -198,12 +277,18 @@ class Daemon:
self._console.emit(VOICE, f'heard "{transcript}" -> {reason} -> '
f'{self._describe(action)} did nothing', "red")
return
self._inject(session, transcript, reason, action)
if action.name == "context":
self._inject_context(session, action)
return
self._inject(session, action)
def _inject(self, session: str, transcript: str, reason: str, action) -> None:
def _inject(self, session: str, action) -> None:
"""run a resolved command against `session`, tracking the uncommitted-input
buffer so backspace/erase delete only back to the last submit boundary"""
heard = f'heard "{transcript}" ({reason})'
buffer so backspace/erase delete only back to the last submit boundary.
the 'heard ...' echo is already printed by _handle and the [session] prefix
names the target, so these lines just report the keystrokes injected.
"""
name = action.name
if name == "type":
@@ -211,41 +296,125 @@ class Daemon:
inject.send_literal(session, text)
self._pending[session] = self._pending.get(session, 0) + len(text)
if self.config.type_autosend:
inject.send_named(session, inject.keys.SUBMIT)
inject.send_named(session, keys.SUBMIT)
self._pending[session] = 0
self._console.emit(session, f"{heard} -> typed {text!r}"
self._console.emit(session, f"typed {text!r}"
+ (" + send" if self.config.type_autosend else ""), "green")
return
if name == "space":
n = int(action.arg)
inject.perform(session, action)
self._pending[session] = self._pending.get(session, 0) + n
self._console.emit(session, f"{heard} -> space x{n}", "green")
self._console.emit(session, f"space x{n}", "green")
return
if name == "backspace":
have = self._pending.get(session, 0)
n = min(int(action.arg), have)
n = int(action.arg)
if n:
inject.perform(session, grammar.Action("backspace", n))
self._pending[session] = have - n
self._console.emit(session, f"{heard} -> backspace x{n}"
+ ("" if n == int(action.arg) else " (capped at boundary)"), "green")
inject.perform(session, action)
self._pending[session] = max(0, self._pending.get(session, 0) - n)
self._console.emit(session, f"backspace x{n}", "green")
return
if name == "erase":
n = self._pending.get(session, 0)
if n:
inject.perform(session, grammar.Action("erase", n))
self._pending[session] = 0
self._console.emit(session, f"{heard} -> erase x{n} (to last boundary)", "green")
self._console.emit(session, f"erase x{n} (to last boundary)", "green")
return
inject.perform(session, action)
if name == "submit":
self._pending[session] = 0
self._console.emit(session, f"{heard} -> {self._describe(action)}", "green")
self._console.emit(session, f"injected {self._describe(action)}", "green")
def _inject_context(self, session: str, action) -> None:
"""inject a named context blurb ahead of the dictated instruction, then WAIT.
read-before-send: never auto-submits — the user says ``send`` separately, and
claude's own permission prompt is the backstop for anything consequential.
routes through inject.send_literal (the same path as ``type``) and tracks the
uncommitted-input buffer so backspace/erase still bound to the last boundary.
assembly (config behavior.context_multiline): true -> blurb, a soft Shift+Enter
newline, then the instruction; false -> blurb + context_separator + instruction
flattened onto one line. a bare ``context <name>`` (no dictation) injects just
the blurb. the soft newline does not count toward the editable-char buffer.
"""
cfg = self.config
name, dictation = str(action.arg[0]), str(action.arg[1])
blurb = self._contexts.get(name) or ""
inject.send_literal(session, blurb)
chars = len(blurb)
if dictation:
if cfg.context_multiline:
inject.send_named(session, keys.NEWLINE)
else:
inject.send_literal(session, cfg.context_separator)
chars += len(cfg.context_separator)
inject.send_literal(session, dictation)
chars += len(dictation)
self._pending[session] = self._pending.get(session, 0) + chars
shape = "blurb" if not dictation else "blurb + dictation"
self._console.emit(session, f"context '{name}' -> {shape} (waiting for send)", "green")
def _do_reload(self, scope: str) -> None:
"""re-read config.toml and/or contexts.toml live without reinitializing the
loaded whisper model (the slow part). scope: all|config|contexts."""
did = []
if scope in ("all", "config"):
try:
new_cfg = load_config()
self._apply_config(new_cfg)
did.append("config")
except ConfigError as exc:
self._console.emit(SYSTEM, f"config reload failed (kept previous): {exc}", "red")
if scope in ("all", "contexts"):
self._load_contexts()
did.append("contexts")
what = " + ".join(did) if did else "nothing"
blue = self._console.paint("reloaded", "brightblue")
self._console.emit(SYSTEM, f"{blue} {what} ({len(self._contexts)} contexts)")
def _apply_config(self, new_cfg: Config) -> None:
"""swap in a reloaded config, preserving the runtime mode the user may have
toggled by voice and leaving the already-loaded transcriber untouched."""
new_cfg.mode = self.mode
self.config = new_cfg
def _do_system(self, arg) -> None:
"""daemon-control namespace (never injects to claude): status / reload."""
if isinstance(arg, tuple) and arg and arg[0] == "reload":
self._do_reload(str(arg[1]))
return
if isinstance(arg, tuple) and arg and arg[0] == "unknown":
self._console.emit(SYSTEM, f"unknown system command '{arg[1]}'", "red")
return
if arg == "status":
cfg = self.config
sticky = target.read_active() or "(none)"
blue = self._console.paint("status", "brightblue")
self._console.emit(SYSTEM, f"{blue}: mode {self.mode}, sticky {sticky}, "
f"model {cfg.stt_model}, {len(self._contexts)} contexts")
return
self._console.emit(SYSTEM, f"unknown system command {arg!r}", "red")
def _timing(self) -> str:
"""compact STT latency suffix for heard lines (transcribe ms on audio secs)"""
return f"({self._last_stt_ms:.0f}ms/{self._last_audio_s:.1f}s)"
@staticmethod
def _describe(action) -> str:
if action.name == "context":
name, dictation = action.arg
tail = " + dictation" if dictation else ""
return f"CONTEXT('{name}'{tail})"
if action.name == "system":
arg = action.arg
if isinstance(arg, tuple):
return f"SYSTEM({arg[0]} {arg[1]})"
return f"SYSTEM({arg})"
if action.arg is None:
return action.name.upper()
return f"{action.name.upper()}({action.arg})"
@@ -257,7 +426,8 @@ class Daemon:
invariant: non-command speech is discarded, never recorded.
"""
cfg = self.config
return grammar.strip_wake(transcript, cfg.wake_phrases, cfg.match_threshold, True) is not None
return grammar.strip_wake(transcript, cfg.wake_phrases,
cfg.wake_fuzzy_threshold, True) is not None
def _print_startup(self) -> None:
cfg = self.config
@@ -265,8 +435,9 @@ class Daemon:
target_now = target.read_active() or "(none — run cc / set <name>)"
self._console.emit(SYSTEM, f"claudedo {self.mode} mode — Ctrl-C to stop", "bold")
self._console.emit(SYSTEM, f"model {cfg.stt_model} ({cfg.stt_language}) · mic {dev} · "
f"target {target_now}")
self._console.emit(SYSTEM, "wake: " + ", ".join(cfg.wake_phrases))
f"target {target_now} · {len(self._contexts)} contexts")
wakes = ", ".join(self._console.paint(p, "magenta") for p in cfg.wake_phrases)
self._console.emit(SYSTEM, f"wake: {wakes}")
def _refresh_state(self) -> None:
write_state(os.getpid(), self.mode, target.read_active())
@@ -281,17 +452,23 @@ class Daemon:
self._refresh_state()
self._print_startup()
while not self._stop:
if self._reload_pending:
self._reload_pending = False
self._do_reload("all")
audio_chunk = self._capture()
if self._stop:
break
if audio_chunk is None:
continue
t0 = time.monotonic()
transcript = self._transcriber.transcribe(audio_chunk, self.config.samplerate)
self._last_stt_ms = (time.monotonic() - t0) * 1000.0
self._last_audio_s = audio_chunk.size / self.config.samplerate
if not transcript:
continue
if self.mode == "listen" and not self._has_wake(transcript):
if self.config.print_heard:
self._console.emit(VOICE, f'heard (dropped) "{transcript}"', "red")
self._console.emit(VOICE, f'heard (dropped) "{transcript}" {self._timing()}', "red")
else:
self._console.emit(VOICE, "dropped: non-wake speech (not recorded)", "dim")
continue
+192 -34
View File
@@ -33,11 +33,42 @@ _COUNT_WORDS = {
"sixteen": 16, "seventeen": 17, "eighteen": 18, "nineteen": 19, "twenty": 20,
}
_YES_VERBS = ("yes", "yeah", "yep", "yup")
_NO_VERBS = ("no", "nope", "nah")
_APPROVE_VERBS = ("approve", "allow")
_DENY_VERBS = ("deny", "reject")
_SUBMIT_VERBS = ("send", "enter", "submit")
_CANCEL_VERBS = ("cancel", "escape")
_TYPE_VERBS = ("type", "dictate", "write")
_BACKSPACE_VERBS = ("backspace", "delete")
_SPACE_VERBS = ("space", "spacebar")
_ADD_VERBS = ("add", "insert")
_ERASE_VERBS = ("erase", "clear", "wipe")
_DEBUG_VERBS = ("debug", "echo")
_MODE_VERBS = ("mode",)
_STICKY_VERBS = ("set", "sticky", "switch")
_ONESHOT_VERBS = ("target",)
_UNSET_VERBS = ("unset", "unsticky")
_LIST_VERBS = ("list", "sessions")
_COMMANDS_VERBS = ("commands", "help", "menu")
_CUSTOMS_VERBS = ("customs", "custom")
_VERSION_VERBS = ("version",)
_SELECT_VERBS = ("select", "option", "choose", "number")
_CONTEXT_VERBS = ("context", "prepare")
_RELOAD_VERBS = ("reload",)
_SYSTEM_VERBS = ("system",)
_RELOAD_SCOPES = ("config", "contexts")
# every command/synonym word, for biasing the STT toward the vocabulary we expect.
_COMMAND_WORDS = (
_YES_VERBS + _NO_VERBS + _APPROVE_VERBS + _DENY_VERBS + _SUBMIT_VERBS
+ _CANCEL_VERBS + _TYPE_VERBS + _BACKSPACE_VERBS + _SPACE_VERBS + _ADD_VERBS
+ _ERASE_VERBS + _DEBUG_VERBS + _MODE_VERBS + _STICKY_VERBS + _ONESHOT_VERBS + _UNSET_VERBS
+ _LIST_VERBS + _COMMANDS_VERBS + _CUSTOMS_VERBS + _VERSION_VERBS
+ _CONTEXT_VERBS + _RELOAD_VERBS + _SYSTEM_VERBS + _RELOAD_SCOPES
+ _SELECT_VERBS + ("ptt", "listen")
+ ("one", "two", "three", "four")
)
DEFAULT_FILLER = ("select", "use", "choose")
@@ -46,9 +77,12 @@ class Action:
"""a matched command: a name plus an optional argument.
names: yes, no, select, approve, deny, submit, type, space, backspace, erase,
cancel, mode, set, unset, list. arg carries the select index (int), the literal
text for ``type``, the count for ``space``/``backspace`` (int), the mode for
``mode``, or the session short-name for ``set``.
cancel, mode, set, unset, list, context, reload, system. arg carries the select
index (int), the literal text for ``type``, the count for ``space``/``backspace``
(int), the mode for ``mode``, the session short-name for ``set``, a
``(name, dictation)`` tuple for ``context``, the scope string for ``reload``
(``"all"``/``"config"``/``"contexts"``), or the system control for ``system``
(``"status"`` or a ``("reload", scope)`` tuple).
"""
name: str
@@ -61,11 +95,14 @@ class ParsedCommand:
one_shot is the session short-name from a leading ``target <name>`` (this command
only; does not change the sticky default), or None. action is the command to run,
or None if nothing matched after the wake phrase / one-shot / filler.
or None if nothing matched after the wake phrase / one-shot / filler. wake is the
configured wake phrase that matched (e.g. "okay claude" for a heard "okay clouds"),
or None.
"""
one_shot: str | None
action: Action | None
wake: str | None = None
def normalize(text: str) -> str:
@@ -79,6 +116,56 @@ def normalize(text: str) -> str:
return " ".join(tokens)
def vocabulary(wake_phrases: list[str]) -> list[str]:
"""the wake + command vocabulary, deduped in first-seen order.
single source for biasing the STT: the same wake phrases the matcher uses plus
every command/synonym word in _COMMAND_WORDS. no separate hardcoded copy.
"""
seen: dict[str, None] = {}
for word in list(wake_phrases) + list(_COMMAND_WORDS):
key = word.strip()
if key and key not in seen:
seen[key] = None
return list(seen)
def initial_prompt(wake_phrases: list[str]) -> str:
"""a comma-joined vocabulary string to pass faster-whisper as initial_prompt,
conditioning transcription toward the words we expect (esp. the coined wake)"""
return ", ".join(vocabulary(wake_phrases))
def command_menu() -> list[tuple[str, str]]:
"""the voice command menu as (usage, description) rows, for the `commands` cmd.
a small curated list keyed off the verb groups — the speakable command surface,
NOT the cc shell kit.
"""
return [
("yes / no", "answer a yes/no prompt"),
("one..four", "pick numbered option 1-4"),
("approve / deny", "allow / deny a permission prompt"),
("send", "submit (Enter)"),
("cancel", "back out (Escape)"),
("type <text>", "insert literal text (no submit)"),
("space [n] / add a space", "insert n spaces"),
("backspace [n]", "delete n chars (to last submit)"),
("erase", "wipe the current input"),
("debug <text>", "echo to console (no inject)"),
("set <name>", "sticky target -> claude-<name>"),
("target <name> <cmd>", "one-shot to another session"),
("unset / list", "clear sticky / list sessions"),
("mode ptt|listen", "switch input mode"),
("context <name> <text>", "inject a contexts.toml blurb + dictation (no submit)"),
("reload", "re-read config.toml + contexts.toml live"),
("system status", "print mode/target/model/contexts to the console"),
("system reload [config|contexts]", "reload one or both config files"),
("commands / customs", "this menu / list loaded contexts"),
("version", "print the claudedo version"),
]
def _ratio(a: str, b: str) -> float:
return SequenceMatcher(None, a, b).ratio()
@@ -89,13 +176,15 @@ def _wake_variants(phrase: str) -> set[str]:
return {norm, norm.replace(" ", "")}
def strip_wake(transcript: str, wake_phrases: list[str], threshold: float,
require_wake: bool) -> str | None:
"""return the command remainder after the wake phrase.
def strip_wake_match(transcript: str, wake_phrases: list[str], threshold: float,
require_wake: bool) -> tuple[str | None, str | None]:
"""return (command remainder, matched wake phrase).
if ``require_wake`` (listen mode) and no wake phrase is found at the start,
return None so the daemon discards the utterance. if not required (ptt mode),
a leading wake phrase is stripped when present but its absence is fine.
if ``require_wake`` (listen mode) and no wake phrase is found at the start, the
remainder is None so the daemon discards the utterance. if not required (ptt
mode), a leading wake phrase is stripped when present but its absence is fine.
the matched phrase is the configured wake phrase that best matched (e.g. "okay
claude" for a heard "okay clouds"), or None when none matched.
matches leniently on a despaced prefix (whisper splits/joins the coined word
inconsistently) but always slices the remainder on a WORD boundary of the
@@ -103,10 +192,11 @@ def strip_wake(transcript: str, wake_phrases: list[str], threshold: float,
"""
norm = normalize(transcript)
if not norm:
return None if require_wake else ""
return (None, None) if require_wake else ("", None)
words = norm.split(" ")
best_remainder: str | None = None
best_phrase: str | None = None
best_score = 0.0
for phrase in wake_phrases:
variants = _wake_variants(phrase)
@@ -120,10 +210,18 @@ def strip_wake(transcript: str, wake_phrases: list[str], threshold: float,
if score >= threshold and score > best_score:
best_score = score
best_remainder = " ".join(words[take:]).strip()
best_phrase = phrase
if best_remainder is not None:
return best_remainder
return None if require_wake else norm
return best_remainder, best_phrase
return (None, None) if require_wake else (norm, None)
def strip_wake(transcript: str, wake_phrases: list[str], threshold: float,
require_wake: bool) -> str | None:
"""return the command remainder after the wake phrase (None if no wake in listen
mode). thin wrapper over strip_wake_match for callers that don't need the phrase"""
return strip_wake_match(transcript, wake_phrases, threshold, require_wake)[0]
def _fuzzy_in(token: str, options: tuple[str, ...], threshold: float) -> bool:
@@ -146,6 +244,39 @@ def _leading_count(rest: list[str], default: int = 1) -> int:
return default
def _match_reload(rest: list[str], threshold: float, bare_default: str) -> Action | None:
"""map the tokens after a ``reload`` verb to a reload Action.
bare reload -> the caller's default scope ("all" for the bare command, the
``("reload", scope)`` tuple for ``system reload``). a trailing ``config``/
``contexts`` narrows the scope; an unrecognized scope falls back to the default.
"""
scope = bare_default
if rest and _fuzzy_in(rest[0], ("config", "configuration"), threshold):
scope = "config"
elif rest and _fuzzy_in(rest[0], ("contexts", "context"), threshold):
scope = "contexts"
return Action("reload", scope)
def _match_system(rest: list[str], threshold: float) -> Action | None:
"""map the tokens after the reserved ``system`` word to a daemon-control Action.
the ``system`` namespace never injects into claude. v0.2.0 scope: ``status`` and
``reload [config|contexts]``. unknown controls return a ``system`` Action with an
``("unknown", word)`` arg so the daemon can report it rather than silently drop.
"""
if not rest:
return Action("system", "status")
head = rest[0]
if _fuzzy_in(head, _RELOAD_VERBS, threshold):
inner = _match_reload(rest[1:], threshold, bare_default="all")
return Action("system", ("reload", inner.arg))
if _fuzzy_in(head, ("status", "state"), threshold):
return Action("system", "status")
return Action("system", ("unknown", head))
def match_command(remainder: str, threshold: float) -> Action | None:
"""map a normalized command remainder to an Action, or None if unrecognized.
@@ -160,37 +291,54 @@ def match_command(remainder: str, threshold: float) -> Action | None:
head = tokens[0]
rest = tokens[1:]
if _fuzzy_in(head, _SYSTEM_VERBS, threshold):
return _match_system(rest, threshold)
if _fuzzy_in(head, _RELOAD_VERBS, threshold):
return _match_reload(rest, threshold, bare_default="all")
if _fuzzy_in(head, _CONTEXT_VERBS, threshold) and rest:
name = rest[0]
dictation = " ".join(rest[1:]).strip()
return Action("context", (name, dictation))
if head in _INDEX_WORDS:
return Action("select", _INDEX_WORDS[head])
if _fuzzy_in(head, ("yes", "yeah", "yep", "yup"), threshold):
if _fuzzy_in(head, _YES_VERBS, threshold):
return Action("yes")
if _fuzzy_in(head, ("no", "nope", "nah"), threshold):
if _fuzzy_in(head, _NO_VERBS, threshold):
return Action("no")
if _fuzzy_in(head, ("approve", "allow"), threshold):
if _fuzzy_in(head, _APPROVE_VERBS, threshold):
return Action("approve")
if _fuzzy_in(head, ("deny", "reject"), threshold):
if _fuzzy_in(head, _DENY_VERBS, threshold):
return Action("deny")
if _fuzzy_in(head, ("send", "enter", "submit"), threshold):
if _fuzzy_in(head, _SUBMIT_VERBS, threshold):
return Action("submit")
if _fuzzy_in(head, ("cancel", "escape"), threshold):
if _fuzzy_in(head, _CANCEL_VERBS, threshold):
return Action("cancel")
if _fuzzy_in(head, _SELECT_VERBS, threshold) and rest and rest[0] in _INDEX_WORDS:
return Action("select", _INDEX_WORDS[rest[0]])
if _fuzzy_in(head, ("type", "dictate", "write"), threshold):
if _fuzzy_in(head, _TYPE_VERBS, threshold):
text = " ".join(rest).strip()
return Action("type", text) if text else None
if _fuzzy_in(head, ("backspace", "delete"), threshold):
if _fuzzy_in(head, _BACKSPACE_VERBS, threshold):
return Action("backspace", _leading_count(rest, default=1))
if _fuzzy_in(head, ("space",), threshold):
if _fuzzy_in(head, _SPACE_VERBS, threshold):
return Action("space", _leading_count(rest, default=1))
if _fuzzy_in(head, ("erase", "clear", "wipe"), threshold):
if _fuzzy_in(head, _ADD_VERBS, threshold) and rest:
tail = [t for t in rest if t not in ("a", "an")]
if any(_fuzzy_in(t, ("space", "spaces"), threshold) for t in tail):
count = next((int(t) for t in tail if t.isdigit()),
next((_COUNT_WORDS[t] for t in tail if t in _COUNT_WORDS), 1))
return Action("space", count)
if _fuzzy_in(head, _ERASE_VERBS, threshold):
return Action("erase")
if _fuzzy_in(head, _DEBUG_VERBS, threshold):
return Action("debug", " ".join(rest).strip())
if _fuzzy_in(head, ("mode",), threshold) and rest:
if _fuzzy_in(head, _MODE_VERBS, threshold) and rest:
if _fuzzy_in(rest[0], ("ptt",), threshold) or "push" in rest[0]:
return Action("mode", "ptt")
if _fuzzy_in(rest[0], ("listen",), threshold):
@@ -202,8 +350,14 @@ def match_command(remainder: str, threshold: float) -> Action | None:
return Action("set", name) if name else None
if _fuzzy_in(head, _UNSET_VERBS, threshold) and not rest:
return Action("unset")
if _fuzzy_in(head, _CUSTOMS_VERBS, threshold):
return Action("customs")
if _fuzzy_in(head, _COMMANDS_VERBS, threshold):
return Action("commands")
if _fuzzy_in(head, _LIST_VERBS, threshold):
return Action("list")
if _fuzzy_in(head, _VERSION_VERBS, threshold):
return Action("version")
return None
@@ -221,24 +375,28 @@ def _strip_filler(tokens: list[str], filler: tuple[str, ...], threshold: float)
return tokens
def parse(transcript: str, wake_phrases: list[str], threshold: float,
require_wake: bool, filler: tuple[str, ...] = DEFAULT_FILLER) -> ParsedCommand | None:
def parse(transcript: str, wake_phrases: list[str], wake_threshold: float,
command_threshold: float, require_wake: bool,
filler: tuple[str, ...] = DEFAULT_FILLER) -> ParsedCommand | None:
"""full parse: wake gate -> optional one-shot target -> filler -> command.
returns a ParsedCommand (one_shot, action), or None if the wake gate dropped the
utterance (listen mode, no wake phrase). a ParsedCommand with action=None means a
wake phrase was present but no command matched.
wake_threshold gates the wake phrase (lenient — a false wake is cheap, it just
finds no command); command_threshold gates the command words (stricter — a false
command fires the wrong action). returns a ParsedCommand (one_shot, action), or
None if the wake gate dropped the utterance (listen mode, no wake phrase). a
ParsedCommand with action=None means a wake phrase was present but no command
matched.
"""
remainder = strip_wake(transcript, wake_phrases, threshold, require_wake)
remainder, wake = strip_wake_match(transcript, wake_phrases, wake_threshold, require_wake)
if remainder is None:
return None
tokens = remainder.split(" ") if remainder else []
one_shot: str | None = None
if tokens and _fuzzy_in(tokens[0], _ONESHOT_VERBS, threshold) and len(tokens) >= 2:
if tokens and _fuzzy_in(tokens[0], _ONESHOT_VERBS, command_threshold) and len(tokens) >= 2:
one_shot = tokens[1]
tokens = tokens[2:]
tokens = _strip_filler(tokens, filler, threshold)
action = match_command(" ".join(tokens), threshold)
return ParsedCommand(one_shot=one_shot, action=action)
tokens = _strip_filler(tokens, filler, command_threshold)
action = match_command(" ".join(tokens), command_threshold)
return ParsedCommand(one_shot=one_shot, action=action, wake=wake)
+7
View File
@@ -37,6 +37,13 @@ DENY = ["3"]
SUBMIT = ["Enter"]
CANCEL = ["Escape"]
# NEWLINE is a soft newline inside the input box that does NOT submit — Shift+Enter,
# which tmux names ``S-Enter`` (requires the extended-keys / xterm extkeys tmux
# settings install.sh appends). used to separate a context blurb from the dictated
# instruction in multiline assembly; if it proves flaky the daemon flattens to one
# line with a separator instead (behavior.context_multiline = false).
NEWLINE = ["S-Enter"]
# BACKSPACE deletes one char left; SPACE inserts one literal space. both are emitted
# repeatedly for `backspace <n>` / `space <n>` and for `erase` (n = the daemon's
# tracked uncommitted-input count). BSpace is tmux's name for the backspace key.
+3 -1
View File
@@ -82,8 +82,9 @@ class Transcriber:
"""a loaded faster-whisper model that transcribes float32 mono audio chunks"""
def __init__(self, model: str = "small", language: str = "en", device: str = "auto",
compute_type: str = "auto") -> None:
compute_type: str = "auto", initial_prompt: str | None = None) -> None:
self.language = language
self.initial_prompt = initial_prompt
self._model = self._load(model, device, compute_type)
self._warm()
@@ -120,6 +121,7 @@ class Transcriber:
beam_size=1,
vad_filter=True,
condition_on_previous_text=False,
initial_prompt=self.initial_prompt,
)
text = " ".join(seg.text for seg in segments).strip()
return text