Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
f177b46a4b | ||
|
|
252385fb67 | ||
|
|
97591eb24d | ||
|
|
5f05a01423 | ||
|
|
e84ef91e7b | ||
|
|
2cbbabfaa1 | ||
|
|
4357b14fad | ||
|
|
8e20b7eb0b | ||
|
|
4abdfd56bc | ||
|
|
e6dadab143 | ||
|
|
5064f912a4 | ||
|
|
a51c2fbdd4 | ||
|
|
bd6597352a | ||
|
|
08bbe3ce58 | ||
|
|
d96dc3898f | ||
|
|
d734161c97 | ||
|
|
b05f6256c1 |
@@ -1,4 +1,5 @@
|
|||||||
CLAUDE.md
|
CLAUDE.md
|
||||||
|
COMPACT.md
|
||||||
|
|
||||||
__pycache__/
|
__pycache__/
|
||||||
*.pyc
|
*.pyc
|
||||||
|
|||||||
@@ -11,7 +11,7 @@ hands-free while another window (a game) is focused.
|
|||||||
It exists because Claude Code's native `/voice` is hardcoded-blocked in WSL (it
|
It exists because Claude Code's native `/voice` is hardcoded-blocked in WSL (it
|
||||||
assumes WSL has no audio). Modern WSL2 + WSLg *does* have working mic input via
|
assumes WSL has no audio). Modern WSL2 + WSLg *does* have working mic input via
|
||||||
PulseAudio/RDP. `claudedo` captures the mic itself, transcribes on-device, and drives
|
PulseAudio/RDP. `claudedo` captures the mic itself, transcribes on-device, and drives
|
||||||
Claude Code over tmux — fully local, private, backgroundable.
|
Claude Code over tmux — fully local and private. You run it in a terminal you watch.
|
||||||
|
|
||||||
## How it works
|
## How it works
|
||||||
|
|
||||||
@@ -20,9 +20,11 @@ mic (WSLg/PulseAudio RDPSource)
|
|||||||
-> sounddevice capture
|
-> sounddevice capture
|
||||||
-> faster-whisper (local STT, on-device)
|
-> faster-whisper (local STT, on-device)
|
||||||
-> wake gate: utterance must start with a wake phrase, else DISCARD locally
|
-> wake gate: utterance must start with a wake phrase, else DISCARD locally
|
||||||
-> grammar match (yes/no/one..four/approve/deny/send/type/mode/set/target/cancel)
|
-> grammar match (yes/no/one..four/approve/deny/send/type/space/backspace/erase/
|
||||||
-> resolve target session (~/.claude-active)
|
mode/set/target/unset/list/cancel)
|
||||||
|
-> resolve target session (one-shot > sticky ~/.claude-active > auto/none)
|
||||||
-> tmux send-keys -t <session> "<keys>"
|
-> tmux send-keys -t <session> "<keys>"
|
||||||
|
-> log the action to the watched terminal ([session]/[SYSTEM]/[VOICE], colored)
|
||||||
```
|
```
|
||||||
|
|
||||||
**Privacy by construction.** STT runs on-device. In listen mode, any speech that
|
**Privacy by construction.** STT runs on-device. In listen mode, any speech that
|
||||||
@@ -62,16 +64,19 @@ claudedo test-audio
|
|||||||
## Usage
|
## Usage
|
||||||
|
|
||||||
**Run it in a terminal you watch — that's the product.** You launch `claudedo
|
**Run it in a terminal you watch — that's the product.** You launch `claudedo
|
||||||
start`, it does a quick mic check, then drops into a visible listen loop that prints
|
start` and it drops into a visible listen loop (pass `--check` to run a mic check
|
||||||
`heard → matched → sent` for every utterance. That terminal is your
|
first). Each utterance prints a timestamped, colored line — `HH:MM:SS [claude-libs]
|
||||||
recognition/action console; you attach to the `claude-<name>` session in another pane
|
heard "…" →
|
||||||
to watch the keystrokes land. There is no backgrounding/daemon mode — the whole point
|
typed 'fix'` (green for injected, red for drops, `[SYSTEM]`/`[VOICE]` for state and
|
||||||
is the console you read.
|
recognition). That terminal is your recognition/action console; you attach to the
|
||||||
|
`claude-<name>` session in another pane to watch the keystrokes land. It runs in the
|
||||||
|
foreground by design — the console is the point — though `claudedo stop` can signal a
|
||||||
|
stray instance.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
claudedo start # mic-check, then the visible listen loop (listen mode default)
|
claudedo start # the visible listen loop (listen mode default; no mic check)
|
||||||
|
claudedo start --check # run a mic check before listening
|
||||||
claudedo start --mode ptt # push-to-talk instead (desk-only — see Modes)
|
claudedo start --mode ptt # push-to-talk instead (desk-only — see Modes)
|
||||||
claudedo start --skip-audio-check # skip the pre-listen mic check
|
|
||||||
claudedo status # running? mode? target session?
|
claudedo status # running? mode? target session?
|
||||||
claudedo stop # stop a running daemon
|
claudedo stop # stop a running daemon
|
||||||
claudedo set <name> # set the sticky target -> claude-<name> (alias: switch)
|
claudedo set <name> # set the sticky target -> claude-<name> (alias: switch)
|
||||||
@@ -97,9 +102,14 @@ Switch at runtime by voice: "claudedo mode listen" / "claudedo mode ptt".
|
|||||||
|
|
||||||
## Command grammar
|
## Command grammar
|
||||||
|
|
||||||
Wake phrases (listen mode), fuzzy-matched: **"claudedo"**, **"hey claude"**.
|
Wake phrases (listen mode), fuzzy-matched. The default list is **"claudedo"**,
|
||||||
"claudedo" is a coined word, so the matcher is lenient (accepts "claude do",
|
**"claude do"**, **"hey claude"**, **"ok claude"**, **"okay claude"** — Whisper has
|
||||||
"clauddo", "cloud do", …). In PTT mode the wake phrase is optional.
|
no token for the coined word "claudedo" and renders it as real words ("claude do"),
|
||||||
|
so that spelling is listed explicitly. Matching is lenient (case/space-insensitive).
|
||||||
|
Add the spellings you actually see (turn on `print_heard` to find them). In PTT mode
|
||||||
|
the wake phrase is optional. When a command's wake phrase matched loosely (e.g. you
|
||||||
|
said "okay clouds"), the heard line notes which phrase it assumed —
|
||||||
|
`heard "okay clouds list" -> LIST (wake: okay claude)`.
|
||||||
|
|
||||||
| Say | Does |
|
| Say | Does |
|
||||||
|---|---|
|
|---|---|
|
||||||
@@ -108,18 +118,27 @@ Wake phrases (listen mode), fuzzy-matched: **"claudedo"**, **"hey claude"**.
|
|||||||
| `approve` / `deny` | allow / deny a permission prompt |
|
| `approve` / `deny` | allow / deny a permission prompt |
|
||||||
| `send` / `enter` | submit (Enter) |
|
| `send` / `enter` | submit (Enter) |
|
||||||
| `type <phrase>` | insert literal text, **no** submit (read-before-send; say "send") |
|
| `type <phrase>` | insert literal text, **no** submit (read-before-send; say "send") |
|
||||||
|
| `space [<n>]` (also `add [a] space`, `insert <n> spaces`) | insert n spaces (default 1) |
|
||||||
|
| `backspace [<n>]` (alias `delete`) | delete n chars (default 1), capped at the last submit boundary |
|
||||||
|
| `erase` (alias `clear`/`wipe`) | delete everything typed since the last submit/boundary |
|
||||||
|
| `debug <text>` (alias `echo`) | just print what you said to the console (test wake/STT; injects nothing) |
|
||||||
| `mode ptt` / `mode listen` | switch input mode |
|
| `mode ptt` / `mode listen` | switch input mode |
|
||||||
| `set <name>` (alias `sticky`/`switch`) | set the **sticky** target → `claude-<name>` (persists) |
|
| `set <name>` (alias `sticky`/`switch`) | set the **sticky** target → `claude-<name>` (persists) |
|
||||||
| `target <name> <command>` | **one-shot** override: run that command on `claude-<name>` for this utterance only; sticky default unchanged |
|
| `target <name> <command>` | **one-shot** override: run that command on `claude-<name>` for this utterance only; sticky default unchanged |
|
||||||
| `unset` (alias `unsticky`) | clear the sticky target |
|
| `unset` (alias `unsticky`) | clear the sticky target |
|
||||||
| `list` | list running `claude-*` sessions to the daemon console |
|
| `list` | list running `claude-*` sessions to the daemon console |
|
||||||
|
| `commands` (alias `help`/`menu`) | print the voice-command menu to the console |
|
||||||
|
| `customs` (alias `custom`) | custom commands — arriving in v0.2.0 (stub for now) |
|
||||||
|
| `version` | print the claudedo version to the console |
|
||||||
| `cancel` / `escape` | back out of a prompt |
|
| `cancel` / `escape` | back out of a prompt |
|
||||||
|
|
||||||
Optional filler (`select` / `use` / `choose`) may precede any command and is ignored:
|
Optional filler (`select` / `use` / `choose`) may precede any command and is ignored:
|
||||||
`select yes` and `use yes` behave like `yes`. (`select 1` is still the select command.)
|
`select yes` and `use yes` behave like `yes`. (`select 1` is still the select command.)
|
||||||
|
|
||||||
When no sticky target is set, a bare command auto-targets the **only** running
|
When no sticky target is set, a bare command does nothing and asks you to `set` one
|
||||||
`claude-*` session; if several are running it does nothing and asks you to `set` one.
|
(the default). Set `auto_target = true` to instead auto-use the single running
|
||||||
|
`claude-*` session when there's exactly one; with several running it always does
|
||||||
|
nothing and asks you to `set` one.
|
||||||
|
|
||||||
Number words are normalized to digits before matching ("one"/"won" → 1).
|
Number words are normalized to digits before matching ("one"/"won" → 1).
|
||||||
|
|
||||||
@@ -132,9 +151,9 @@ A `target <name>` voice command is a **one-shot** that does NOT touch the sticky
|
|||||||
default — it routes a single command and the next bare command reverts to sticky.
|
default — it routes a single command and the next bare command reverts to sticky.
|
||||||
|
|
||||||
Resolution order (one place — `target.resolve()`): one-shot if present →
|
Resolution order (one place — `target.resolve()`): one-shot if present →
|
||||||
sticky if set and the session exists → else the only running `claude-*` session →
|
sticky if set and the session exists → else, only if `auto_target = true`, the single
|
||||||
else (zero or several) do nothing and say so. It never guesses, and never injects
|
running `claude-*` session → else (default, or zero/several sessions) do nothing and
|
||||||
into a nonexistent session.
|
say so. It never guesses, and never injects into a nonexistent session.
|
||||||
|
|
||||||
Every name maps to `claude-<name>` through one helper (`target.session_name()`), and
|
Every name maps to `claude-<name>` through one helper (`target.session_name()`), and
|
||||||
the cc kit mirrors it exactly — so `cc libs` (shell) and `set libs` (voice) refer
|
the cc kit mirrors it exactly — so `cc libs` (shell) and `set libs` (voice) refer
|
||||||
@@ -171,11 +190,34 @@ If Claude Code changes its prompt UI, re-confirm against a live session and upda
|
|||||||
## Config
|
## Config
|
||||||
|
|
||||||
Everything tunable lives in [`config.toml`](config.toml): wake phrases, mode + PTT
|
Everything tunable lives in [`config.toml`](config.toml): wake phrases, mode + PTT
|
||||||
key, Whisper model/language/device, audio segmentation thresholds, and
|
key, Whisper model/language/device, `[vad]` endpointing, and `[behavior]`
|
||||||
`type_autosend = false`. The default model is `small`; bump to `medium` if the coined
|
(`type_autosend`, fuzzy thresholds, `filler_words`, `auto_target`, `print_heard`).
|
||||||
wake word is recognized poorly. `claudedo -c <path> ...` points at a specific config;
|
The default model is **`small.en`** (the English-only small model — ~1s/command on a
|
||||||
otherwise it searches `$CLAUDEDO_CONFIG`, `~/.config/claudedo/config.toml`, then
|
strong CPU, more accurate on English than multilingual `small` at the same speed);
|
||||||
`./config.toml`.
|
`medium`/`medium.en` are more accurate but ~3× slower (noticeable lag), `base.en` is
|
||||||
|
snappier/less accurate, `large-v3` most accurate/slowest. Every `heard` line shows the
|
||||||
|
STT latency as `(<ms>/<audio>s)` so you can see what a model change costs. VAD
|
||||||
|
endpointing ends a capture after `[vad].silence_ms` (700) of trailing silence, capped
|
||||||
|
at `max_seconds` (15). `claudedo -c <path> ...` points at a specific config; otherwise
|
||||||
|
it searches
|
||||||
|
`$CLAUDEDO_CONFIG`, `~/.config/claudedo/config.toml`, then `./config.toml`.
|
||||||
|
|
||||||
|
- **STT biasing.** The transcriber is seeded with an `initial_prompt` built from the
|
||||||
|
configured wake phrases + command vocabulary (one source — `grammar.vocabulary()`),
|
||||||
|
so Whisper is conditioned to expect "claudedo" and the command words.
|
||||||
|
- **Split fuzzy thresholds.** `wake_fuzzy_threshold` (default `0.65`, lenient) vs
|
||||||
|
`command_fuzzy_threshold` (default `0.8`, tight). The asymmetry is deliberate: a
|
||||||
|
false *wake* is cheap (it wakes, finds no command, does nothing), but a false
|
||||||
|
*command* fires the wrong action. Prefer expanding command synonyms over loosening
|
||||||
|
the command threshold.
|
||||||
|
- **`[vad]` endpointing.** Capture starts on speech and ends after `silence_ms`
|
||||||
|
(default 700) of trailing silence — Alexa-style record-until-pause — capped at
|
||||||
|
`max_seconds` (default 15). The pause both ends a command and separates it from
|
||||||
|
following chatter (the chatter is a separate capture the wake gate discards).
|
||||||
|
- **`auto_target`** (default `false`): with no sticky target and one session running,
|
||||||
|
`false` does nothing and asks you to `set`; `true` auto-uses that session.
|
||||||
|
- **`print_heard`** (default `false`, debug): prints non-wake transcripts so you can
|
||||||
|
see how Whisper renders your wake word, then tune the wake list/threshold.
|
||||||
|
|
||||||
## Requirements
|
## Requirements
|
||||||
|
|
||||||
|
|||||||
+35
-12
@@ -5,7 +5,7 @@
|
|||||||
# wake phrases for listen mode. fuzzy-matched: case/space-insensitive, lenient on
|
# wake phrases for listen mode. fuzzy-matched: case/space-insensitive, lenient on
|
||||||
# the coined word "claudedo" (whisper renders it inconsistently). number words are
|
# the coined word "claudedo" (whisper renders it inconsistently). number words are
|
||||||
# normalized to digits before command matching.
|
# normalized to digits before command matching.
|
||||||
phrases = ["claudedo", "hey claude"]
|
phrases = ["claudedo", "claude do", "hey claude", "ok claude", "okay claude"]
|
||||||
|
|
||||||
[input]
|
[input]
|
||||||
# "listen" (default): continuous capture; only acts on utterances that start with a
|
# "listen" (default): continuous capture; only acts on utterances that start with a
|
||||||
@@ -21,10 +21,12 @@ mode = "listen"
|
|||||||
ptt_key = "space"
|
ptt_key = "space"
|
||||||
|
|
||||||
[stt]
|
[stt]
|
||||||
# faster-whisper model size. "small" is a good accuracy/latency balance for the
|
# faster-whisper model size. "small.en" is the default — the English-only small model
|
||||||
# short command grammar (~sub-second per chunk on a strong cpu). if the coined wake
|
# (~1s/command on a strong cpu, more accurate on english than multilingual "small" at
|
||||||
# word "claudedo" is recognized poorly, bump to "medium" (slower per chunk).
|
# the same speed). "medium"/"medium.en" are more accurate but ~3x slower (noticeable
|
||||||
model = "small"
|
# lag); "large-v3" is most accurate and slowest. drop to "base.en" for max snappiness
|
||||||
|
# (less accurate). bump only if recognition is poor.
|
||||||
|
model = "small.en"
|
||||||
language = "en"
|
language = "en"
|
||||||
# mic device: "auto", or a sounddevice device index (integer) / substring of a
|
# mic device: "auto", or a sounddevice device index (integer) / substring of a
|
||||||
# device name. run `claudedo test-audio` to list devices.
|
# device name. run `claudedo test-audio` to list devices.
|
||||||
@@ -36,22 +38,43 @@ compute = "auto"
|
|||||||
# capture parameters. 16 kHz mono is what whisper expects.
|
# capture parameters. 16 kHz mono is what whisper expects.
|
||||||
samplerate = 16000
|
samplerate = 16000
|
||||||
channels = 1
|
channels = 1
|
||||||
# listen-mode silence segmentation: an utterance ends after this many seconds below
|
# rms energy below this counts as silence (the VAD onset/endpoint floor).
|
||||||
# the rms threshold. keeps latency low without streaming.
|
|
||||||
silence_threshold = 0.012
|
silence_threshold = 0.012
|
||||||
silence_duration = 0.8
|
|
||||||
# ignore utterances shorter than this (clicks, coughs).
|
# ignore utterances shorter than this (clicks, coughs).
|
||||||
min_utterance = 0.3
|
min_utterance = 0.3
|
||||||
# hard cap on a single utterance so a stuck stream can't grow unbounded.
|
|
||||||
max_utterance = 15.0
|
[vad]
|
||||||
|
# Alexa-style record-until-pause endpointing (listen mode). capture starts on speech
|
||||||
|
# onset and ends after this much trailing silence — the natural end of an utterance.
|
||||||
|
# a real pause both ends the command AND separates it from following chatter (the
|
||||||
|
# chatter becomes a separate capture that the wake gate then discards).
|
||||||
|
silence_ms = 700
|
||||||
|
# hard cap so continuous noise can't record forever (also the ceiling for a long
|
||||||
|
# dictated `type` phrase).
|
||||||
|
max_seconds = 15.0
|
||||||
|
|
||||||
[behavior]
|
[behavior]
|
||||||
# dictation never auto-submits: "type <phrase>" inserts literal text only; you say
|
# dictation never auto-submits: "type <phrase>" inserts literal text only; you say
|
||||||
# "send" separately to submit (read-before-send).
|
# "send" separately to submit (read-before-send).
|
||||||
type_autosend = false
|
type_autosend = false
|
||||||
# fuzzy match ratio (0..1) required to accept a wake phrase / command token.
|
# fuzzy match ratios (0..1). the asymmetry is deliberate: a false WAKE is cheap (it
|
||||||
match_threshold = 0.8
|
# wakes, finds no command, does nothing), so wake is lenient; a false COMMAND fires
|
||||||
|
# the WRONG action, so commands stay tight. lower = more lenient = more matches.
|
||||||
|
# prefer expanding command synonyms over loosening command_fuzzy_threshold.
|
||||||
|
wake_fuzzy_threshold = 0.65
|
||||||
|
command_fuzzy_threshold = 0.8
|
||||||
# optional filler words that may precede a command and are ignored for matching:
|
# optional filler words that may precede a command and are ignored for matching:
|
||||||
# "select yes" / "use yes" behave like "yes". (a filler word followed by a digit is
|
# "select yes" / "use yes" behave like "yes". (a filler word followed by a digit is
|
||||||
# the select command, e.g. "select 1", and is not dropped.)
|
# the select command, e.g. "select 1", and is not dropped.)
|
||||||
filler_words = ["select", "use", "choose"]
|
filler_words = ["select", "use", "choose"]
|
||||||
|
# when no sticky target is set and exactly ONE claude-* session is running:
|
||||||
|
# false (default) -> require an explicit `set <name>` or one-shot `target <name>`;
|
||||||
|
# a bare command does nothing and tells you to set one.
|
||||||
|
# true -> auto-target that single session (convenience).
|
||||||
|
auto_target = false
|
||||||
|
# DEBUG ONLY — relaxes the privacy invariant. when true, the daemon console prints
|
||||||
|
# the raw transcript of EVERY utterance, including non-wake speech it would otherwise
|
||||||
|
# drop silently (shown as `heard (dropped): "<transcript>"`). use it to see exactly
|
||||||
|
# how Whisper renders your wake word, then turn it OFF. default false: non-wake speech
|
||||||
|
# is discarded without ever printing the transcript.
|
||||||
|
print_heard = false
|
||||||
|
|||||||
+12
@@ -94,6 +94,18 @@ mkdir -p "$CONF_DIR"
|
|||||||
install -m 0644 "$REPO_DIR/shell/cc.sh" "$CONF_DIR/cc.sh"
|
install -m 0644 "$REPO_DIR/shell/cc.sh" "$CONF_DIR/cc.sh"
|
||||||
echo " wrote $CONF_DIR/cc.sh"
|
echo " wrote $CONF_DIR/cc.sh"
|
||||||
|
|
||||||
|
# install config.toml to the standard location so the daemon finds it from any dir.
|
||||||
|
# never clobber an edited user config: copy only if absent, else drop a .new to diff.
|
||||||
|
if [ ! -f "$CONF_DIR/config.toml" ]; then
|
||||||
|
install -m 0644 "$REPO_DIR/config.toml" "$CONF_DIR/config.toml"
|
||||||
|
echo " wrote $CONF_DIR/config.toml"
|
||||||
|
elif ! cmp -s "$REPO_DIR/config.toml" "$CONF_DIR/config.toml"; then
|
||||||
|
install -m 0644 "$REPO_DIR/config.toml" "$CONF_DIR/config.toml.new"
|
||||||
|
echo " kept your $CONF_DIR/config.toml; new default written to config.toml.new (diff to merge)"
|
||||||
|
else
|
||||||
|
echo " $CONF_DIR/config.toml already current"
|
||||||
|
fi
|
||||||
|
|
||||||
# wire EVERY rc that exists (the user may have both zsh and bash).
|
# wire EVERY rc that exists (the user may have both zsh and bash).
|
||||||
wired_any=0
|
wired_any=0
|
||||||
for RC in "$HOME/.zshrc" "$HOME/.bashrc"; do
|
for RC in "$HOME/.zshrc" "$HOME/.bashrc"; do
|
||||||
|
|||||||
+1
-1
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
|
|||||||
|
|
||||||
[project]
|
[project]
|
||||||
name = "claudedo"
|
name = "claudedo"
|
||||||
version = "0.1.1"
|
version = "0.1.4"
|
||||||
description = "voice-control daemon for claude code (local STT -> tmux send-keys)"
|
description = "voice-control daemon for claude code (local STT -> tmux send-keys)"
|
||||||
readme = "README.md"
|
readme = "README.md"
|
||||||
requires-python = ">=3.10"
|
requires-python = ">=3.10"
|
||||||
|
|||||||
@@ -1,3 +1,3 @@
|
|||||||
"""claudedo — voice-control daemon for claude code (local STT -> tmux send-keys)"""
|
"""claudedo — voice-control daemon for claude code (local STT -> tmux send-keys)"""
|
||||||
|
|
||||||
__version__ = "0.1.1"
|
__version__ = "0.1.4"
|
||||||
|
|||||||
@@ -33,12 +33,12 @@ def cmd_start(args: argparse.Namespace) -> int:
|
|||||||
config = _load_or_die(args.config)
|
config = _load_or_die(args.config)
|
||||||
if args.mode:
|
if args.mode:
|
||||||
config.mode = args.mode
|
config.mode = args.mode
|
||||||
if not args.skip_audio_check:
|
if args.check:
|
||||||
print("checking mic before listening (speak briefly) ...")
|
print("checking mic before listening (speak briefly) ...")
|
||||||
peak = _probe_mic(config, seconds=2.0, verbose=False)
|
peak = _probe_mic(config, seconds=2.0, verbose=False)
|
||||||
if peak is None or peak < 0.02:
|
if peak is None or peak < 0.02:
|
||||||
print("mic check failed — no usable input.", file=sys.stderr)
|
print("mic check failed — no usable input.", file=sys.stderr)
|
||||||
print("run `claudedo test-audio` to debug; or `claudedo start --skip-audio-check`",
|
print("run `claudedo test-audio` to debug, or `claudedo start` to skip the check",
|
||||||
file=sys.stderr)
|
file=sys.stderr)
|
||||||
return 1
|
return 1
|
||||||
print(f"mic OK (peak {peak:.3f}).")
|
print(f"mic OK (peak {peak:.3f}).")
|
||||||
@@ -217,8 +217,8 @@ def build_parser() -> argparse.ArgumentParser:
|
|||||||
|
|
||||||
sp = sub.add_parser("start", help="run the daemon (foreground)")
|
sp = sub.add_parser("start", help="run the daemon (foreground)")
|
||||||
sp.add_argument("--mode", choices=("listen", "ptt"), help="override input mode")
|
sp.add_argument("--mode", choices=("listen", "ptt"), help="override input mode")
|
||||||
sp.add_argument("--skip-audio-check", action="store_true",
|
sp.add_argument("--check", action="store_true",
|
||||||
help="skip the pre-listen mic check")
|
help="run a mic check before listening (off by default)")
|
||||||
sp.set_defaults(func=cmd_start)
|
sp.set_defaults(func=cmd_start)
|
||||||
|
|
||||||
sub.add_parser("stop", help="stop a running daemon").set_defaults(func=cmd_stop)
|
sub.add_parser("stop", help="stop a running daemon").set_defaults(func=cmd_stop)
|
||||||
|
|||||||
+24
-10
@@ -17,7 +17,10 @@ except ModuleNotFoundError:
|
|||||||
log = logging.getLogger(__name__)
|
log = logging.getLogger(__name__)
|
||||||
|
|
||||||
_VALID_MODES = ("listen", "ptt")
|
_VALID_MODES = ("listen", "ptt")
|
||||||
_VALID_MODELS = ("tiny", "base", "small", "medium", "large-v2", "large-v3")
|
_VALID_MODELS = (
|
||||||
|
"tiny", "base", "small", "medium", "large-v1", "large-v2", "large-v3",
|
||||||
|
"tiny.en", "base.en", "small.en", "medium.en",
|
||||||
|
)
|
||||||
|
|
||||||
DEFAULT_CONFIG_PATHS = (
|
DEFAULT_CONFIG_PATHS = (
|
||||||
Path(os.environ.get("CLAUDEDO_CONFIG", "")) if os.environ.get("CLAUDEDO_CONFIG") else None,
|
Path(os.environ.get("CLAUDEDO_CONFIG", "")) if os.environ.get("CLAUDEDO_CONFIG") else None,
|
||||||
@@ -44,12 +47,15 @@ class Config:
|
|||||||
samplerate: int
|
samplerate: int
|
||||||
channels: int
|
channels: int
|
||||||
silence_threshold: float
|
silence_threshold: float
|
||||||
silence_duration: float
|
vad_silence_ms: int
|
||||||
|
vad_max_seconds: float
|
||||||
min_utterance: float
|
min_utterance: float
|
||||||
max_utterance: float
|
|
||||||
type_autosend: bool
|
type_autosend: bool
|
||||||
match_threshold: float
|
wake_fuzzy_threshold: float
|
||||||
|
command_fuzzy_threshold: float
|
||||||
filler_words: tuple[str, ...]
|
filler_words: tuple[str, ...]
|
||||||
|
auto_target: bool
|
||||||
|
print_heard: bool
|
||||||
source_path: Path | None = field(default=None)
|
source_path: Path | None = field(default=None)
|
||||||
|
|
||||||
|
|
||||||
@@ -96,7 +102,7 @@ def load_config(explicit: str | os.PathLike | None = None) -> Config:
|
|||||||
if mode not in _VALID_MODES:
|
if mode not in _VALID_MODES:
|
||||||
raise ConfigError(f"[input].mode must be one of {_VALID_MODES}, got {mode!r}")
|
raise ConfigError(f"[input].mode must be one of {_VALID_MODES}, got {mode!r}")
|
||||||
|
|
||||||
model = _require(raw, "stt", "model", (str,), "small")
|
model = _require(raw, "stt", "model", (str,), "small.en")
|
||||||
if model not in _VALID_MODELS:
|
if model not in _VALID_MODELS:
|
||||||
log.warning("unknown stt model %r — passing through to faster-whisper", model)
|
log.warning("unknown stt model %r — passing through to faster-whisper", model)
|
||||||
|
|
||||||
@@ -111,17 +117,25 @@ def load_config(explicit: str | os.PathLike | None = None) -> Config:
|
|||||||
samplerate=int(_require(raw, "audio", "samplerate", (int,), 16000)),
|
samplerate=int(_require(raw, "audio", "samplerate", (int,), 16000)),
|
||||||
channels=int(_require(raw, "audio", "channels", (int,), 1)),
|
channels=int(_require(raw, "audio", "channels", (int,), 1)),
|
||||||
silence_threshold=float(_require(raw, "audio", "silence_threshold", (int, float), 0.012)),
|
silence_threshold=float(_require(raw, "audio", "silence_threshold", (int, float), 0.012)),
|
||||||
silence_duration=float(_require(raw, "audio", "silence_duration", (int, float), 0.8)),
|
vad_silence_ms=int(_require(raw, "vad", "silence_ms", (int,), 700)),
|
||||||
|
vad_max_seconds=float(_require(raw, "vad", "max_seconds", (int, float), 15.0)),
|
||||||
min_utterance=float(_require(raw, "audio", "min_utterance", (int, float), 0.3)),
|
min_utterance=float(_require(raw, "audio", "min_utterance", (int, float), 0.3)),
|
||||||
max_utterance=float(_require(raw, "audio", "max_utterance", (int, float), 15.0)),
|
|
||||||
type_autosend=bool(_require(raw, "behavior", "type_autosend", (bool,), False)),
|
type_autosend=bool(_require(raw, "behavior", "type_autosend", (bool,), False)),
|
||||||
match_threshold=float(_require(raw, "behavior", "match_threshold", (int, float), 0.8)),
|
wake_fuzzy_threshold=float(_require(raw, "behavior", "wake_fuzzy_threshold", (int, float), 0.65)),
|
||||||
|
command_fuzzy_threshold=float(_require(raw, "behavior", "command_fuzzy_threshold",
|
||||||
|
(int, float), 0.8)),
|
||||||
filler_words=tuple(_require(raw, "behavior", "filler_words", (list,),
|
filler_words=tuple(_require(raw, "behavior", "filler_words", (list,),
|
||||||
["select", "use", "choose"])),
|
["select", "use", "choose"])),
|
||||||
|
auto_target=bool(_require(raw, "behavior", "auto_target", (bool,), False)),
|
||||||
|
print_heard=bool(_require(raw, "behavior", "print_heard", (bool,), False)),
|
||||||
source_path=path,
|
source_path=path,
|
||||||
)
|
)
|
||||||
if not 0.0 < cfg.match_threshold <= 1.0:
|
for label, val in (("wake_fuzzy_threshold", cfg.wake_fuzzy_threshold),
|
||||||
raise ConfigError("[behavior].match_threshold must be in (0, 1]")
|
("command_fuzzy_threshold", cfg.command_fuzzy_threshold)):
|
||||||
|
if not 0.0 < val <= 1.0:
|
||||||
|
raise ConfigError(f"[behavior].{label} must be in (0, 1]")
|
||||||
|
if cfg.vad_silence_ms <= 0 or cfg.vad_max_seconds <= 0:
|
||||||
|
raise ConfigError("[vad].silence_ms and max_seconds must be positive")
|
||||||
if cfg.samplerate <= 0 or cfg.channels <= 0:
|
if cfg.samplerate <= 0 or cfg.channels <= 0:
|
||||||
raise ConfigError("[audio].samplerate and channels must be positive")
|
raise ConfigError("[audio].samplerate and channels must be positive")
|
||||||
return cfg
|
return cfg
|
||||||
|
|||||||
@@ -0,0 +1,65 @@
|
|||||||
|
"""colored, prefixed console output for the daemon's recognition/action feed.
|
||||||
|
|
||||||
|
every line is ``HH:MM:SS [PREFIX] message``. prefixes group the source: a session
|
||||||
|
name (e.g. ``[claude-libs]``) for anything injected into a tmux session, ``[SYSTEM]``
|
||||||
|
for daemon-control/state lines, and ``[VOICE]`` for STT/recognition lines. color is
|
||||||
|
opt-in via tty detection (or forced): green for successful injections, red for
|
||||||
|
drops/errors, dim for routine. falls back to plain text when stdout is not a tty.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import sys
|
||||||
|
import time
|
||||||
|
|
||||||
|
RESET = "\033[0m"
|
||||||
|
_COLORS = {
|
||||||
|
"green": "\033[32m",
|
||||||
|
"red": "\033[31m",
|
||||||
|
"yellow": "\033[33m",
|
||||||
|
"cyan": "\033[36m",
|
||||||
|
"blue": "\033[34m",
|
||||||
|
"brightblue": "\033[94m",
|
||||||
|
"magenta": "\033[35m",
|
||||||
|
"dim": "\033[2m",
|
||||||
|
"bold": "\033[1m",
|
||||||
|
}
|
||||||
|
|
||||||
|
SYSTEM = "SYSTEM"
|
||||||
|
VOICE = "VOICE"
|
||||||
|
HELP = "HELP"
|
||||||
|
|
||||||
|
|
||||||
|
class Console:
|
||||||
|
"""formats and prints daemon log lines with timestamp, prefix, and color"""
|
||||||
|
|
||||||
|
def __init__(self, color: bool | None = None, stream=None, clock=None) -> None:
|
||||||
|
self.stream = stream if stream is not None else sys.stdout
|
||||||
|
self._clock = clock or time.localtime
|
||||||
|
if color is None:
|
||||||
|
color = hasattr(self.stream, "isatty") and self.stream.isatty()
|
||||||
|
self.color = bool(color)
|
||||||
|
|
||||||
|
def _stamp(self) -> str:
|
||||||
|
t = self._clock()
|
||||||
|
return f"{t.tm_hour:02d}:{t.tm_min:02d}:{t.tm_sec:02d}"
|
||||||
|
|
||||||
|
def _paint(self, text: str, color: str | None) -> str:
|
||||||
|
if not self.color or not color or color not in _COLORS:
|
||||||
|
return text
|
||||||
|
return f"{_COLORS[color]}{text}{RESET}"
|
||||||
|
|
||||||
|
def paint(self, text: str, color: str | None) -> str:
|
||||||
|
"""public colorizer for pre-coloring a fragment of a message (e.g. a command
|
||||||
|
word) before passing it to emit() with color=None"""
|
||||||
|
return self._paint(text, color)
|
||||||
|
|
||||||
|
def emit(self, prefix: str, message: str, color: str | None = None) -> None:
|
||||||
|
"""print one line: ``HH:MM:SS [prefix] message`` (message optionally colored)"""
|
||||||
|
line = f"{self._stamp()} {self._paint(f'[{prefix}]', 'dim')} {self._paint(message, color)}"
|
||||||
|
print(line, file=self.stream, flush=True)
|
||||||
|
|
||||||
|
def line(self, message: str, color: str | None = None) -> None:
|
||||||
|
"""print a bare continuation line (no timestamp/prefix) — for multi-row blocks
|
||||||
|
like the help menu, indented under a preceding header"""
|
||||||
|
print(self._paint(message, color), file=self.stream, flush=True)
|
||||||
+114
-35
@@ -16,8 +16,9 @@ import sys
|
|||||||
import time
|
import time
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
from . import audio, grammar, inject, target
|
from . import __version__, audio, grammar, inject, target
|
||||||
from .config import Config
|
from .config import Config
|
||||||
|
from .console import HELP, SYSTEM, VOICE, Console
|
||||||
from .stt import Transcriber
|
from .stt import Transcriber
|
||||||
|
|
||||||
log = logging.getLogger(__name__)
|
log = logging.getLogger(__name__)
|
||||||
@@ -114,6 +115,10 @@ class Daemon:
|
|||||||
self._transcriber: Transcriber | None = None
|
self._transcriber: Transcriber | None = None
|
||||||
self._device: int | None = None
|
self._device: int | None = None
|
||||||
self._ptt = _PTTKey()
|
self._ptt = _PTTKey()
|
||||||
|
self._pending: dict[str, int] = {}
|
||||||
|
self._console = Console()
|
||||||
|
self._last_stt_ms = 0.0
|
||||||
|
self._last_audio_s = 0.0
|
||||||
|
|
||||||
def _install_signals(self) -> None:
|
def _install_signals(self) -> None:
|
||||||
signal.signal(signal.SIGTERM, self._on_signal)
|
signal.signal(signal.SIGTERM, self._on_signal)
|
||||||
@@ -133,6 +138,7 @@ class Daemon:
|
|||||||
model=cfg.stt_model, language=cfg.stt_language,
|
model=cfg.stt_model, language=cfg.stt_language,
|
||||||
device=cfg.stt_compute if cfg.stt_compute in ("cpu", "cuda") else "auto",
|
device=cfg.stt_compute if cfg.stt_compute in ("cpu", "cuda") else "auto",
|
||||||
compute_type="auto",
|
compute_type="auto",
|
||||||
|
initial_prompt=grammar.initial_prompt(cfg.wake_phrases),
|
||||||
)
|
)
|
||||||
if audio.warm_up(cfg.samplerate, cfg.channels, self._device):
|
if audio.warm_up(cfg.samplerate, cfg.channels, self._device):
|
||||||
log.info("mic warmed up (source live)")
|
log.info("mic warmed up (source live)")
|
||||||
@@ -148,59 +154,133 @@ class Daemon:
|
|||||||
return audio.record_while(
|
return audio.record_while(
|
||||||
cfg.samplerate, cfg.channels, self._device,
|
cfg.samplerate, cfg.channels, self._device,
|
||||||
held=lambda: not self._ptt.wait_press(self.stopped),
|
held=lambda: not self._ptt.wait_press(self.stopped),
|
||||||
max_utterance=cfg.max_utterance, min_utterance=cfg.min_utterance,
|
max_utterance=cfg.vad_max_seconds, min_utterance=cfg.min_utterance,
|
||||||
)
|
)
|
||||||
return audio.record_until_silence(
|
return audio.record_until_silence(
|
||||||
cfg.samplerate, cfg.channels, self._device,
|
cfg.samplerate, cfg.channels, self._device,
|
||||||
silence_threshold=cfg.silence_threshold, silence_duration=cfg.silence_duration,
|
silence_threshold=cfg.silence_threshold, silence_duration=cfg.vad_silence_ms / 1000.0,
|
||||||
min_utterance=cfg.min_utterance, max_utterance=cfg.max_utterance,
|
min_utterance=cfg.min_utterance, max_utterance=cfg.vad_max_seconds,
|
||||||
stop=self.stopped,
|
stop=self.stopped,
|
||||||
)
|
)
|
||||||
|
|
||||||
def _handle(self, transcript: str) -> None:
|
def _handle(self, transcript: str) -> None:
|
||||||
cfg = self.config
|
cfg = self.config
|
||||||
require_wake = self.mode == "listen"
|
require_wake = self.mode == "listen"
|
||||||
parsed = grammar.parse(transcript, cfg.wake_phrases, cfg.match_threshold, require_wake,
|
parsed = grammar.parse(transcript, cfg.wake_phrases, cfg.wake_fuzzy_threshold,
|
||||||
filler=cfg.filler_words)
|
cfg.command_fuzzy_threshold, require_wake, filler=cfg.filler_words)
|
||||||
if parsed is None or parsed.action is None:
|
if parsed is None or parsed.action is None:
|
||||||
self._emit(f'heard: "{transcript}" -> no command matched')
|
self._console.emit(VOICE, f'heard "{transcript}" -> no command matched {self._timing()}',
|
||||||
|
"yellow")
|
||||||
return
|
return
|
||||||
action = parsed.action
|
action = parsed.action
|
||||||
|
|
||||||
|
# a command was recognized — echo what we heard (green) before acting. note the
|
||||||
|
# matched wake phrase (magenta) when the transcript didn't literally contain it
|
||||||
|
# (so a loose match like "okay clouds" -> "okay claude" is visible).
|
||||||
|
head = self._console.paint(f'heard "{transcript}" -> {self._describe(action)}', "green")
|
||||||
|
note = ""
|
||||||
|
if parsed.wake and parsed.wake.replace(" ", "") not in transcript.lower().replace(" ", ""):
|
||||||
|
note = (self._console.paint(" (wake: ", "green")
|
||||||
|
+ self._console.paint(parsed.wake, "magenta")
|
||||||
|
+ self._console.paint(")", "green"))
|
||||||
|
tail = self._console.paint(f" {self._timing()}", "green")
|
||||||
|
self._console.emit(VOICE, f"{head}{note}{tail}")
|
||||||
|
|
||||||
|
def blue(s):
|
||||||
|
return self._console.paint(s, "brightblue")
|
||||||
if action.name == "mode":
|
if action.name == "mode":
|
||||||
new_mode = str(action.arg)
|
new_mode = str(action.arg)
|
||||||
if new_mode != self.mode:
|
if new_mode != self.mode:
|
||||||
self.mode = new_mode
|
self.mode = new_mode
|
||||||
self._emit(f"mode -> {new_mode}")
|
self._console.emit(SYSTEM, f"{blue('mode')} -> {new_mode}")
|
||||||
self._refresh_state()
|
self._refresh_state()
|
||||||
return
|
return
|
||||||
if action.name == "set":
|
if action.name == "set":
|
||||||
session = target.set_target(str(action.arg))
|
session = target.set_target(str(action.arg))
|
||||||
self._emit(f"set sticky -> {session}")
|
self._pending.pop(session, None)
|
||||||
|
self._console.emit(SYSTEM, f"{blue('set sticky')} -> {session}")
|
||||||
self._refresh_state()
|
self._refresh_state()
|
||||||
return
|
return
|
||||||
if action.name == "unset":
|
if action.name == "unset":
|
||||||
target.unset_target()
|
target.unset_target()
|
||||||
self._emit("unset (cleared)")
|
self._console.emit(SYSTEM, f"{blue('unset')} (cleared)")
|
||||||
self._refresh_state()
|
self._refresh_state()
|
||||||
return
|
return
|
||||||
if action.name == "list":
|
if action.name == "list":
|
||||||
sessions = target.list_sessions()
|
sessions = target.list_sessions()
|
||||||
self._emit("list -> " + (", ".join(sessions) if sessions else "(none running)"))
|
self._console.emit(SYSTEM, f"{blue('list')} -> "
|
||||||
|
+ (", ".join(sessions) if sessions else "(none running)"))
|
||||||
|
return
|
||||||
|
if action.name == "commands":
|
||||||
|
self._console.emit(HELP, "voice commands:")
|
||||||
|
for usage, desc in grammar.command_menu():
|
||||||
|
self._console.line(f" {self._console.paint(f'{usage:<26}', 'brightblue')} {desc}")
|
||||||
|
return
|
||||||
|
if action.name == "customs":
|
||||||
|
self._console.emit(SYSTEM, "custom commands arrive in v0.2.0 (contexts.toml)")
|
||||||
|
return
|
||||||
|
if action.name == "version":
|
||||||
|
self._console.emit(SYSTEM, f"claudedo {__version__}")
|
||||||
|
return
|
||||||
|
if action.name == "debug":
|
||||||
|
self._console.emit(VOICE, f'debug: "{action.arg}"', "yellow")
|
||||||
return
|
return
|
||||||
|
|
||||||
session, reason = target.resolve(parsed.one_shot)
|
session, reason = target.resolve(parsed.one_shot, auto_target=cfg.auto_target)
|
||||||
if session is None:
|
if session is None:
|
||||||
self._emit(f'heard: "{transcript}" -> {reason} -> matched {self._describe(action)} '
|
self._console.emit(VOICE, f'heard "{transcript}" -> {reason} -> '
|
||||||
f'-> did nothing')
|
f'{self._describe(action)} did nothing', "red")
|
||||||
return
|
return
|
||||||
prefix = f'heard: "{transcript}" -> {reason} -> matched {self._describe(action)}'
|
self._inject(session, action)
|
||||||
if action.name == "type" and not cfg.type_autosend:
|
|
||||||
inject.send_literal(session, str(action.arg))
|
def _inject(self, session: str, action) -> None:
|
||||||
self._emit(f"{prefix} -> injected literal {str(action.arg)!r} -> {session}")
|
"""run a resolved command against `session`, tracking the uncommitted-input
|
||||||
|
buffer so backspace/erase delete only back to the last submit boundary.
|
||||||
|
|
||||||
|
the 'heard ...' echo is already printed by _handle and the [session] prefix
|
||||||
|
names the target, so these lines just report the keystrokes injected.
|
||||||
|
"""
|
||||||
|
name = action.name
|
||||||
|
|
||||||
|
if name == "type":
|
||||||
|
text = str(action.arg)
|
||||||
|
inject.send_literal(session, text)
|
||||||
|
self._pending[session] = self._pending.get(session, 0) + len(text)
|
||||||
|
if self.config.type_autosend:
|
||||||
|
inject.send_named(session, inject.keys.SUBMIT)
|
||||||
|
self._pending[session] = 0
|
||||||
|
self._console.emit(session, f"typed {text!r}"
|
||||||
|
+ (" + send" if self.config.type_autosend else ""), "green")
|
||||||
return
|
return
|
||||||
|
if name == "space":
|
||||||
|
n = int(action.arg)
|
||||||
inject.perform(session, action)
|
inject.perform(session, action)
|
||||||
self._emit(f"{prefix} -> injected {self._describe(action)} -> {session}")
|
self._pending[session] = self._pending.get(session, 0) + n
|
||||||
|
self._console.emit(session, f"space x{n}", "green")
|
||||||
|
return
|
||||||
|
if name == "backspace":
|
||||||
|
n = int(action.arg)
|
||||||
|
if n:
|
||||||
|
inject.perform(session, action)
|
||||||
|
self._pending[session] = max(0, self._pending.get(session, 0) - n)
|
||||||
|
self._console.emit(session, f"backspace x{n}", "green")
|
||||||
|
return
|
||||||
|
if name == "erase":
|
||||||
|
n = self._pending.get(session, 0)
|
||||||
|
if n:
|
||||||
|
inject.perform(session, grammar.Action("erase", n))
|
||||||
|
self._pending[session] = 0
|
||||||
|
self._console.emit(session, f"erase x{n} (to last boundary)", "green")
|
||||||
|
return
|
||||||
|
|
||||||
|
inject.perform(session, action)
|
||||||
|
if name == "submit":
|
||||||
|
self._pending[session] = 0
|
||||||
|
self._console.emit(session, f"injected {self._describe(action)}", "green")
|
||||||
|
|
||||||
|
def _timing(self) -> str:
|
||||||
|
"""compact STT latency suffix for heard lines (transcribe ms on audio secs)"""
|
||||||
|
return f"({self._last_stt_ms:.0f}ms/{self._last_audio_s:.1f}s)"
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def _describe(action) -> str:
|
def _describe(action) -> str:
|
||||||
@@ -208,11 +288,6 @@ class Daemon:
|
|||||||
return action.name.upper()
|
return action.name.upper()
|
||||||
return f"{action.name.upper()}({action.arg})"
|
return f"{action.name.upper()}({action.arg})"
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def _emit(line: str) -> None:
|
|
||||||
"""print a recognition/action line to the watched terminal"""
|
|
||||||
print(line, flush=True)
|
|
||||||
|
|
||||||
def _has_wake(self, transcript: str) -> bool:
|
def _has_wake(self, transcript: str) -> bool:
|
||||||
"""true if the utterance starts with a wake phrase (listen-mode gate).
|
"""true if the utterance starts with a wake phrase (listen-mode gate).
|
||||||
|
|
||||||
@@ -220,20 +295,18 @@ class Daemon:
|
|||||||
invariant: non-command speech is discarded, never recorded.
|
invariant: non-command speech is discarded, never recorded.
|
||||||
"""
|
"""
|
||||||
cfg = self.config
|
cfg = self.config
|
||||||
return grammar.strip_wake(transcript, cfg.wake_phrases, cfg.match_threshold, True) is not None
|
return grammar.strip_wake(transcript, cfg.wake_phrases,
|
||||||
|
cfg.wake_fuzzy_threshold, True) is not None
|
||||||
|
|
||||||
def _print_startup(self) -> None:
|
def _print_startup(self) -> None:
|
||||||
cfg = self.config
|
cfg = self.config
|
||||||
dev = cfg.stt_device if cfg.stt_device != "auto" else "default"
|
dev = cfg.stt_device if cfg.stt_device != "auto" else "default"
|
||||||
target_now = target.read_active() or "(none — run cc to attach)"
|
target_now = target.read_active() or "(none — run cc / set <name>)"
|
||||||
self._emit("── claudedo ─────────────────────────────────")
|
self._console.emit(SYSTEM, f"claudedo {self.mode} mode — Ctrl-C to stop", "bold")
|
||||||
self._emit(f" model: {cfg.stt_model} ({cfg.stt_language})")
|
self._console.emit(SYSTEM, f"model {cfg.stt_model} ({cfg.stt_language}) · mic {dev} · "
|
||||||
self._emit(f" mic: {dev}")
|
f"target {target_now}")
|
||||||
self._emit(f" mode: {self.mode}")
|
wakes = ", ".join(self._console.paint(p, "magenta") for p in cfg.wake_phrases)
|
||||||
self._emit(f" target: {target_now}")
|
self._console.emit(SYSTEM, f"wake: {wakes}")
|
||||||
self._emit(f" wake: {', '.join(cfg.wake_phrases)}")
|
|
||||||
self._emit(" Ctrl-C to stop")
|
|
||||||
self._emit("─────────────────────────────────────────────")
|
|
||||||
|
|
||||||
def _refresh_state(self) -> None:
|
def _refresh_state(self) -> None:
|
||||||
write_state(os.getpid(), self.mode, target.read_active())
|
write_state(os.getpid(), self.mode, target.read_active())
|
||||||
@@ -253,11 +326,17 @@ class Daemon:
|
|||||||
break
|
break
|
||||||
if audio_chunk is None:
|
if audio_chunk is None:
|
||||||
continue
|
continue
|
||||||
|
t0 = time.monotonic()
|
||||||
transcript = self._transcriber.transcribe(audio_chunk, self.config.samplerate)
|
transcript = self._transcriber.transcribe(audio_chunk, self.config.samplerate)
|
||||||
|
self._last_stt_ms = (time.monotonic() - t0) * 1000.0
|
||||||
|
self._last_audio_s = audio_chunk.size / self.config.samplerate
|
||||||
if not transcript:
|
if not transcript:
|
||||||
continue
|
continue
|
||||||
if self.mode == "listen" and not self._has_wake(transcript):
|
if self.mode == "listen" and not self._has_wake(transcript):
|
||||||
self._emit("dropped: non-wake speech (not recorded)")
|
if self.config.print_heard:
|
||||||
|
self._console.emit(VOICE, f'heard (dropped) "{transcript}" {self._timing()}', "red")
|
||||||
|
else:
|
||||||
|
self._console.emit(VOICE, "dropped: non-wake speech (not recorded)", "dim")
|
||||||
continue
|
continue
|
||||||
self._handle(transcript)
|
self._handle(transcript)
|
||||||
finally:
|
finally:
|
||||||
|
|||||||
+165
-31
@@ -27,11 +27,43 @@ _NUMBER_WORDS = {
|
|||||||
|
|
||||||
_INDEX_WORDS = {"1": 1, "2": 2, "3": 3, "4": 4}
|
_INDEX_WORDS = {"1": 1, "2": 2, "3": 3, "4": 4}
|
||||||
|
|
||||||
|
_COUNT_WORDS = {
|
||||||
|
"five": 5, "six": 6, "seven": 7, "eight": 8, "nine": 9, "ten": 10,
|
||||||
|
"eleven": 11, "twelve": 12, "thirteen": 13, "fourteen": 14, "fifteen": 15,
|
||||||
|
"sixteen": 16, "seventeen": 17, "eighteen": 18, "nineteen": 19, "twenty": 20,
|
||||||
|
}
|
||||||
|
|
||||||
|
_YES_VERBS = ("yes", "yeah", "yep", "yup")
|
||||||
|
_NO_VERBS = ("no", "nope", "nah")
|
||||||
|
_APPROVE_VERBS = ("approve", "allow")
|
||||||
|
_DENY_VERBS = ("deny", "reject")
|
||||||
|
_SUBMIT_VERBS = ("send", "enter", "submit")
|
||||||
|
_CANCEL_VERBS = ("cancel", "escape")
|
||||||
|
_TYPE_VERBS = ("type", "dictate", "write")
|
||||||
|
_BACKSPACE_VERBS = ("backspace", "delete")
|
||||||
|
_SPACE_VERBS = ("space", "spacebar")
|
||||||
|
_ADD_VERBS = ("add", "insert")
|
||||||
|
_ERASE_VERBS = ("erase", "clear", "wipe")
|
||||||
|
_DEBUG_VERBS = ("debug", "echo")
|
||||||
|
_MODE_VERBS = ("mode",)
|
||||||
_STICKY_VERBS = ("set", "sticky", "switch")
|
_STICKY_VERBS = ("set", "sticky", "switch")
|
||||||
_ONESHOT_VERBS = ("target",)
|
_ONESHOT_VERBS = ("target",)
|
||||||
_UNSET_VERBS = ("unset", "unsticky")
|
_UNSET_VERBS = ("unset", "unsticky")
|
||||||
_LIST_VERBS = ("list", "sessions")
|
_LIST_VERBS = ("list", "sessions")
|
||||||
|
_COMMANDS_VERBS = ("commands", "help", "menu")
|
||||||
|
_CUSTOMS_VERBS = ("customs", "custom")
|
||||||
|
_VERSION_VERBS = ("version",)
|
||||||
_SELECT_VERBS = ("select", "option", "choose", "number")
|
_SELECT_VERBS = ("select", "option", "choose", "number")
|
||||||
|
|
||||||
|
# every command/synonym word, for biasing the STT toward the vocabulary we expect.
|
||||||
|
_COMMAND_WORDS = (
|
||||||
|
_YES_VERBS + _NO_VERBS + _APPROVE_VERBS + _DENY_VERBS + _SUBMIT_VERBS
|
||||||
|
+ _CANCEL_VERBS + _TYPE_VERBS + _BACKSPACE_VERBS + _SPACE_VERBS + _ADD_VERBS
|
||||||
|
+ _ERASE_VERBS + _DEBUG_VERBS + _MODE_VERBS + _STICKY_VERBS + _ONESHOT_VERBS + _UNSET_VERBS
|
||||||
|
+ _LIST_VERBS + _COMMANDS_VERBS + _CUSTOMS_VERBS + _VERSION_VERBS
|
||||||
|
+ _SELECT_VERBS + ("ptt", "listen")
|
||||||
|
+ ("one", "two", "three", "four")
|
||||||
|
)
|
||||||
DEFAULT_FILLER = ("select", "use", "choose")
|
DEFAULT_FILLER = ("select", "use", "choose")
|
||||||
|
|
||||||
|
|
||||||
@@ -39,9 +71,10 @@ DEFAULT_FILLER = ("select", "use", "choose")
|
|||||||
class Action:
|
class Action:
|
||||||
"""a matched command: a name plus an optional argument.
|
"""a matched command: a name plus an optional argument.
|
||||||
|
|
||||||
names: yes, no, select, approve, deny, submit, type, cancel, mode, set, unset,
|
names: yes, no, select, approve, deny, submit, type, space, backspace, erase,
|
||||||
list. arg carries the select index (int), the literal text for ``type``, the mode
|
cancel, mode, set, unset, list. arg carries the select index (int), the literal
|
||||||
for ``mode``, or the session short-name for ``set``.
|
text for ``type``, the count for ``space``/``backspace`` (int), the mode for
|
||||||
|
``mode``, or the session short-name for ``set``.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
name: str
|
name: str
|
||||||
@@ -54,11 +87,14 @@ class ParsedCommand:
|
|||||||
|
|
||||||
one_shot is the session short-name from a leading ``target <name>`` (this command
|
one_shot is the session short-name from a leading ``target <name>`` (this command
|
||||||
only; does not change the sticky default), or None. action is the command to run,
|
only; does not change the sticky default), or None. action is the command to run,
|
||||||
or None if nothing matched after the wake phrase / one-shot / filler.
|
or None if nothing matched after the wake phrase / one-shot / filler. wake is the
|
||||||
|
configured wake phrase that matched (e.g. "okay claude" for a heard "okay clouds"),
|
||||||
|
or None.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
one_shot: str | None
|
one_shot: str | None
|
||||||
action: Action | None
|
action: Action | None
|
||||||
|
wake: str | None = None
|
||||||
|
|
||||||
|
|
||||||
def normalize(text: str) -> str:
|
def normalize(text: str) -> str:
|
||||||
@@ -72,6 +108,52 @@ def normalize(text: str) -> str:
|
|||||||
return " ".join(tokens)
|
return " ".join(tokens)
|
||||||
|
|
||||||
|
|
||||||
|
def vocabulary(wake_phrases: list[str]) -> list[str]:
|
||||||
|
"""the wake + command vocabulary, deduped in first-seen order.
|
||||||
|
|
||||||
|
single source for biasing the STT: the same wake phrases the matcher uses plus
|
||||||
|
every command/synonym word in _COMMAND_WORDS. no separate hardcoded copy.
|
||||||
|
"""
|
||||||
|
seen: dict[str, None] = {}
|
||||||
|
for word in list(wake_phrases) + list(_COMMAND_WORDS):
|
||||||
|
key = word.strip()
|
||||||
|
if key and key not in seen:
|
||||||
|
seen[key] = None
|
||||||
|
return list(seen)
|
||||||
|
|
||||||
|
|
||||||
|
def initial_prompt(wake_phrases: list[str]) -> str:
|
||||||
|
"""a comma-joined vocabulary string to pass faster-whisper as initial_prompt,
|
||||||
|
conditioning transcription toward the words we expect (esp. the coined wake)"""
|
||||||
|
return ", ".join(vocabulary(wake_phrases))
|
||||||
|
|
||||||
|
|
||||||
|
def command_menu() -> list[tuple[str, str]]:
|
||||||
|
"""the voice command menu as (usage, description) rows, for the `commands` cmd.
|
||||||
|
|
||||||
|
a small curated list keyed off the verb groups — the speakable command surface,
|
||||||
|
NOT the cc shell kit.
|
||||||
|
"""
|
||||||
|
return [
|
||||||
|
("yes / no", "answer a yes/no prompt"),
|
||||||
|
("one..four", "pick numbered option 1-4"),
|
||||||
|
("approve / deny", "allow / deny a permission prompt"),
|
||||||
|
("send", "submit (Enter)"),
|
||||||
|
("cancel", "back out (Escape)"),
|
||||||
|
("type <text>", "insert literal text (no submit)"),
|
||||||
|
("space [n] / add a space", "insert n spaces"),
|
||||||
|
("backspace [n]", "delete n chars (to last submit)"),
|
||||||
|
("erase", "wipe the current input"),
|
||||||
|
("debug <text>", "echo to console (no inject)"),
|
||||||
|
("set <name>", "sticky target -> claude-<name>"),
|
||||||
|
("target <name> <cmd>", "one-shot to another session"),
|
||||||
|
("unset / list", "clear sticky / list sessions"),
|
||||||
|
("mode ptt|listen", "switch input mode"),
|
||||||
|
("commands / customs", "this menu / custom commands (v0.2.0)"),
|
||||||
|
("version", "print the claudedo version"),
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
def _ratio(a: str, b: str) -> float:
|
def _ratio(a: str, b: str) -> float:
|
||||||
return SequenceMatcher(None, a, b).ratio()
|
return SequenceMatcher(None, a, b).ratio()
|
||||||
|
|
||||||
@@ -82,13 +164,15 @@ def _wake_variants(phrase: str) -> set[str]:
|
|||||||
return {norm, norm.replace(" ", "")}
|
return {norm, norm.replace(" ", "")}
|
||||||
|
|
||||||
|
|
||||||
def strip_wake(transcript: str, wake_phrases: list[str], threshold: float,
|
def strip_wake_match(transcript: str, wake_phrases: list[str], threshold: float,
|
||||||
require_wake: bool) -> str | None:
|
require_wake: bool) -> tuple[str | None, str | None]:
|
||||||
"""return the command remainder after the wake phrase.
|
"""return (command remainder, matched wake phrase).
|
||||||
|
|
||||||
if ``require_wake`` (listen mode) and no wake phrase is found at the start,
|
if ``require_wake`` (listen mode) and no wake phrase is found at the start, the
|
||||||
return None so the daemon discards the utterance. if not required (ptt mode),
|
remainder is None so the daemon discards the utterance. if not required (ptt
|
||||||
a leading wake phrase is stripped when present but its absence is fine.
|
mode), a leading wake phrase is stripped when present but its absence is fine.
|
||||||
|
the matched phrase is the configured wake phrase that best matched (e.g. "okay
|
||||||
|
claude" for a heard "okay clouds"), or None when none matched.
|
||||||
|
|
||||||
matches leniently on a despaced prefix (whisper splits/joins the coined word
|
matches leniently on a despaced prefix (whisper splits/joins the coined word
|
||||||
inconsistently) but always slices the remainder on a WORD boundary of the
|
inconsistently) but always slices the remainder on a WORD boundary of the
|
||||||
@@ -96,10 +180,11 @@ def strip_wake(transcript: str, wake_phrases: list[str], threshold: float,
|
|||||||
"""
|
"""
|
||||||
norm = normalize(transcript)
|
norm = normalize(transcript)
|
||||||
if not norm:
|
if not norm:
|
||||||
return None if require_wake else ""
|
return (None, None) if require_wake else ("", None)
|
||||||
words = norm.split(" ")
|
words = norm.split(" ")
|
||||||
|
|
||||||
best_remainder: str | None = None
|
best_remainder: str | None = None
|
||||||
|
best_phrase: str | None = None
|
||||||
best_score = 0.0
|
best_score = 0.0
|
||||||
for phrase in wake_phrases:
|
for phrase in wake_phrases:
|
||||||
variants = _wake_variants(phrase)
|
variants = _wake_variants(phrase)
|
||||||
@@ -113,16 +198,40 @@ def strip_wake(transcript: str, wake_phrases: list[str], threshold: float,
|
|||||||
if score >= threshold and score > best_score:
|
if score >= threshold and score > best_score:
|
||||||
best_score = score
|
best_score = score
|
||||||
best_remainder = " ".join(words[take:]).strip()
|
best_remainder = " ".join(words[take:]).strip()
|
||||||
|
best_phrase = phrase
|
||||||
|
|
||||||
if best_remainder is not None:
|
if best_remainder is not None:
|
||||||
return best_remainder
|
return best_remainder, best_phrase
|
||||||
return None if require_wake else norm
|
return (None, None) if require_wake else (norm, None)
|
||||||
|
|
||||||
|
|
||||||
|
def strip_wake(transcript: str, wake_phrases: list[str], threshold: float,
|
||||||
|
require_wake: bool) -> str | None:
|
||||||
|
"""return the command remainder after the wake phrase (None if no wake in listen
|
||||||
|
mode). thin wrapper over strip_wake_match for callers that don't need the phrase"""
|
||||||
|
return strip_wake_match(transcript, wake_phrases, threshold, require_wake)[0]
|
||||||
|
|
||||||
|
|
||||||
def _fuzzy_in(token: str, options: tuple[str, ...], threshold: float) -> bool:
|
def _fuzzy_in(token: str, options: tuple[str, ...], threshold: float) -> bool:
|
||||||
return any(_ratio(token, opt) >= threshold for opt in options)
|
return any(_ratio(token, opt) >= threshold for opt in options)
|
||||||
|
|
||||||
|
|
||||||
|
def _leading_count(rest: list[str], default: int = 1) -> int:
|
||||||
|
"""read a count from the first token (digit or number word), else the default.
|
||||||
|
|
||||||
|
'backspace 3' -> 3, 'backspace ten' -> 10 (normalize maps small words to digits;
|
||||||
|
larger words come from _COUNT_WORDS), 'backspace' -> default.
|
||||||
|
"""
|
||||||
|
if not rest:
|
||||||
|
return default
|
||||||
|
tok = rest[0]
|
||||||
|
if tok.isdigit():
|
||||||
|
return max(0, int(tok))
|
||||||
|
if tok in _COUNT_WORDS:
|
||||||
|
return _COUNT_WORDS[tok]
|
||||||
|
return default
|
||||||
|
|
||||||
|
|
||||||
def match_command(remainder: str, threshold: float) -> Action | None:
|
def match_command(remainder: str, threshold: float) -> Action | None:
|
||||||
"""map a normalized command remainder to an Action, or None if unrecognized.
|
"""map a normalized command remainder to an Action, or None if unrecognized.
|
||||||
|
|
||||||
@@ -140,27 +249,42 @@ def match_command(remainder: str, threshold: float) -> Action | None:
|
|||||||
if head in _INDEX_WORDS:
|
if head in _INDEX_WORDS:
|
||||||
return Action("select", _INDEX_WORDS[head])
|
return Action("select", _INDEX_WORDS[head])
|
||||||
|
|
||||||
if _fuzzy_in(head, ("yes", "yeah", "yep", "yup"), threshold):
|
if _fuzzy_in(head, _YES_VERBS, threshold):
|
||||||
return Action("yes")
|
return Action("yes")
|
||||||
if _fuzzy_in(head, ("no", "nope", "nah"), threshold):
|
if _fuzzy_in(head, _NO_VERBS, threshold):
|
||||||
return Action("no")
|
return Action("no")
|
||||||
if _fuzzy_in(head, ("approve", "allow"), threshold):
|
if _fuzzy_in(head, _APPROVE_VERBS, threshold):
|
||||||
return Action("approve")
|
return Action("approve")
|
||||||
if _fuzzy_in(head, ("deny", "reject"), threshold):
|
if _fuzzy_in(head, _DENY_VERBS, threshold):
|
||||||
return Action("deny")
|
return Action("deny")
|
||||||
if _fuzzy_in(head, ("send", "enter", "submit"), threshold):
|
if _fuzzy_in(head, _SUBMIT_VERBS, threshold):
|
||||||
return Action("submit")
|
return Action("submit")
|
||||||
if _fuzzy_in(head, ("cancel", "escape"), threshold):
|
if _fuzzy_in(head, _CANCEL_VERBS, threshold):
|
||||||
return Action("cancel")
|
return Action("cancel")
|
||||||
|
|
||||||
if _fuzzy_in(head, _SELECT_VERBS, threshold) and rest and rest[0] in _INDEX_WORDS:
|
if _fuzzy_in(head, _SELECT_VERBS, threshold) and rest and rest[0] in _INDEX_WORDS:
|
||||||
return Action("select", _INDEX_WORDS[rest[0]])
|
return Action("select", _INDEX_WORDS[rest[0]])
|
||||||
|
|
||||||
if _fuzzy_in(head, ("type", "dictate", "write"), threshold):
|
if _fuzzy_in(head, _TYPE_VERBS, threshold):
|
||||||
text = " ".join(rest).strip()
|
text = " ".join(rest).strip()
|
||||||
return Action("type", text) if text else None
|
return Action("type", text) if text else None
|
||||||
|
|
||||||
if _fuzzy_in(head, ("mode",), threshold) and rest:
|
if _fuzzy_in(head, _BACKSPACE_VERBS, threshold):
|
||||||
|
return Action("backspace", _leading_count(rest, default=1))
|
||||||
|
if _fuzzy_in(head, _SPACE_VERBS, threshold):
|
||||||
|
return Action("space", _leading_count(rest, default=1))
|
||||||
|
if _fuzzy_in(head, _ADD_VERBS, threshold) and rest:
|
||||||
|
tail = [t for t in rest if t not in ("a", "an")]
|
||||||
|
if any(_fuzzy_in(t, ("space", "spaces"), threshold) for t in tail):
|
||||||
|
count = next((int(t) for t in tail if t.isdigit()),
|
||||||
|
next((_COUNT_WORDS[t] for t in tail if t in _COUNT_WORDS), 1))
|
||||||
|
return Action("space", count)
|
||||||
|
if _fuzzy_in(head, _ERASE_VERBS, threshold):
|
||||||
|
return Action("erase")
|
||||||
|
if _fuzzy_in(head, _DEBUG_VERBS, threshold):
|
||||||
|
return Action("debug", " ".join(rest).strip())
|
||||||
|
|
||||||
|
if _fuzzy_in(head, _MODE_VERBS, threshold) and rest:
|
||||||
if _fuzzy_in(rest[0], ("ptt",), threshold) or "push" in rest[0]:
|
if _fuzzy_in(rest[0], ("ptt",), threshold) or "push" in rest[0]:
|
||||||
return Action("mode", "ptt")
|
return Action("mode", "ptt")
|
||||||
if _fuzzy_in(rest[0], ("listen",), threshold):
|
if _fuzzy_in(rest[0], ("listen",), threshold):
|
||||||
@@ -172,8 +296,14 @@ def match_command(remainder: str, threshold: float) -> Action | None:
|
|||||||
return Action("set", name) if name else None
|
return Action("set", name) if name else None
|
||||||
if _fuzzy_in(head, _UNSET_VERBS, threshold) and not rest:
|
if _fuzzy_in(head, _UNSET_VERBS, threshold) and not rest:
|
||||||
return Action("unset")
|
return Action("unset")
|
||||||
|
if _fuzzy_in(head, _CUSTOMS_VERBS, threshold):
|
||||||
|
return Action("customs")
|
||||||
|
if _fuzzy_in(head, _COMMANDS_VERBS, threshold):
|
||||||
|
return Action("commands")
|
||||||
if _fuzzy_in(head, _LIST_VERBS, threshold):
|
if _fuzzy_in(head, _LIST_VERBS, threshold):
|
||||||
return Action("list")
|
return Action("list")
|
||||||
|
if _fuzzy_in(head, _VERSION_VERBS, threshold):
|
||||||
|
return Action("version")
|
||||||
|
|
||||||
return None
|
return None
|
||||||
|
|
||||||
@@ -191,24 +321,28 @@ def _strip_filler(tokens: list[str], filler: tuple[str, ...], threshold: float)
|
|||||||
return tokens
|
return tokens
|
||||||
|
|
||||||
|
|
||||||
def parse(transcript: str, wake_phrases: list[str], threshold: float,
|
def parse(transcript: str, wake_phrases: list[str], wake_threshold: float,
|
||||||
require_wake: bool, filler: tuple[str, ...] = DEFAULT_FILLER) -> ParsedCommand | None:
|
command_threshold: float, require_wake: bool,
|
||||||
|
filler: tuple[str, ...] = DEFAULT_FILLER) -> ParsedCommand | None:
|
||||||
"""full parse: wake gate -> optional one-shot target -> filler -> command.
|
"""full parse: wake gate -> optional one-shot target -> filler -> command.
|
||||||
|
|
||||||
returns a ParsedCommand (one_shot, action), or None if the wake gate dropped the
|
wake_threshold gates the wake phrase (lenient — a false wake is cheap, it just
|
||||||
utterance (listen mode, no wake phrase). a ParsedCommand with action=None means a
|
finds no command); command_threshold gates the command words (stricter — a false
|
||||||
wake phrase was present but no command matched.
|
command fires the wrong action). returns a ParsedCommand (one_shot, action), or
|
||||||
|
None if the wake gate dropped the utterance (listen mode, no wake phrase). a
|
||||||
|
ParsedCommand with action=None means a wake phrase was present but no command
|
||||||
|
matched.
|
||||||
"""
|
"""
|
||||||
remainder = strip_wake(transcript, wake_phrases, threshold, require_wake)
|
remainder, wake = strip_wake_match(transcript, wake_phrases, wake_threshold, require_wake)
|
||||||
if remainder is None:
|
if remainder is None:
|
||||||
return None
|
return None
|
||||||
|
|
||||||
tokens = remainder.split(" ") if remainder else []
|
tokens = remainder.split(" ") if remainder else []
|
||||||
one_shot: str | None = None
|
one_shot: str | None = None
|
||||||
if tokens and _fuzzy_in(tokens[0], _ONESHOT_VERBS, threshold) and len(tokens) >= 2:
|
if tokens and _fuzzy_in(tokens[0], _ONESHOT_VERBS, command_threshold) and len(tokens) >= 2:
|
||||||
one_shot = tokens[1]
|
one_shot = tokens[1]
|
||||||
tokens = tokens[2:]
|
tokens = tokens[2:]
|
||||||
|
|
||||||
tokens = _strip_filler(tokens, filler, threshold)
|
tokens = _strip_filler(tokens, filler, command_threshold)
|
||||||
action = match_command(" ".join(tokens), threshold)
|
action = match_command(" ".join(tokens), command_threshold)
|
||||||
return ParsedCommand(one_shot=one_shot, action=action)
|
return ParsedCommand(one_shot=one_shot, action=action, wake=wake)
|
||||||
|
|||||||
+13
-2
@@ -45,11 +45,18 @@ class OutputHandler(ABC):
|
|||||||
def send_literal(self, session: str, text: str) -> None:
|
def send_literal(self, session: str, text: str) -> None:
|
||||||
"""emit literal text into the input box without submitting (``type``)"""
|
"""emit literal text into the input box without submitting (``type``)"""
|
||||||
|
|
||||||
|
def send_repeat(self, session: str, token: str, count: int) -> None:
|
||||||
|
"""emit a named key `count` times (e.g. BSpace x n). default impl loops."""
|
||||||
|
if count <= 0:
|
||||||
|
return
|
||||||
|
self.send_named(session, [token] * count)
|
||||||
|
|
||||||
def perform(self, session: str, action) -> bool:
|
def perform(self, session: str, action) -> bool:
|
||||||
"""resolve a grammar.Action to keystrokes and emit them. returns acted?.
|
"""resolve a grammar.Action to keystrokes and emit them. returns acted?.
|
||||||
|
|
||||||
``switch`` and ``mode`` are handled by the daemon (they change daemon state,
|
``switch``/``set``/``mode`` etc. are handled by the daemon (they change daemon
|
||||||
not the claude session), so they are ignored here.
|
state, not the claude session), so they are ignored here. ``erase`` arrives
|
||||||
|
with action.arg already set to the count the daemon wants backspaced.
|
||||||
"""
|
"""
|
||||||
name = action.name
|
name = action.name
|
||||||
if name == "yes":
|
if name == "yes":
|
||||||
@@ -72,6 +79,10 @@ class OutputHandler(ABC):
|
|||||||
self.send_named(session, seq)
|
self.send_named(session, seq)
|
||||||
elif name == "type":
|
elif name == "type":
|
||||||
self.send_literal(session, str(action.arg))
|
self.send_literal(session, str(action.arg))
|
||||||
|
elif name == "space":
|
||||||
|
self.send_literal(session, " " * int(action.arg))
|
||||||
|
elif name in ("backspace", "erase"):
|
||||||
|
self.send_repeat(session, keys.BACKSPACE[0], int(action.arg))
|
||||||
else:
|
else:
|
||||||
return False
|
return False
|
||||||
return True
|
return True
|
||||||
|
|||||||
@@ -37,6 +37,12 @@ DENY = ["3"]
|
|||||||
SUBMIT = ["Enter"]
|
SUBMIT = ["Enter"]
|
||||||
CANCEL = ["Escape"]
|
CANCEL = ["Escape"]
|
||||||
|
|
||||||
|
# BACKSPACE deletes one char left; SPACE inserts one literal space. both are emitted
|
||||||
|
# repeatedly for `backspace <n>` / `space <n>` and for `erase` (n = the daemon's
|
||||||
|
# tracked uncommitted-input count). BSpace is tmux's name for the backspace key.
|
||||||
|
BACKSPACE = ["BSpace"]
|
||||||
|
SPACE = [" "]
|
||||||
|
|
||||||
SELECT_BY_INDEX = {
|
SELECT_BY_INDEX = {
|
||||||
1: SELECT_1,
|
1: SELECT_1,
|
||||||
2: SELECT_2,
|
2: SELECT_2,
|
||||||
|
|||||||
+78
-3
@@ -6,32 +6,106 @@ short in-memory chunk; nothing is written to disk or sent anywhere.
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import contextlib
|
||||||
import logging
|
import logging
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
import sys
|
||||||
|
|
||||||
import numpy as np
|
import numpy as np
|
||||||
|
|
||||||
log = logging.getLogger(__name__)
|
log = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
_NOISE = re.compile(r"GPU device discovery failed|device_discovery\.cc|DiscoverDevicesForPlatform")
|
||||||
|
|
||||||
|
|
||||||
|
def _quiet_backends() -> None:
|
||||||
|
"""quiet onnxruntime/ctranslate2 chatter and the faster_whisper INFO log.
|
||||||
|
|
||||||
|
faster-whisper's VAD loads an onnx model whose device discovery prints a noisy
|
||||||
|
'GPU device discovery failed' warning on headless/WSL hosts with no GPU sysfs.
|
||||||
|
the env var + logger severity stop most onnx logging; the warning itself is
|
||||||
|
emitted at C++ init and is filtered out of stderr by _filter_stderr().
|
||||||
|
"""
|
||||||
|
os.environ.setdefault("ORT_LOGGING_LEVEL", "3")
|
||||||
|
os.environ.setdefault("OMP_NUM_THREADS", os.environ.get("OMP_NUM_THREADS", "4"))
|
||||||
|
logging.getLogger("faster_whisper").setLevel(logging.WARNING)
|
||||||
|
try:
|
||||||
|
import onnxruntime
|
||||||
|
onnxruntime.set_default_logger_severity(3)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
@contextlib.contextmanager
|
||||||
|
def _filter_stderr():
|
||||||
|
"""drop onnxruntime's GPU-discovery warning lines from stderr for this block.
|
||||||
|
|
||||||
|
a pipe temporarily replaces fd 2; a pump thread forwards every line to the real
|
||||||
|
stderr EXCEPT the known GPU-discovery noise, so real errors still surface. the
|
||||||
|
original fd is always restored on exit.
|
||||||
|
"""
|
||||||
|
import threading
|
||||||
|
|
||||||
|
try:
|
||||||
|
stderr_fd = sys.stderr.fileno()
|
||||||
|
except (AttributeError, OSError):
|
||||||
|
yield
|
||||||
|
return
|
||||||
|
|
||||||
|
saved_fd = os.dup(stderr_fd)
|
||||||
|
read_fd, write_fd = os.pipe()
|
||||||
|
os.dup2(write_fd, stderr_fd)
|
||||||
|
os.close(write_fd)
|
||||||
|
|
||||||
|
def pump():
|
||||||
|
with os.fdopen(read_fd, "rb") as reader, os.fdopen(saved_fd, "wb", closefd=False) as out:
|
||||||
|
for line in reader:
|
||||||
|
if not _NOISE.search(line.decode("utf-8", "replace")):
|
||||||
|
out.write(line)
|
||||||
|
out.flush()
|
||||||
|
|
||||||
|
thread = threading.Thread(target=pump, daemon=True)
|
||||||
|
thread.start()
|
||||||
|
try:
|
||||||
|
yield
|
||||||
|
finally:
|
||||||
|
import time
|
||||||
|
|
||||||
|
time.sleep(0.05)
|
||||||
|
os.dup2(saved_fd, stderr_fd)
|
||||||
|
os.close(saved_fd)
|
||||||
|
thread.join(timeout=1.0)
|
||||||
|
|
||||||
|
|
||||||
class Transcriber:
|
class Transcriber:
|
||||||
"""a loaded faster-whisper model that transcribes float32 mono audio chunks"""
|
"""a loaded faster-whisper model that transcribes float32 mono audio chunks"""
|
||||||
|
|
||||||
def __init__(self, model: str = "small", language: str = "en", device: str = "auto",
|
def __init__(self, model: str = "small", language: str = "en", device: str = "auto",
|
||||||
compute_type: str = "auto") -> None:
|
compute_type: str = "auto", initial_prompt: str | None = None) -> None:
|
||||||
self.language = language
|
self.language = language
|
||||||
|
self.initial_prompt = initial_prompt
|
||||||
self._model = self._load(model, device, compute_type)
|
self._model = self._load(model, device, compute_type)
|
||||||
|
self._warm()
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def _load(model: str, device: str, compute_type: str):
|
def _load(model: str, device: str, compute_type: str):
|
||||||
from faster_whisper import WhisperModel
|
|
||||||
|
|
||||||
if device == "auto":
|
if device == "auto":
|
||||||
device = "cpu"
|
device = "cpu"
|
||||||
if compute_type == "auto":
|
if compute_type == "auto":
|
||||||
compute_type = "int8" if device == "cpu" else "float16"
|
compute_type = "int8" if device == "cpu" else "float16"
|
||||||
log.info("loading faster-whisper model=%s device=%s compute=%s", model, device, compute_type)
|
log.info("loading faster-whisper model=%s device=%s compute=%s", model, device, compute_type)
|
||||||
|
with _filter_stderr():
|
||||||
|
_quiet_backends()
|
||||||
|
from faster_whisper import WhisperModel
|
||||||
return WhisperModel(model, device=device, compute_type=compute_type)
|
return WhisperModel(model, device=device, compute_type=compute_type)
|
||||||
|
|
||||||
|
def _warm(self) -> None:
|
||||||
|
"""run one throwaway transcribe so the VAD onnx session inits now, under the
|
||||||
|
stderr filter — the GPU-discovery warning fires here once, not in the loop"""
|
||||||
|
with _filter_stderr():
|
||||||
|
list(self._model.transcribe(np.zeros(1600, dtype=np.float32), vad_filter=True)[0])
|
||||||
|
|
||||||
def transcribe(self, audio: np.ndarray, samplerate: int = 16000) -> str:
|
def transcribe(self, audio: np.ndarray, samplerate: int = 16000) -> str:
|
||||||
"""transcribe a mono float32 numpy array to a stripped text string.
|
"""transcribe a mono float32 numpy array to a stripped text string.
|
||||||
|
|
||||||
@@ -47,6 +121,7 @@ class Transcriber:
|
|||||||
beam_size=1,
|
beam_size=1,
|
||||||
vad_filter=True,
|
vad_filter=True,
|
||||||
condition_on_previous_text=False,
|
condition_on_previous_text=False,
|
||||||
|
initial_prompt=self.initial_prompt,
|
||||||
)
|
)
|
||||||
text = " ".join(seg.text for seg in segments).strip()
|
text = " ".join(seg.text for seg in segments).strip()
|
||||||
return text
|
return text
|
||||||
|
|||||||
@@ -85,7 +85,7 @@ def list_sessions() -> list[str]:
|
|||||||
return sorted(n for n in names if n.startswith(SESSION_PREFIX))
|
return sorted(n for n in names if n.startswith(SESSION_PREFIX))
|
||||||
|
|
||||||
|
|
||||||
def resolve(one_shot: str | None = None) -> tuple[str | None, str]:
|
def resolve(one_shot: str | None = None, auto_target: bool = False) -> tuple[str | None, str]:
|
||||||
"""resolve the destination session and a short reason describing the choice.
|
"""resolve the destination session and a short reason describing the choice.
|
||||||
|
|
||||||
single source of truth for targeting, used by both the voice and CLI paths.
|
single source of truth for targeting, used by both the voice and CLI paths.
|
||||||
@@ -95,7 +95,9 @@ def resolve(one_shot: str | None = None) -> tuple[str | None, str]:
|
|||||||
1. one-shot present -> claude-<name> for THIS command only; never falls through
|
1. one-shot present -> claude-<name> for THIS command only; never falls through
|
||||||
to a different session if it doesn't exist (explicit beats convenience).
|
to a different session if it doesn't exist (explicit beats convenience).
|
||||||
2. sticky set + exists -> use it.
|
2. sticky set + exists -> use it.
|
||||||
3. nothing sticky, exactly one claude-* session -> auto-use it.
|
3. nothing sticky, exactly one claude-* session:
|
||||||
|
auto_target=True -> auto-use it;
|
||||||
|
auto_target=False -> require an explicit set/target, do nothing.
|
||||||
4. nothing sticky, multiple sessions -> ambiguous, do nothing.
|
4. nothing sticky, multiple sessions -> ambiguous, do nothing.
|
||||||
5. nothing sticky, zero sessions -> do nothing.
|
5. nothing sticky, zero sessions -> do nothing.
|
||||||
"""
|
"""
|
||||||
@@ -113,7 +115,9 @@ def resolve(one_shot: str | None = None) -> tuple[str | None, str]:
|
|||||||
|
|
||||||
sessions = list_sessions()
|
sessions = list_sessions()
|
||||||
if len(sessions) == 1:
|
if len(sessions) == 1:
|
||||||
|
if auto_target:
|
||||||
return sessions[0], f"auto-target {sessions[0]} (only session)"
|
return sessions[0], f"auto-target {sessions[0]} (only session)"
|
||||||
|
return None, f"no target set ({sessions[0]} running — set one)"
|
||||||
if len(sessions) > 1:
|
if len(sessions) > 1:
|
||||||
return None, f"no target set, {len(sessions)} sessions (set one)"
|
return None, f"no target set, {len(sessions)} sessions (set one)"
|
||||||
return None, "no claude sessions"
|
return None, "no claude sessions"
|
||||||
|
|||||||
Reference in New Issue
Block a user