dsh-client-ui-voice-input
Composer voice control for DeepSeek Harness: a minimal linear mic button in the composer tool row that turns your speech into text — with a tap-to-monitor mode (continuous, live 逐字 streaming, send-anytime) and a hold-to-talk voice-chat mode (release to send, reply read aloud). Zero API key: recognition runs in the browser via the Web Speech API; reply reading uses the host's Edge TTS (/api/tts) with a browser speechSynthesis fallback.
dsh-plugin · TypeScript · React
Features
- Tap to monitor: click the mic, speak — text streams into the draft live (逐字输入), the mic keeps listening even in silence, and you can send or keep adding speech anytime. Tap again to stop.
- Hold to talk: press-and-hold to record a voice-chat message, release to send it; the assistant's reply is read aloud — host Edge neural TTS (
/api/tts) first, browser speechSynthesis as fallback.
- Continuous across silences: each recognition segment auto-restarts so monitoring never drops.
- Respects the composer: speech appends to the draft (base preserved); a send clears the draft cleanly without re-filling old text; monitoring continues after a send on a fresh recognizer.
- DeepSeek-blue listening state: the icon pulses in DeepSeek brand blue while listening; borderless linear icon, no clutter.
- Configurable: recognition language (default
zh-CN) and interim results.
Install
The package is a dsh.bundle installable, published on npm as @zhangbo-cn/dsh-client-ui-voice-input. One command:
dsh plugin add @zhangbo-cn/dsh-client-ui-voice-input
0.1.1+ required. 0.1.0 registered the browser bundle under the wrong ModuleLoader id (@deepseek-ai/...), so Harness failed with loaded without registering "@zhangbo-cn/dsh-client-ui-voice-input". Upgrade / reinstall, then hard-refresh the Web UI.
(It also installs from the GitHub repo via dsh plugin add github:Zhangbo-cn/dsh-voice-input-plugin.)
If you develop from a DeepSeek Harness checkout, you can mount it directly in the web-app browser roster (packages/bundle/web-app/cordis.patch.yml):
- id: ui-voice-input
name: '@zhangbo-cn/dsh-client-ui-voice-input'
For reliable reply reading, also mount the host Edge TTS capability (@deepseek-ai/dsh-tts-edge), which registers /api/tts:
- id: tts-edge
name: '@deepseek-ai/dsh-tts-edge'
Without it, reply reading still works but falls back to the browser's speechSynthesis (less natural, occasionally silent on Chrome after an idle gap).
Then build the client bundle with the repo's tsdown preset:
pnpm --filter @zhangbo-cn/dsh-client-ui-voice-input run bundle
Usage
After refreshing the Web UI, the composer tool row shows a linear mic button.
Voice input (tap)
- Click the mic → the icon turns DeepSeek blue and pulses (listening).
- Speak → text appears in the input box live, word by word.
- Send anytime with the composer's send button; keep talking to add more.
- Click the mic again to stop monitoring.
Voice chat (hold)
- Press-and-hold the mic (longer than ~250 ms) and speak.
- Release → your message is sent.
- The assistant's reply is read aloud automatically.
Reply reading after any send
A send that follows mic use (within 5 minutes) — hold or tap-monitoring + the composer send button — arms reply reading for the next assistant reply. Typed sends without recent mic use do not trigger it.
Configuration
- id: ui-voice-input
name: '@zhangbo-cn/dsh-client-ui-voice-input'
config:
language: 'zh-CN' # Web Speech recognition language tag
interimResults: true # stream live interim transcript into the draft
How it works
MicButton (conversation.input.left)
├─ tap → beginMonitoring()
│ → SpeechRecognition (continuous:false, interimResults) // reliable results
│ → onresult → TranscriptAccumulator → inputActions.setDraft(base + transcript)
│ → onend (silence) → auto-restart (keep monitoring) // continuous
│ → tap again → stop
└─ hold → submitChat()
→ on release: stop + inputActions.setDraft(text) + inputActions.submit()
→ reply streams → complete sentences read aloud WHILE the model
generates (sentence-chunked queue)
→ tail (last incomplete sentence) read on finalize
→ each segment → fetch /api/tts (host Edge neural MP3)
→ play via gesture-unlocked AudioContext (else <audio> element)
→ fallback: browser speechSynthesis
- Recognition starts on pointer-down (a user gesture — required by the Web Speech API); tap vs hold is decided on release.
- The same pointer-down gesture unlocks reply audio (a shared
AudioContext is resumed), so the assistant's reply — which arrives seconds later — is exempt from the browser autoplay policy that would otherwise block a plain HTMLMediaElement.play().
- Reply reading streams: complete sentences are read aloud while the model is still generating (a sentence-chunked queue, flushed at ~30 chars for delimiter-less runs); the last incomplete sentence is read on finalize. The mic icon pulses deep blue while reading, and tapping the mic stops the reading.
- No speaker-echo: while the reply is being read, recognition is paused (the mic physically picks up the speaker), then resumes when reading finishes if monitoring was on.
continuous: false per segment is intentional: Chrome's continuous: true fails to deliver onresult, so monitoring is achieved by auto-restarting segments.- The append base resets when the draft changes externally, so a send never lets stale voice text re-fill the box.
- The console logs
[dsh-voice] diagnostics for each read segment and any fallback.
Compatibility
| Browser | Mic (input, SpeechRecognition) | Reply playback (host /api/tts, fallback speechSynthesis) |
|---|
| Chrome / Edge (Windows) | ✅ Web Speech | ✅ host Edge neural MP3; browser speechSynthesis fallback |
| Safari | ✅ webkitSpeechRecognition (re-trigger on each gesture) | ✅ host Edge neural MP3 (playable); browser fallback works |
| Firefox | ⚠️ not supported — browser limitation (Mozilla has not shipped SpeechRecognition; local on-device recognition is still early-stage) | ✅ host Edge neural MP3 (playable); speechSynthesis fallback supported but less natural |
Notes:
- Firefox mic input: this is a genuine browser limitation, not a plugin issue. The plugin feature-detects and disables the mic with a "not supported in this browser" hint. A cross-browser fallback would need
MediaRecorder + an external transcription service (out of scope for a zero-backend plugin).
- Reply playback: the preferred path is the host's
/api/tts (Microsoft Edge neural voices, synthesized server-side) — reliable and natural on every browser that can play MP3. Without the tts-edge host plugin, the client falls back to speechSynthesis (Chrome may silently drop speak() after an idle gap; voices are OS-default).
- Mic input requires a browser with Web Speech; reply playback requires either the
tts-edge host plugin or a browser with speechSynthesis.
Tests
npx vitest run # 26 tests: tap monitoring, hold submit, auto-restart, send-clear, streaming reply reading, tap-send arming, stop-reading
License
MIT
