Skip to content

Commit a4c1dc9

Browse files
committed
feat: add local multilingual voice input
1 parent c3d1bad commit a4c1dc9

18 files changed

Lines changed: 1914 additions & 45 deletions

File tree

‎.github/workflows/ci.yml‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,3 +31,22 @@ jobs:
3131

3232
- run: npm run test:server
3333
name: Run pi-server tests
34+
35+
windows-native:
36+
runs-on: windows-latest
37+
38+
steps:
39+
- uses: actions/checkout@v4
40+
41+
- uses: dtolnay/rust-toolchain@stable
42+
43+
- name: Configure LLVM
44+
shell: pwsh
45+
run: |
46+
"LIBCLANG_PATH=C:\Program Files\LLVM\bin" >> $env:GITHUB_ENV
47+
48+
- run: cargo test --manifest-path stt/Cargo.toml --lib
49+
name: Test local speech engine
50+
51+
- run: cargo check --manifest-path src-tauri/Cargo.toml
52+
name: Check Tauri native layer

‎.gitignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -59,6 +59,8 @@ tmp_*
5959

6060
# Test coverage output
6161
coverage/
62+
/stt/target/
63+
/stt/Cargo.lock
6264

6365
# Windows temp file
6466
nul

‎THIRD_PARTY_NOTICES.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
# Third-party notices
2+
3+
## OpenWorker local speech-to-text design
4+
5+
The structure and portions of the local speech-to-text implementation in `stt/` were adapted from
6+
OpenWorker's `ocw-stt` crate.
7+
8+
Copyright (c) 2024 Andrew Ng
9+
10+
Licensed under the MIT License. The full license text is available in the OpenWorker source
11+
distribution and at <https://opensource.org/license/mit>.

‎docs/voice-input.md‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
# Local voice input
2+
3+
Snotra's first voice feature intentionally matches OpenWorker's current scope: local dictation
4+
only. It does not implement text-to-speech, a voice assistant loop, partial streaming transcripts,
5+
or remote audio processing.
6+
7+
## Architecture
8+
9+
- `stt/` is a Tauri-independent Rust crate that owns microphone capture, model provisioning, and
10+
final Whisper transcription.
11+
- `src-tauri/src/dictation.rs` exposes the engine as Tauri commands and reports compatibility.
12+
- `src/utils/voiceInput.ts` is the typed frontend boundary.
13+
- Settings own model download, checksum verification, microphone testing, and deletion.
14+
- The chat input owns the recording UX. A transcript is appended to the editable draft and is
15+
never sent automatically.
16+
17+
Audio samples remain in memory for the active recording and are discarded after transcription or
18+
cancel. The only persistent files are the downloaded model and small verification/readiness
19+
markers in the app data `models` directory.
20+
21+
## Model
22+
23+
Snotra uses multilingual Whisper Base (`ggml-base.bin`) so Chinese and English can be detected
24+
automatically. The model is downloaded on explicit user action and is not bundled into `.exe`,
25+
`.msi`, or `.dmg` installers.
26+
27+
- Download size: `147951465` bytes
28+
- SHA-256: `60ed5bc3dd14eea856493d334349b405782ddcaf0028d4b5df4088345fba2efe`
29+
- Source: `ggerganov/whisper.cpp` on Hugging Face
30+
31+
Downloads are written to a `.part` file, can be cancelled, and only replace the active model after
32+
both the expected size and SHA-256 pass.
33+
34+
## Native build requirements
35+
36+
`whisper-rs` compiles whisper.cpp during the Rust build. Build machines need:
37+
38+
- Rust with the MSVC target on Windows
39+
- Visual Studio C++ Build Tools and a Windows SDK
40+
- CMake
41+
- LLVM/libclang (set `LIBCLANG_PATH` if it is not discoverable)
42+
43+
These are build-time requirements only. End users do not need Node.js, Git, Python, LLVM, CMake, or
44+
Visual Studio to use voice input in an installed Snotra build.
45+
46+
macOS builds also merge `src-tauri/Info.plist`, which contains the microphone usage description.

0 commit comments

Comments
 (0)