Voice assistant: remaining delivery — checkpoints B–D, FR-T12/T14, FR-M4/M6

[thedancingdeveloper]

Tracks the remaining voice-assistant delivery and validation.

Delivered

The implementation and repository-side delivery work are complete on dev:

• Checkpoint A: typed assistant flow, provider profiles, notifications tool, model/effort, repair pass and Assistant tab.
• Checkpoint B / voice (b): Checkpoint B — desktop microphone via Web Speech + the spoken five-utterance pass (FR-T13) (#188) #189: desktop microphone via Web Speech.
• Checkpoint C / voice (c): Checkpoint C — server-side STT/TTS pipeline, the voicemode pattern (FR-T12) (#188) #190 and voice: enable server-side STT/TTS — provision an audio provider (FR-T12 pipeline is built, unconfigured) #317: server-side STT/TTS proxy and the configured-provider path.
• voice (t14): durable, attributable assistant interaction log (FR-T14) (#188) #193: durable attributable interaction log.
• voice (ops): vogt-dev assistant enabled on OpenRouter — validate end-to-end, harden the key (#188) #194: vogt-dev assistant validation.
• Checkpoint D / voice (d): Checkpoint D — background voice on the phone: foreground service, speak-the-push, battery number (FR-M6) (#188) #192: foreground service and speak-the-push implementation.

The closed child issues remain the historical record for those changes.

Remaining gate

Only #191 remains open. Its acceptance requires a live-configured com.sprooty.vogt.dev APK installed beside `com.sprooty.vogt), both registering for FCM, and environment-specific pushes reaching the intended installation. Builds, package identities, Firebase client metadata, and side-by-side installation have been verified; token registration and routing have not.

The software Android AVD used for local validation became unusable with repeated System UI ANRs, and the production front door was unreachable from that session. No FCM acceptance is claimed.

Close condition

Close this parent after #191 records its coexistence and correctly routed push evidence. Any regression found during that device pass should be filed as a new issue against the affected surface.

Closure plan

This parent is complete when #191 records stable side-by-side installation, registration, and environment-isolated push delivery. No additional repository architecture is required unless that device pass finds a regression.

[thedancingdeveloper]

Voice review 2026-08-23 — tracking state reconciled against the code on dev

Two checkpoints this issue lists as pending are already delivered on dev (open only because closing keywords fire on main):

• Checkpoint B / voice (b): Checkpoint B — desktop microphone via Web Speech + the spoken five-utterance pass (FR-T13) (#188) #189 (desktop mic via Web Speech) — DELIVERED, commit 26b449c. The mic is fully wired in web/src/Assistant.tsx: phone → @capacitor-community/speech-recognition, desktop → webkitSpeechRecognition/SpeechRecognition, else the server route / typed input.
• Checkpoint C / voice (c): Checkpoint C — server-side STT/TTS pipeline, the voicemode pattern (FR-T12) (#188) #190 (server STT/TTS pipeline) — DELIVERED, commit 889d24f. POST /api/assistant/stt → /audio/transcriptions, POST /api/assistant/tts → /audio/speech, ENGINE_ASSISTANT_STT_*/_TTS_* config, web client api.assistantStt/assistantTts.

So the two delivery options for real voice are:

• Option 1 — on-device speech: code-complete. Desktop Web Speech (voice (b): Checkpoint B — desktop microphone via Web Speech + the spoken five-utterance pass (FR-T13) (#188) #189) + phone Capacitor STT + on-device TTS + OpenRouter chat. Remaining is hardware validation only — still gated by voice (m4): FCM client entry for the dev application id — unblocks all phone validation (FR-M4) (#188) #191 (FCM client entry for the dev app id) and covered by voice (d): Checkpoint D — background voice on the phone: foreground service, speak-the-push, battery number (FR-M6) (#188) #192 (Checkpoint D). Nothing new to build here.

• Option 2 — server-side STT/TTS: pipeline built (voice (c): Checkpoint C — server-side STT/TTS pipeline, the voicemode pattern (FR-T12) (#188) #190), unconfigured. https://vogt-dev.sprooty.com/api/config reports assistant_stt_enabled: false / assistant_tts_enabled: false (no provider set → 404 → client falls back, FR-T6). Enabling it (provision an OpenAI-audio provider or local Whisper.cpp+Kokoro, set the env, redeploy, validate) is now tracked as voice: enable server-side STT/TTS — provision an audio provider (FR-T12 pipeline is built, unconfigured) #317. OpenRouter can't front audio; Gemini isn't a drop-in for the /audio/* routes.

• Chat (OpenRouter, voice (ops): vogt-dev assistant enabled on OpenRouter — validate end-to-end, harden the key (#188) #194) is enabled and live: assistant_enabled: true, assistant_model: openai/gpt-5.4-mini. Survives a dev redeploy via the ENGINE_→MYDEVENV2_ env alias.

Net: the LLM works; on-device voice is built-but-unvalidated-on-hardware (#191/#192); server voice is built-but-unprovisioned (#317).

[thedancingdeveloper]

Tracker reconciled: #194's vogt-dev GUI validation is complete and #194 is now closed. Remaining close gates are #189's desktop spoken pass, #191's dev/prod FCM coexistence test, and #192's phone Checkpoint D plus battery/screen-off evidence.

[thedancingdeveloper]

Status reconciliation (2026-08-26): dev implementation is complete for the desktop and phone voice paths (#189 and #192 are closed). This parent remains open only for #191, whose acceptance explicitly requires dev/prod APK coexistence and push-routing evidence.

[thedancingdeveloper]

Production verification (2026-08-26): direct prod front door http://100.93.94.7:8910/#/ returns 200; /health/ready is ready; /version is 0.2.2; /api/config reports source_ref prod, source_sha 028b3e3, assistant_enabled true, assistant_stt_enabled true, assistant_tts_enabled true. The remaining parent scope is specifically mobile hardware coexistence/routing (#191), not missing prod implementation.

[thedancingdeveloper]

Status update after PR #432 (merged to dev as 20b7f53): the repository now records the dev/prod FCM identity gate and CI checks that live Firebase configuration stays outside git. The remaining acceptance is real device validation of notification delivery; no hardware result is being claimed here, so the tracker remains open.

[thedancingdeveloper]

Final status: desktop/engine/mobile implementation and delivery checks are in dev; the remaining parent gate is the hardware FCM coexistence/routing evidence tracked by #191.

[thedancingdeveloper]

Status reconciliation (2026-08-26): code-side work for #191 and the #192/#189 dependencies is merged into dev (PR #436). The only remaining acceptance is external: install com.sprooty.vogt and com.sprooty.vogt.dev together on a real Android device, confirm both register for FCM, and verify correctly routed pushes. No code-only change can provide that hardware evidence.

[thedancingdeveloper]

PR #440 (merged to dev as 5aa4a763) reconciles the voice delivery documentation with the current issue states. #191 is explicitly the only remaining voice-delivery gate; #189, #190, #192, #193 and #194 are recorded as closed. No hardware result was fabricated, so #188 remains open pending #191's real-device FCM coexistence/routing evidence.

[thedancingdeveloper]

Voice delivery update (2026-08-26): #191 remains the sole open gate. Both APK package identities build and both renamed Firebase clients are registered, but side-by-side FCM registration and routing remain unverified because no Android device or emulator is available. No hardware result is being claimed; #188 stays open until #191 records that evidence.

[thedancingdeveloper]

Additional validation evidence (2026-08-26): live-config prod/dev APKs build and install side by side on a disposable Android 15 Google APIs AVD as com.sprooty.vogt and com.sprooty.vogt.dev. The software AVD then suffered repeated System UI ANRs and the prod front door was unreachable, so FCM token registration/routing could not be proven. #191 remains the sole open gate; #188 stays open.

[thedancingdeveloper]

Final local validation update (2026-08-26): both live-config prod/dev APKs built and co-installed on an Android 15 Google APIs AVD with distinct package IDs, and Firebase messaging components/state initialized. The software AVD repeatedly ANR'd and the production front door was unreachable, so end-to-end FCM routing remains unverified. #191 is still the sole remaining voice gate; #188 remains open.

[thedancingdeveloper]

Status update after PR #441 (merged to dev as 1dec5e8): docs/VOICE_POC.md now reconciles the delivered desktop/server/phone implementations with the remaining human validation work. #191 remains the sole open voice gate. No FCM token or routing acceptance is claimed; the stable-device/emulator test remains outstanding.

[thedancingdeveloper]

PR #442 is merged into dev as 718247e and hardens the native FCM listener path used by the voice delivery flow. The implementation/documentation work is complete; issue #191 remains the sole open dependency because stable real-device FCM token and routing acceptance is still outstanding.