Repository navigation
fix(desk-floater): keep voice recordings until their words are delivered - #264
Merged
Merged
Conversation
…r mailbox notes Keep each recording until its words are delivered: a recording whose transcription fails, comes back empty though long enough to hold words, or cannot be delivered is saved in the home's private state/desk-voice/unsent/ and retried automatically, with fm-desk-voice.sh keep/recordings/retry to save, list and transcribe it again later. Captures are recorded under the home so a floater that stops mid-message saves the recording at its next launch. Bound the Deepgram request with a size-scaled timeout and two retries instead of letting it hang, with no length cap; a ten-minute recording transcribes whole. A message saved to the mailbox now rings the busy primary, retrying until its chat can take the line, so it is collected within seconds rather than at the end of the current task, and the floater says "Saved for Firstmate" for it.
…nd dictation auto-retry
…ull desk voice test suite passes locally. The other two failing shards hit tests in code this PR doesn't touch, so I made no change for them. **Shard 4: fm-deepgram-desk, "the unsent folder must be private" (caused by this PR, fixed).** - Invariant: the test helper `mode_of` must return a file's octal mode on both macOS (BSD `stat`) and Linux (GNU `stat`). - Cause: the PR's helper ran the BSD form `stat -f '%Lp'` first. On Linux, GNU `stat -f` reports on the filesystem instead and still exits 0, so the fallback never ran. The helper returned something that wasn't a mode, and the check for 700 failed. The checks for 600 on the saved recording and its JSON record use the same helper. - Fix: one line at `tests/fm-deepgram-desk.test.sh:1109`, now `stat -c '%a' "$1" 2>/dev/null || stat -f '%Lp' "$1"`. This is the order `tests/fm-captain-ledger.test.sh` already uses. On macOS `stat -c` fails and the BSD form runs; on Linux the GNU form works. - The other mode check in the change already picks the `stat` form by `uname`, so it was fine. **Verification:** - A full local run of `bash tests/fm-deepgram-desk.test.sh` exited 0 with 82 passing checks and no failures. These include "fm-desk-voice keep/retry: a recording is kept, private, until its words are delivered" and "desk floater: Swift tests pass". - GNU `stat` isn't on this Mac, so I couldn't run the Linux path. Only the CI re-run will confirm it. - A background watcher I started to wait on an earlier run was stopped at its time limit. It was looking for a line the suite never prints. The stop says nothing about the tests, and the full run above is the result that counts. **Shard 11: fm-watcher-lock, "dead link-lock owner did not publish its pid" (not caused by this PR, no change).** - The test gives a background lock holder about one second (50 × 0.02s) to write its pid. The PR changes no lock code, nothing in `lib/`, and not this test. This looks like a timing flake on a busy CI runner. **Shard 2: fm-supervision-host, "the host-only transition veto changed Pi's existing offer rule" (not caused by this PR, no change).** - The test calls `branchOfferForWake` in `.pi/extensions/lib/fm-branch-dispatch.ts` through node. The PR changes no Pi or supervision code, and I couldn't tie the failure to this change. If it fails again on the re-run, it needs its own look
Amplify-Logic
force-pushed
the
fm/desk-voice-never-lose-g1
branch
from
October 7, 2026 08:11
b9e214b to
208f8c4
Compare
Amplify-Logic
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
"Can you check the last voice note? I just did a really long one, but it seems like it didn't send to you. Can you also make sure that that can't happen again? I think it went on for too long, but it should still be stored."
Context from firstmate's check: the note did arrive, but in the desk-voice mailbox rather than the chat, and only reached firstmate about six minutes later. Separately,
desk-floater/Sources/DeskFloater.swifttranscribeAndDeliver deletes the recorded audio file right after transcription, before checking the result, so a recording whose transcription fails or comes back empty (shown as "No speech") is lost for good.What Changed
state/desk-voice/recording/and no longer deletes the audio as soon as transcription finishes. A recording is removed only once its words are delivered. If transcription fails (0.5 s or longer), comes back empty (2 s or longer), or delivery fails, the recording goes to a privatestate/desk-voice/unsent/folder. Talk-to-Firstmate recordings are retried automatically after 20 s, 60 s and 5 min, and the floater shows "Saved - retrying". Dictation recordings show "Saved, not sent" and wait for a manual retry. When the floater launches, it saves any recordings a previous run left behind and picks up retries that still have attempts left. The newRecording,SendOutcomeandRetryResulthelpers live inRecordings.swift, with Swift tests.bin/fm-desk-voice.shgets three new commands.keepsaves a recording to the unsent folder with a JSON record of its purpose, attempt count and last failure reason.recordingslists what is saved.retryre-transcribes saved recordings under a lock, then delivers the words, or prints them for dictation. When a message falls back to the mailbox,delivernow also rings the primary in the background on a retry schedule (FM_DESK_VOICE_RING_DELAYS), so the message doesn't wait for the primary's next turn end. The ring stops once the message is drained, once a ring is typed, when no proven primary holds the session lock, and in away or quiet mode.sendnow fails if it can't save to the mailbox.bin/fm-deepgram-stt.shstill uploads long recordings whole, with no length cap. Each request now has limits: a 15 s connect timeout, a total timeout that grows with the file size (120 s plus 1 s per 100 kB), and two retries on timeouts, dropped connections or busy responses. The docs (docs/desk-floater.md,docs/scripts.md, the operational home layout skill),fork-surface.confandtests/fm-deepgram-desk.test.share updated to cover the new folders, commands and behavior.🤖 Generated with Claude Code
Risk Assessment
✅ Low: The fix round now sends ring_mailbox through the same ring_line → submit_ring path the ring subcommand uses, with its stash refusal and payload re-check before each Enter. The outcome mapping is unchanged: rung or rung-unconfirmed stops the ring, not-rung retries, and no proven primary (rc 1) stops it. The ring subcommand still prints the same line and exits 0, and the new mailbox race test would fail against the old code, both by submitting a draft and by typing over a stash. I found no remaining defects in the rest of the change.
Testing
I ran the focused desk-voice suite (84 checks, including the Swift recording tests; all passed), then drove the change live. Every setup was disposable: lab FM_HOMEs, a stand-in Deepgram, a real Claude primary on a private lab tmux socket, and the real build-only floater app. Those runs covered keeping and retrying a ten-minute note, dictation retry, the path guards, a failed delivery leaving the recording saved, the mailbox ring reaching a real primary and waiting out a draft, launch recovery with automatic retry (with screenshots of the floater status line), and a never-answering endpoint failing in bounded time. Everything passed. A fresh microphone capture failing while the floater runs was not driven live because it needs microphone and Accessibility permissions this session doesn't have. All lab homes, helper processes and the desk-floater/.build output were removed afterwards.
retrygives back its words and then removes itrecordingswas empty afterwards. live-floater-recovery-transcript.txt: the…Evidence: Live floater recovery and auto-retry transcript
Source: Live floater recovery and auto-retry transcript
Evidence: CLI keep/recordings/retry of a 10-minute note (Deepgram down, then empty, then back)
Source: CLI keep/recordings/retry of a 10-minute note (Deepgram down, then empty, then back)
Evidence: CLI dictation retry and guard cases
Source: CLI dictation retry and guard cases
Evidence: Mailbox ring reaching a real lab Claude primary, which drains the note
Source: Mailbox ring reaching a real lab Claude primary, which drains the note
Evidence: Mailbox ring waits while a draft is in the chat box, then rings once it is cleared
Source: Mailbox ring waits while a draft is in the chat box, then rings once it is cleared
Evidence: Transcription request against a never-answering endpoint fails in bounded time
Source: Transcription request against a never-answering endpoint fails in bounded time
Pipeline
Updates from git push no-mistakes
✅ **intent** - passed
✅ No issues found.
🔧 **Rebase** - 2 issues found → auto-fixed ✅
.agents/skills/operational-home-layout/SKILL.md- merge conflict rebasing onto origin/maindocs/desk-floater.md- merge conflict rebasing onto origin/main🔧 Fix applied.
✅ Re-checked - no issues remain.
🔧 **Review** - 1 issue found → auto-fixed ✅
bin/fm-desk-voice.sh:328- ring_mailbox callsPRIMARY_SUBMIT_COMPOSER=empty primary_submit "$line"withoutPRIMARY_SUBMIT_RING=1, so the delayed mailbox ring goes through the ordinary send path:fm_backend_send_text_submit, then fm_tmux_submit_core, which types and then retries Enter up to 3 times. It does not usesubmit_ring. That leaves out the two protections the header and docs say a ring has. (1) Before every ring Enter, including retries, the box must still show only the ring's literal payload. (2) A ring leaves a Claude stash alone; the send path types over a stash with tries=1. The bad case is likely because this ring is unsolicited and runs detached for about 4 minutes (delays 0 2 5 10 20 30 60 60 60). Sequence: a mailbox message is queued and the captain starts typing a reply in the primary chat just as a ring fires. The composer reads empty, the ring line is typed, the captain's keystrokes land in the same box, and Enter submits '[firstmate desk-voice] … <half-typed thought>' as one message. A retried Enter can also submit a draft that reappeared. docs/desk-floater.md:226 says the ring works 'the way a captain inbox note does' and is 'never over … your draft'. The inbox-note ring (bin/fm-inbox.sh:389) goes throughfm-desk-voice.sh ring, which has these proofs, and test_ring_checks_the_payload_before_each_enter covers only that path. Fix: have ring_mailbox use the existingringfunction (or set PRIMARY_SUBMIT_RING=1 with the same verdict handling: rung/rung-unconfirmed stop, not-rung retries, rc 1 stops). This also removes the second copy of the ring rule. Add a race case for the mailbox ring.🔧 Fix applied.
✅ Re-checked - no issues remain.
✅ **Test** - passed
✅ No issues found.
retrygives back its words and then removes itrecordingswas empty afterwards. live-floater-recovery-transcript.txt: the…bash tests/fm-deepgram-desk.test.sh(focused desk-voice suite, includes the Swift RecordingTests): 84 ok, exit 0Live CLI in a disposable lab home with a stand-in Deepgram (FM_DEEPGRAM_CURL):fm-desk-voice.sh keepof a 19,200,044-byte 10-minute WAV, thenrecordings, thenretrythree times (Deepgram down, empty transcript, Deepgram back) ending in a mailbox deliveryLive CLI: dictationkeep --purpose dictatethenretryprinted atranscriptline, sent nothing to the mailbox, and removed the recordingLive CLI guards:retryof a file outside unsent/ (including a ../ path) refused with exit 2;keepof a non-audio file and of a symlink refused; with the mailbox path replaced by a plain file, the retry failed and the recording stayed savedLive mailbox ring: a realclaudeprimary in a lab home on the privatetmux -L fm-labsocket;fm-desk-voice.sh delivertyped the ring line within seconds and the primary ran fm-wake-drain.sh andfm-desk-voice.sh drainLive adversarial ring: a draft typed withtmux send-keys -lwas left alone, nothing was submitted for 12s, then after Ctrl-U cleared it the ring landed on its next try about 21s laterLive floater: built withbin/fm-desk-floater.sh --build-onlyand ran the DeskFloater binary with FM_HOME=lab. Launch recovery: the cut-off WAV and the dictation were saved, the 0.2s stray was deleted, and the in-progress capture was left alone. Automatic retries at +20s (Deepgram down) and +60s (delivered). Relaunch recovered the capture left alone earlierscreencapture -l <floater window>of the floater status line: 'Saved - retrying', then 'Saved for Firstmate'Livebin/fm-deepgram-stt.shwith real curl routed through--connect-toto a local socket that accepts and never answers: three bounded attempts, then exit 1 with the bound named, after 52s✅ **Document** - passed
✅ No issues found.
✅ **Lint** - passed
✅ No issues found.
✅ **Push** - passed
✅ No issues found.