Repository navigation
feat(transcription): offer Voxtral Mini Transcribe 3 and Qwen3 ASR on OpenRouter - #544
Conversation
gpt-4o-transcribe-diarize rejects requests with more than 1400 s of audio, but MP3 chunks were split by size alone. Any MP3 under about 120 kbps put more than 1400 s in a 20 MB chunk, so every chunk failed. For diarization, split MP3 at frame boundaries into the fewest equal parts that stay under both 20 MB and 1400 s. Whisper keeps its size-only chunks. Fixes #532
A chunk that OpenAI rejected with a status such as 400 or 401 was sent three times, although each send fails the same way. A long diarized episode went up three times before the error showed. Retry only the statuses openai-node retries: 408, 409, 429 and 5xx, plus network errors. Refs #532
gpt-4o-transcribe-diarize takes up to 0.58 s per second of audio, measured over 81 requests, so the 1,072 s parts a 54 min episode got would take about 10 min each, at the request timeout. Parts are now at most 600 s, which the model answers in under 6 min. Refs #532
Each test passes on the hand-written code. Together they catch wrong derivations that no existing test sees: the -1 SecretStorage ID suffix that released versions accept, the provider an export error names, the kind a settings key row adopts a secret for, and the key migration notices.
CREDENTIALS in src/types/Credentials.ts holds each kind's settings field, export key and label. The credential repository, settings decoder, settings transfer, settings tab and key migration notice loop over it. IPodNotesSettings takes its key fields from it, so a kind with no default in DEFAULT_SETTINGS fails typecheck. The plaintext key map from before 2.19.0 stays fixed.
The published transcript renderer is the reader of the format that stays when the diarization folder goes. The default template now lives only in DEFAULT_SETTINGS, so src/constants.ts imports no service code.
src/services/transcription/ turns an episode into a transcript with any provider. A provider is one file: its models, its request, and how to read its answers. OpenAI (gpt-transcribe) and Deepgram (nova-3) are the first two. The engine owns what providers share: - audioSource.ts decides whether a provider may be given the episode's address. Only an address that a saved public feed lists, that carries no credential, and that names a public host passes. All other audio is uploaded. - split.ts is audioChunker.ts with the caps as arguments and a start time on each part. - http.ts sends a request at most three times, and only after a 429, a 5xx, or a lost connection. Its failure sentences never include the provider's own text, which can quote a private address. - cues.ts turns words or text into the shapes of a published transcript. Nothing calls the engine yet. Lint rejects an import of the folder's files from outside it, except its index.
TranscriptionService keeps its queue, the published-transcript path, the progress notice, and note saving. It calls the engine for audio. - OpenAI transcribes with gpt-transcribe. whisper-1 and gpt-4o-transcribe-diarize, which OpenAI shuts down on 2027-02-26, are gone. A tooling test searches the built main.js for them. - With speaker labels on and OpenAI selected, the transcript is saved without labels. A notice that stays until clicked names the providers that label speakers. - Deepgram transcripts go through renderTranscriptBody, so each paragraph has a timestamp. Speakers are numbered in the order they first speak. - Deepgram gets the address of a public episode and downloads it itself. - A failure notice stays until clicked. Parts go out one at a time, and a 401, 402, 404, or 400 stops the job before the next part. Deletes src/services/diarization/, openaiTranscription.ts, the Whisper worker pool, and createBinaryChunkFiles. getEpisodeAudioBuffer no longer returns a basename, because uploads are named for their format only.
net.mjs answers OpenAI with { text } and Deepgram with the words of two
speakers, and logs the content type of each request. fixtures.mjs serves a
feed at priv3.supercast.com, an address that PodNotes keeps private. The
transcripts recipe covers OpenAI in parts, and Deepgram by address and by
upload.
The marketplace review test called TranscriptionService.waitForRetry, which the engine replaced. An aborted run now rejects through waitForLifecycle, so the test aborts a pending wait there and expects the same Error.
An aborted job no longer downloads the episode or decodes it into parts. Until now only its requests were refused. A 429 or a 5xx that asks to wait over a minute is sent once. A sooner send would only upload the audio to be refused again.
… feat/transcription-providers
A provider's name was written twice: once in its credential row and once in its provider file. The row now owns it, so a provider file holds only its models and its request. Each model gains the name the settings will show.
The provider was derived from the speaker settings: Deepgram with speaker labels on, OpenAI otherwise. It is now stored as transcript.provider, with transcript.model beside it, where "" follows the provider's recommended model. schemaVersion stays 2 and both fields are additions, so a device on 2.24.0 keeps loading the library and keeps the fields on its own saves. A library that has no transcript.provider gets the provider it ran with: Deepgram for speaker labels with Deepgram, with or without its key, and OpenAI otherwise. The migration reads only the stored document, adds no decode warning, and gives the same answer on every load until a save stores it. transcript.diarization.provider belongs to versions up to 2.24.0. This version reads it for that migration alone, never writes it, keeps it in data.json as found, and leaves it out of settings exports. Importing a 2.24.0 export that labels speakers with Deepgram selects Deepgram. A provider or model ID this version does not have is kept as stored and shown under that name. An unknown provider refuses to transcribe with a notice. An unknown model runs the recommended model and the notice says so. The Transcripts settings now read: Transcription provider, the chosen provider's API key, Model, Speaker labels, Speaker label format, then the path and template. Speaker labels is disabled for a model that cannot label speakers. The Diarization provider row is gone.
ElevenLabs is a credential row, one provider file and a registry line. Its key reference is the new root field elevenLabsSecretId, stored in SecretStorage as podnotes-elevenlabs-api-key, and a settings export carries the key as secrets.elevenLabs. The export envelope stays at version 2. A public episode goes to ElevenLabs by its address, so nothing is downloaded. Any other episode is uploaded in one request. Word times and speakers reach the note, and turning speaker labels off asks for none. ElevenLabs answers 401 for a used-up quota as for a rejected key, and 400 for a model the key may not use as for an address it could not download, so the provider reads detail.code. Only the last of these is followed by an upload. A request waits 10 minutes for its answer. A new install starts on ElevenLabs with speaker labels on. A library saved up to 2.24.0 that holds no OpenAI key and has speaker labels off moves to ElevenLabs too, because it never set transcription up. A library that never stored the speaker labels choice keeps them off. The notice for a model without speaker labels now names ElevenLabs first.
A journey now seeds its transcription provider with its key, because a new vault is on ElevenLabs. The OpenAI journeys choose OpenAI. A new journey runs the default: a public episode goes to ElevenLabs by its address, no audio is fetched, and the note has timestamps and speaker labels. The ElevenLabs stub answers with the words ElevenLabs returns and records what each request named. The settings tab test walks the new Transcripts rows, and the key recovery test links the ElevenLabs key, the one a new vault shows.
The fixture provider answers api.elevenlabs.io with ElevenLabs' words, two speakers with the second 12 s in, and logs the address a request named in place of a file. The transcript guide follows the new settings rows and adds the ElevenLabs runs and a 2.24.0 library check.
…choice The provider row decided for itself whether the stored provider is one this version has, and the Speaker label format row read the stored speaker setting. Both now ask transcriptionChoice, as the key rows, the model row and the Speaker labels row do. The engine no longer exports a type nothing imports.
The Transcripts tests looked each control up by a CSS selector inside its row and asserted on a hidden row's inline style. They now query by role and by the row's name, as a user or a screen reader finds a control, so a hidden row is one the query does not find.
…ings measure The transcription engine left main.js at 103,795 gzip bytes. The provider and model settings, their migration and the ElevenLabs provider add 866, which takes it to 104,661 and over the 104,000 budget. The budget rises by those 866 bytes to 104,866, so the 205 bytes of headroom stay as they were.
…m library 2.24.0 imports the diarization object whole and fills a missing provider with OpenAI, so omitting the field moved Deepgram libraries to OpenAI. Imports here keep the provider this library stored.
A library migrated with OpenAI speaker labels on could not turn them off, so every transcription repeated the notice.
… one The tests for a provider and a model a newer PodNotes stored used OpenRouter and Grok STT 2.0, which this version is about to run. They now use AssemblyAI and universal-3-6-pro.
OpenRouter is a credential row, one provider file and a registry line. Its key reference is the new root field openRouterSecretId, stored in SecretStorage as podnotes-openrouter-api-key, and a settings export carries the key as secrets.openRouter. It offers three models, each with the request measured to give word times and speakers: - Grok STT 2.0, recommended: base64 JSON with verbose_json and word and segment times, routed to xAI with provider.only, which takes diarize from provider.options.xai. A request routed elsewhere is answered with no speakers. - ElevenLabs Scribe v2: multipart with diarize beside the model, or JSON with input_audio.url for an episode a saved public feed lists. - MAI-Transcribe 2: multipart with diarize beside the model. With speaker labels off, no request carries diarize. Sounds Scribe v2 lists as audio_event items are left out. OpenRouter takes 25 MB a request and gives the model 60 s, so an upload goes in parts of at most 18 MB and 1,800 s. Parts of 24 MB were answered in 20 to 31 s. A 402 says the account is out of credit, a 401 that the key was rejected, and a 404 that the model is not offered. A 404 whose message names the account's data policy or zero data retention says the model is not offered under the account's OpenRouter privacy settings. Each ends the job at the first part, and none quotes OpenRouter.
…t each part A note saved with speaker labels asked for and none returned now says so: "Saved without speaker labels, because Grok STT 2.0 returned none." OpenRouter answers 200 with no speakers when a request reaches a route that ignores the speaker option, so this is checked for every provider. Labels from two parts already name different speakers. An episode sent in parts now also says that its labels restart in each part, so one person can have several labels, until labels are merged across parts.
…ll lists PodNotes reads OpenRouter's public catalog of transcription models, without the key, when the Model row draws and when an OpenRouter job starts. It keeps the read in memory for 24 hours. A read that fails is not kept, so the next one asks again, and the tested list is offered as it is. The catalog only takes models away. A model it does not list is not offered, and one whose expiration_date has come is skipped. A model with a later end date reads "Grok STT 2.0, ends <date>" in the Model row. A catalog that would leave no model is not followed. A chosen model the catalog no longer lists stays selected in the Model row as "<label>, no longer offered". A job runs the first offered model instead and says "OpenRouter no longer offers ElevenLabs Scribe v2, so PodNotes used Grok STT 2.0." The row redraws when a read lands while it is open. A provider gets its catalog through an optional catalog() beside failure(), so the registry owns the cache and the filter for any provider that has one.
A stub answers OpenRouter's transcriptions with two speakers, the second 12 s in, and its model catalog with the three models PodNotes offers. It records what each request named and a hash of the audio it carried, decoded from base64 for JSON. J-TR-09 plays a public episode on Scribe v2: one JSON request names the episode's address, and the episode's audio is never fetched. J-TR-10 plays it on the recommended Grok STT 2.0: one JSON request carries the audio as base64, the same bytes the fixture server holds. Both notes have timestamps and numbered speakers, and the catalog is read once, with no Authorization. A journey seeds any provider by its credential row, and a model with it.
…eads them The fixture provider answers openrouter.ai transcriptions with verbose_json words from two speakers, the second 12 s in, and the model catalog with the three models PodNotes offers. The log keeps openrouter.ai paths and reads the address a JSON request names in input_audio.url. The transcript guide adds the OpenRouter rows, its runs (Scribe v2 by address, Grok by base64, a long MP3 in parts) and the notices that now name OpenRouter.
|
FAIL
|
…igit A mark hidden behind an emoji, as Hello…🙂 or 你好。🙂, slipped past the already-punctuated checks, and the word gained it twice. It also reached Grok, Scribe v2 and MAI, which share the reader. A word now gains an opening mark only if it starts with a letter or digit, and closing punctuation only if it ends with one, so a word with any mark, symbol or emoji at that end keeps it as given.
|
On the d8099b5 verdict: fixed in c2b1c7c with one rule instead of more mark sets. A word gains an opening mark only if it starts with a letter or digit, and closing punctuation only if it ends with one. A word that has any mark, symbol or emoji at that end keeps it as given, so no mark can be added twice, whatever script or emoji is beside it. That covers Qwen, and Grok, Scribe v2 and MAI too, since all four use this reader.
|
The shared verbose_json reader adds marks only where a word ends in a letter or digit, as #544 now does.
|
FAIL
|
Emoji such as 1️⃣ and ℹ️ start with a digit or letter, so a word like 1️⃣"Hello passed the edge check and gained a second quote, with Grok, Scribe v2 and MAI too. Only a word of letters and digits, with apostrophes and hyphens between them, now gains marks; any other word keeps its text as given.
|
On the c2b1c7c verdict: fixed in 360cccb. Checking a word's ends kept losing to emoji that begin with a letter or digit, so the rule is now about the whole word. Only a plain word, made of letters and digits with apostrophes or hyphens between them, gains marks. Any word with a mark, a symbol or an emoji anywhere keeps its text as given. No mark can be added beside an emoji, keycap or not, for Qwen, Grok, Scribe v2 or MAI.
|
The shared verbose_json reader adds marks only to plain words, as #544 now does.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 360cccb. Configure here.
Plain words took letters and digits only, so 3.5, $5, 3:30, 5% and 1,000 kept no comma or period from the text. A word now may also hold a number's .,:/ between digits, a currency sign before and % after, none of which is a mark it can gain twice.
The shared verbose_json reader gives numbers, amounts, times and percentages their punctuation, as #544 now does.
|
FAIL
|
…r end A model that attached a mark to the start of the next word, as Hello ,world, slipped past the check for punctuated answers, and the plain word before it took the same mark from the text. Any word with a mark before its first letter or digit or after its last now leaves the whole answer as given. Apostrophes do not count, so dogs' still gets its sentence's punctuation. The opening and closing sets are now written once.
|
On the dea6eba verdict: fixed in 474c515. A model that attaches a mark to either end of a word punctuates its own words, so the check for an already punctuated answer now looks at both ends of every word. Any word with a quote, bracket, dash or sentence mark before its first letter or digit or after its last leaves the whole answer as given. That covers
|
The shared verbose_json reader restores nothing when any word has a mark at either end, as #544 now does.
|
FAIL
|
Checking each word's edges kept missing shapes that hid a mark from the check, such as a keycap emoji before a comma or a single quote on the word before, and a plain neighbor then took the same mark from the text. Marks now come from the text only when every word is plain: letters and digits, with apostrophes, hyphens or a number's .,:/ between them. Then no word has a mark of its own, and each token's marks go to one word, so none can be added twice. An answer with any other word is kept as the model gave it.
|
On the 474c515 verdict: fixed in 648738c by changing the rule rather than patching another shape. Marks now come from the text only when every word is plain: letters and digits, with apostrophes, hyphens or a number's
|
The shared verbose_json reader restores punctuation only when no word has any, as #544 now does.
|
FAIL
|
…igits The plain pattern let a comma, period, colon or slash sit between letters, so world,again counted as plain, kept its comma, and the word before it took the same comma from the text. Those separators now belong to numbers alone.
|
On the 648738c verdict: fixed in c245255. The plain pattern now has two shapes: letters and digits with apostrophes or hyphens between them, or a number with |
The shared verbose_json reader counts .,:/ as part of a plain word only between digits, as #544 now does.
A word with a combining mark, as a decomposed é or any Hindi word, failed the plain pattern, so its whole answer got no punctuation from the text. Letters now include combining marks, in the pattern and the matching keys, and the Devanagari danda joins the closing marks.
The shared verbose_json reader counts combining marks as part of a plain word, as #544 now does.
|
PASS+NOTES
|

Why
Users should be able to pick from current transcription models without anyone landing on a worse or retired one. This PR adds the two plain OpenRouter models (no speaker labels) that measured best this week against NPR human transcripts.
Summary
The OpenRouter
Modelrow now lists five models, in this order: Grok STT 2.0 (recommended), ElevenLabs Scribe v2, MAI-Transcribe 2, Voxtral Mini Transcribe 3, Qwen3 ASR 1.7B.mistralai/voxtral-mini-transcribe-3-0model,file,response_format=verbose_json, notimestamp_granularities(it returns 400 "Provider returned 400")segments[]withstart,end,text, nowordsqwen/qwen3-asr-1.7bmodel,file,response_format=verbose_json,timestamp_granularities[]=wordwords[]PartTranscriptgains a{ segments }variant. The wire shapes stay insideopenrouter.ts.Speaker labelsrow reads " does not label speakers." and the note is saved without labels, with a notice that stays on screen.master, so no released wording changes.Stacked on #542.
Verification
Commands, all with Node 22.23.2 through
fnm exec:npm run lint,npm run format:check,npm run typecheck,npm run build,npm run test(124 files, 2,894 tests),npm run test:tooling(48 tests),npm run docs:build: all exit 0.npm run test:e2e -- tests/e2e/journey-resilience.test.ts -t "J-TR-": 11 passed, including the new J-TR-11 (Voxtral) and J-TR-12 (Qwen3 ASR) on Obsidian 1.14.4.npm run size: gzip 107,084 bytes. The base commit 5eebf37, built from agit archiveexport with the same command, measured 106,873, so this PR adds 211 gzip bytes (728 raw). The gzip budget goes up by exactly those 211 bytes, from 106,970 to 107,181, which leaves the same 97 bytes of headroom as before.scripts/size-baseline.mdrecords this.Planted defects, each run with
npm run plant. Every one failed the named test and passed again once restored:timestamp_granularities[]=wordmistralai/voxtral-mini-transcribe-3-0 gets the upload as its measured multipart request, ...(unit), and J-TR-11 in real Obsidian, where the stub answers it with the measured 400timestamp_granularities(uses Voxtral's request)qwen/qwen3-asr-1.7b gets the upload as its measured multipart request, ...("words" in transcript ? start : 0) + item.start)Voxtral's segments from two parts of at most 900 s become paragraphs timed from the episode's startan answer with neither words nor segments writes no noteisTimedchecks forwordsonly)choosing it shows its API key row and its five models, Grok STT 2.0 recommendedspeakers: truechoosing mistralai/voxtral-mini-transcribe-3-0 says Speaker labels are not available, ...choice.label)with speaker labels on is saved without them, and the notice names the providers that label speakers and staysRuntime check in this worktree's isolated Obsidian 1.14.4 on Linux (headless, Xvfb), using a fresh vault and a fake key. The
.claude/skills/verify/net.mjsfixture provider answered every request, so no real key was involved. The local file wasFixtures/Two Voices.mp3, 70 min of 8 kbps mono, written with the recipe'sffmpegline (4,200,452 bytes). For each model I chose it in Settings > PodNotes >Modelwith the keyboard, leftSpeaker labelson, played the file throughPlay with PodNotesin the file menu, and ranTranscribe current episode from audiofrom the palette.multipart/form-dataPOSTs of 840,311 to 840,715 bytes (840 s each), one after the other, and no reel. The fixture answers a word-times request with a 400, and the note was saved, so the request did not ask for word times. The note has paragraphs at00:00:00,00:00:12,00:14:00(time=840.096),00:14:12,00:28:00and so on up to00:56:12, with no speaker labels. Clicking00:28:12moved the player to 1,693.66 s, 1.5 s past the link's 1,692.192 while it kept playing. The notice read "Saved without speaker labels, because Voxtral Mini Transcribe 3 does not label speakers. These providers do: ElevenLabs, Deepgram, OpenRouter."dev:errors: none.timestamp_granularities[]field. The note times match Voxtral's, and the notice names Qwen3 ASR 1.7B.data.jsonstoredtranscript.model: "qwen/qwen3-asr-1.7b".Two things in the screenshots come from elsewhere. The warning icon on the key is Obsidian's own "Secrets are not encrypted", because headless Linux has no keyring. The doubled "Two Voices" is Obsidian's inline title shown above the template's
# {{title}}.I have not run this against the real OpenRouter API. The lead verifies with a real key.
Real OpenRouter key (lead, after review)
Obsidian 1.14.4, Linux, palette command, speaker labels off, a 20 min local clip of E231 (expect 2 parts at 900 s), a pass-through request log:
wordscarry no punctuation (0 sentence ends in 3,785 words on Planet Money), while itstextdoes (322).dev:errorsnone.Review fix (lead)
text, matched in order with three tokens of lookahead, only when no word carries punctuation. Plants: keep the bare word on a match; one token of lookahead. Each is caught byQwen's words, which it lists without punctuation, take their punctuation from its text.Release impact
There are no settings or storage changes.
transcript.modelcan now hold two more OpenRouter IDs. An older PodNotes that reads them treats them as unknown and runs its recommended model, which is existing behavior. The speaker-label notice now names the model instead of the provider, and that notice has not been released yet.Note
Add Voxtral Mini Transcribe 3 and Qwen3 ASR 1.7B to OpenRouter transcription
SegmentinterfacesegmentOfMacroscope summarized c69d365.
Note
Medium Risk
Touches transcription provider requests, response parsing, and note formatting for new OpenRouter models; no auth or storage schema changes, but incorrect parsing or part limits could affect saved transcripts.
Overview
Adds Voxtral Mini Transcribe 3 and Qwen3 ASR 1.7B to the OpenRouter transcription model list, each with smaller upload caps (12 MB / 900 s), no speaker labels, and multipart-only audio (no URL transcription).
OpenRouter provider gains model-specific request shapes (Voxtral: segment times only—word-time requests fail; Qwen: word times) and parsing for segment-based answers plus Qwen punctuation restoration from response
textonto bare words. Transcript assembly (cues.ts, types) now builds timed notes from segments as well as words, with the same episode-offset timing across parts.When speaker labels are enabled but the model cannot diarize, the completion notice names the model (e.g. GPT Transcribe) instead of the provider. Bundle size budgets rise to match the added logic (~546 raw / ~781 gzip bytes). E2E fixtures, verify docs, and broad unit/journey tests cover the new models and Qwen edge cases.
Reviewed by Cursor Bugbot for commit c69d365. Bugbot is set up for automated code reviews on this repo. Configure here.