Repository navigation
feat(transcription): choose a provider and model, with ElevenLabs as the default - #539
Merged
Merged
Conversation
gpt-4o-transcribe-diarize rejects requests with more than 1400 s of audio, but MP3 chunks were split by size alone. Any MP3 under about 120 kbps put more than 1400 s in a 20 MB chunk, so every chunk failed. For diarization, split MP3 at frame boundaries into the fewest equal parts that stay under both 20 MB and 1400 s. Whisper keeps its size-only chunks. Fixes #532
A chunk that OpenAI rejected with a status such as 400 or 401 was sent three times, although each send fails the same way. A long diarized episode went up three times before the error showed. Retry only the statuses openai-node retries: 408, 409, 429 and 5xx, plus network errors. Refs #532
gpt-4o-transcribe-diarize takes up to 0.58 s per second of audio, measured over 81 requests, so the 1,072 s parts a 54 min episode got would take about 10 min each, at the request timeout. Parts are now at most 600 s, which the model answers in under 6 min. Refs #532
Each test passes on the hand-written code. Together they catch wrong derivations that no existing test sees: the -1 SecretStorage ID suffix that released versions accept, the provider an export error names, the kind a settings key row adopts a secret for, and the key migration notices.
CREDENTIALS in src/types/Credentials.ts holds each kind's settings field, export key and label. The credential repository, settings decoder, settings transfer, settings tab and key migration notice loop over it. IPodNotesSettings takes its key fields from it, so a kind with no default in DEFAULT_SETTINGS fails typecheck. The plaintext key map from before 2.19.0 stays fixed.
The published transcript renderer is the reader of the format that stays when the diarization folder goes. The default template now lives only in DEFAULT_SETTINGS, so src/constants.ts imports no service code.
src/services/transcription/ turns an episode into a transcript with any provider. A provider is one file: its models, its request, and how to read its answers. OpenAI (gpt-transcribe) and Deepgram (nova-3) are the first two. The engine owns what providers share: - audioSource.ts decides whether a provider may be given the episode's address. Only an address that a saved public feed lists, that carries no credential, and that names a public host passes. All other audio is uploaded. - split.ts is audioChunker.ts with the caps as arguments and a start time on each part. - http.ts sends a request at most three times, and only after a 429, a 5xx, or a lost connection. Its failure sentences never include the provider's own text, which can quote a private address. - cues.ts turns words or text into the shapes of a published transcript. Nothing calls the engine yet. Lint rejects an import of the folder's files from outside it, except its index.
TranscriptionService keeps its queue, the published-transcript path, the progress notice, and note saving. It calls the engine for audio. - OpenAI transcribes with gpt-transcribe. whisper-1 and gpt-4o-transcribe-diarize, which OpenAI shuts down on 2027-02-26, are gone. A tooling test searches the built main.js for them. - With speaker labels on and OpenAI selected, the transcript is saved without labels. A notice that stays until clicked names the providers that label speakers. - Deepgram transcripts go through renderTranscriptBody, so each paragraph has a timestamp. Speakers are numbered in the order they first speak. - Deepgram gets the address of a public episode and downloads it itself. - A failure notice stays until clicked. Parts go out one at a time, and a 401, 402, 404, or 400 stops the job before the next part. Deletes src/services/diarization/, openaiTranscription.ts, the Whisper worker pool, and createBinaryChunkFiles. getEpisodeAudioBuffer no longer returns a basename, because uploads are named for their format only.
net.mjs answers OpenAI with { text } and Deepgram with the words of two
speakers, and logs the content type of each request. fixtures.mjs serves a
feed at priv3.supercast.com, an address that PodNotes keeps private. The
transcripts recipe covers OpenAI in parts, and Deepgram by address and by
upload.
The marketplace review test called TranscriptionService.waitForRetry, which the engine replaced. An aborted run now rejects through waitForLifecycle, so the test aborts a pending wait there and expects the same Error.
An aborted job no longer downloads the episode or decodes it into parts. Until now only its requests were refused. A 429 or a 5xx that asks to wait over a minute is sent once. A sooner send would only upload the audio to be refused again.
… feat/transcription-providers
A provider's name was written twice: once in its credential row and once in its provider file. The row now owns it, so a provider file holds only its models and its request. Each model gains the name the settings will show.
The provider was derived from the speaker settings: Deepgram with speaker labels on, OpenAI otherwise. It is now stored as transcript.provider, with transcript.model beside it, where "" follows the provider's recommended model. schemaVersion stays 2 and both fields are additions, so a device on 2.24.0 keeps loading the library and keeps the fields on its own saves. A library that has no transcript.provider gets the provider it ran with: Deepgram for speaker labels with Deepgram, with or without its key, and OpenAI otherwise. The migration reads only the stored document, adds no decode warning, and gives the same answer on every load until a save stores it. transcript.diarization.provider belongs to versions up to 2.24.0. This version reads it for that migration alone, never writes it, keeps it in data.json as found, and leaves it out of settings exports. Importing a 2.24.0 export that labels speakers with Deepgram selects Deepgram. A provider or model ID this version does not have is kept as stored and shown under that name. An unknown provider refuses to transcribe with a notice. An unknown model runs the recommended model and the notice says so. The Transcripts settings now read: Transcription provider, the chosen provider's API key, Model, Speaker labels, Speaker label format, then the path and template. Speaker labels is disabled for a model that cannot label speakers. The Diarization provider row is gone.
ElevenLabs is a credential row, one provider file and a registry line. Its key reference is the new root field elevenLabsSecretId, stored in SecretStorage as podnotes-elevenlabs-api-key, and a settings export carries the key as secrets.elevenLabs. The export envelope stays at version 2. A public episode goes to ElevenLabs by its address, so nothing is downloaded. Any other episode is uploaded in one request. Word times and speakers reach the note, and turning speaker labels off asks for none. ElevenLabs answers 401 for a used-up quota as for a rejected key, and 400 for a model the key may not use as for an address it could not download, so the provider reads detail.code. Only the last of these is followed by an upload. A request waits 10 minutes for its answer. A new install starts on ElevenLabs with speaker labels on. A library saved up to 2.24.0 that holds no OpenAI key and has speaker labels off moves to ElevenLabs too, because it never set transcription up. A library that never stored the speaker labels choice keeps them off. The notice for a model without speaker labels now names ElevenLabs first.
A journey now seeds its transcription provider with its key, because a new vault is on ElevenLabs. The OpenAI journeys choose OpenAI. A new journey runs the default: a public episode goes to ElevenLabs by its address, no audio is fetched, and the note has timestamps and speaker labels. The ElevenLabs stub answers with the words ElevenLabs returns and records what each request named. The settings tab test walks the new Transcripts rows, and the key recovery test links the ElevenLabs key, the one a new vault shows.
The fixture provider answers api.elevenlabs.io with ElevenLabs' words, two speakers with the second 12 s in, and logs the address a request named in place of a file. The transcript guide follows the new settings rows and adds the ElevenLabs runs and a 2.24.0 library check.
…choice The provider row decided for itself whether the stored provider is one this version has, and the Speaker label format row read the stored speaker setting. Both now ask transcriptionChoice, as the key rows, the model row and the Speaker labels row do. The engine no longer exports a type nothing imports.
The Transcripts tests looked each control up by a CSS selector inside its row and asserted on a hidden row's inline style. They now query by role and by the row's name, as a user or a screen reader finds a control, so a hidden row is one the query does not find.
…ings measure The transcription engine left main.js at 103,795 gzip bytes. The provider and model settings, their migration and the ElevenLabs provider add 866, which takes it to 104,661 and over the 104,000 budget. The budget rises by those 866 bytes to 104,866, so the 205 bytes of headroom stay as they were.
…m library 2.24.0 imports the diarization object whole and fills a missing provider with OpenAI, so omitting the field moved Deepgram libraries to OpenAI. Imports here keep the provider this library stored.
A library migrated with OpenAI speaker labels on could not turn them off, so every transcription repeated the notice.
Deploying podnotes with
|
| Latest commit: |
c6adf83
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://60ea47f1.podnotes.pages.dev |
| Branch Preview URL: | https://feat-transcription-providers.podnotes.pages.dev |
chhoumann
marked this pull request as ready for review
November 14, 2026 14:49
…unted last frame Parts were counted from rounded seconds, so an episode a frame over a whole number of capped parts got equal parts one frame too long, and the time cap then split off a final part of a single frame. Parts are now counted in frames. A last frame followed by metadata, such as an ID3v1 tag, failed the next-frame check and was not counted, so the last part could run one frame over maxSeconds. A frame that continues the frames before it now counts.
…ame parts Summing frame durations drifts past the cap: 25,000 frames of 24 ms sum to 600.0000000000218 s, so the per-frame seconds check cut each part a frame early and a 49,999-frame episode went as 24,999 + 24,999 + 1 frames. Parts are already counted in frames, which keeps them under maxSeconds, so the check goes.
The planner fixed a frame count per part and checked the size cap separately. When a leading tag or a variable bitrate put an equal split over the cap, the size check cut a part short and the next kept the old frame count, leaving a one-frame part that cannot be decoded: 43,690 frames of a 320 kbps MP3 with a 512-byte tag went as 21,844 + 21,845 + 1. The planner now takes the fewest equal splits that fit both caps, adding a part when an equal split does not fit.
Equal splits put the odd frame first, so a 512-byte tag and 43,689 frames of a 320 kbps MP3 put the first of two parts 192 bytes over 20 MiB, and the planner sent three where two fit. Each cut now takes an equal share of the frames still to place and steps back while the part is over the size cap, so 21,844 + 21,845 goes in two.
Cuts only moved back from an even split, so where the bitrate rose a part was added although moving the cut forward fit: 21,847 low-bitrate frames then 21,845 at 320 kbps and an ID3v1 tag went as three parts, not two. The planner now finds the fewest parts by filling each as far as it fits, then places each cut as close to an equal share as the parts before and after it allow. A property test checks the count against a brute-force minimum on 300 random variable-bitrate streams.
Frame-like bytes inside a leading tag set the first frame's duration, and counting every part in that frame's length sent a 1,728 s episode of 72 ms frames up in one request. Parts now fit by the summed duration of their own frames, and each part's start is the sum of the frames before it.
Audio frames inside a tag's declared size were counted and could be cut through, sending an undecodable first part, and frame-like bytes in a trailing tag added a part.
…engine # Conflicts: # src/services/TranscriptionService.test.ts # src/services/TranscriptionService.ts # src/services/audioChunker.test.ts # src/services/audioChunker.ts # src/services/diarization/openaiProvider.ts # src/services/diarization/types.ts # src/services/openaiTranscription.test.ts # src/services/openaiTranscription.ts # tests/mocks/openaiFailures.ts
…h cap A file in a format other than MP3 under the size cap went whole whatever its length, so a long .m4a passed a model's maxSeconds. Its length now comes from its metadata through a media element, without decoding it, and audio over the cap is decoded into WAV parts that fit both caps. Audio of an unknown length still goes whole.
A public feed's enclosure with AWS or Google Cloud signing parameters passed the feed credential check, so Deepgram got the whole signed address. A query parameter whose name holds signature, credential or token, or Akamai's hdnts, now uploads the audio instead.
…nload A WebView may load no metadata for media it does not play, so the length probe could wait forever, and an unload could not stop it before a decode. The probe now reads the length as unknown after 5 s or when the job is aborted, and an aborted job decodes nothing.
…nscription-providers # Conflicts: # AGENTS.md # scripts/size-baseline.md # scripts/size.mjs # scripts/size.test.ts # src/persistence/settingsCodec.ts # src/settingsTransfer.ts # src/types/Credentials.ts # src/ui/settings/PodNotesSettingsTab.ts
Owner
Author
|
Retargeted to
|
Owner
Author
|
PASS+NOTES
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
PodNotes is moving transcription to current models from several providers, so that a provider is a credential row, one provider file and a registry line, and no user stays on a model OpenAI retires. This PR is step 3 of that plan. A user now chooses the provider and the model in settings, and ElevenLabs Scribe v3 is the default. It stacks on #538.
The diff includes #537, which is merged into this branch, until #537 lands.
Transcription provider, the chosen provider's API key,Model,Speaker labels,Speaker label format, then the path and template. Keys linked for other providers stay stored.Speaker labelsis disabled for a model that cannot label speakers, and the row says so.Diarization provideris gone.Recommended (<name>)follows the provider's first model, so a later release can move everyone who never picked one.scribe_v3gets the address of an episode that a saved public feed lists, so PodNotes downloads and uploads nothing. Any other episode is uploaded in one request. The note has a timestamp on each paragraph and numbered speakers.Not in this PR: OpenRouter, merged speaker labels across parts, and the docs, which describe the released version until the release.
Verification
Run on Linux with Node 22.23.4, on the final tree:
npm run lint,npm run format:check,npm run typecheck,npm run buildandnpm run docs:buildpass.npm run test: 124 files, 2,857 tests pass.npm run test:tooling: 8 files, 48 tests pass.npm run test:e2eon Obsidian 1.14.4: 33 files, 255 tests pass.PODNOTES_OBSIDIAN_APP=/opt/Obsidian/obsidian npm run test:e2e -- settings-tab journey-resilienceon Obsidian 1.13.7: 2 files, 21 tests pass.npm run plant: 52 planted defects, each caught by the unit test named below, and 2 more caught bynpm run test:e2e.Obsidian 1.14.4 on Linux under Xvfb, in this worktree's isolated vault, with the verify skill's fixture provider answering the three providers. No real key was used. The fixture feed is
http://podnotes.test:28062/feed.xmland the private one is atpriv3.supercast.com.data.jsonthen held the provider withmodel: "".Transcribe current episode from audiosent one request:POST https://api.elevenlabs.io/v1/speech-to-text, 604 bytes, naminghttp://podnotes.test:28062/episode-2.wav. The fixture server saw no request during the command. The note has**1:**at 0 s and**2:**at 12 s.urlSecretId. The command sentGET /p/episode-1.wav, then onePOSTof 960,649 bytes with no address in it.main.jswrote two libraries through its own settings tab. Its local build has the SHA-256 of the release asset,b7ba1dfe...0530484. This build loaded speaker labels with Deepgram, no Deepgram key and a linked OpenAI key as Deepgram, asked for a Deepgram key, and sent nothing. It loaded speaker labels with OpenAI as OpenAI, showed the disabledSpeaker labelsrow, saved the transcript without labels, and the notice was still up 20 s later.data.jsonwas byte-identical after each load.transcript.diarization.provider: "openai", which 2.24.0 adds for itself.transcript.provider,transcript.modelandelevenLabsSecretIdwere kept. This build then read its fields back unchanged.dev:errorsreported no error after each step. The lead verifies with real keys separately.Planted defects, each caught by the named test through
npm run plantsettingsMigrations.tssettingsMigrations.ts,persistence/settingsCodec.tssettingsMigrations.tssettingsMigrations.tssettingsMigrations.tssettingsMigrations.tspersistence/settingsCodec.tspersistence/settingsCodec.tspersistence/settingsCodec.tspersistence/settingsCodec.tssettingsMigrations.tssettingsMigrations.tsconstants.tsconstants.tsconstants.tssettingsTransfer.tssettingsTransfer.tssettingsTransfer.tssettingsTransfer.tssettingsTransfer.tssettingsTransfer.tsservices/transcription/registry.tsservices/TranscriptionService.tsservices/transcription/registry.tsservices/transcription/transcribe.tsservices/transcription/registry.tsservices/transcription/providers/deepgram.tsservices/transcription/cues.tsservices/transcription/providers/elevenlabs.tsservices/transcription/transcribe.tsservices/transcription/providers/elevenlabs.tsservices/transcription/providers/elevenlabs.tsservices/transcription/providers/elevenlabs.tsservices/transcription/providers/elevenlabs.tsservices/transcription/providers/elevenlabs.tsservices/transcription/providers/elevenlabs.tsservices/transcription/providers/elevenlabs.tsservices/transcription/providers/elevenlabs.tsservices/transcription/providers/elevenlabs.tsservices/transcription/providers/elevenlabs.tsui/settings/PodNotesSettingsTab.tsui/settings/PodNotesSettingsTab.tsui/settings/PodNotesSettingsTab.tsui/settings/PodNotesSettingsTab.tsui/settings/PodNotesSettingsTab.tsui/settings/PodNotesSettingsTab.tsui/settings/PodNotesSettingsTab.tsui/settings/PodNotesSettingsTab.tsui/settings/PodNotesSettingsTab.tsui/settings/PodNotesSettingsTab.tsconstants.tssettingsMigrations.tsservices/transcription/providers/elevenlabs.tsui/settings/PodNotesSettingsTab.tsRenaming the ElevenLabs row's
settingsKeyorexportKeyfailsnpm run typecheck, inconstants.tsand in the tests that passelevenLabs, so no plant covers those names.Real ElevenLabs key (lead, after review)
Obsidian 1.14.4, Linux, build of
1eef3e6f, palette command "Transcribe current episode from audio", speaker labels on, a pass-through request log (no fixture):https://feeds.npr.org/500005/podcast.xml: one 929-byte POST toapi.elevenlabs.io/v1/speech-to-textcarryingsource_url, no enclosure download. Saved in 5 s with timestamp links and speakers 1 to 4.dev:errors: none. Free-tier credits limited this to short audio; the lab measured a 54 min episode in about 30 s.Review fixes (lead)
diarization.provideris now"deepgram"exactly when the library uses Deepgram, else"openai", because 2.24.0 imports the object whole and fills a missing provider with OpenAI. Importing here copies onlyenabledandspeakerTemplate, so the stored field stays as the library wrote it. Plants: omit the provider (caught byexports the provider and model, and gives versions up to 2.24.0 Deepgram only for a Deepgram library); spread the file's object again (caught byimporting this version's export leaves the provider field of versions up to 2.24.0 as the library stored it).Speaker labels can only be turned off for a model without them, and the row says why).Release impact
Storage.
schemaVersionstays 2. The first save addstranscript.provider,transcript.modelandelevenLabsSecretId. Nothing else in a 2.24.0data.jsonchanges, and the load itself writes nothing.Migration, when
transcript.provideris absent. It reads only the stored document, adds no decode warning, and gives the same answer on every load until a save stores it.gpt-transcribe, as on #538.Every row gets
model: "". A library that never stored the speaker labels choice keeps them off. A library from before 2.19.0 that holds its OpenAI key as text counts as holding an OpenAI key.Sync with 2.24.0. A device on 2.24.0 keeps loading the library and keeps the three new fields on its own saves. It transcribes as before, from
transcript.diarization, and writestranscript.diarization.providerthere. This version reads that field only for the migration, never writes it, and keeps it as found. A provider chosen here does not reach a 2.24.0 device, and a change toDiarization providerthere does not reach this version oncetranscript.provideris stored.Settings export. The envelope stays at version 2.
secretsgainselevenLabs, andsettings.transcriptcarriesproviderandmodel.diarization.provideris written as"deepgram"exactly when the library uses Deepgram, else"openai", because 2.24.0 imports thediarizationobject whole and fills a missing provider with OpenAI. Importing a 2.24.0 export that labels speakers with Deepgram selects Deepgram. Any other 2.24.0 export keeps the provider. An import leaves the storeddiarization.provideralone.A 2.24.0 device that imports this version's export therefore keeps a Deepgram choice and otherwise uses OpenAI, as it would for its own export.
Speaker labels with OpenAI. A library that arrives with speaker labels on can turn them off. The row then stays disabled, and says why, while the model cannot label speakers.
Bundle. After #538 merged,
main.jsmeasures 322,274 raw and 105,220 gzip bytes, which is 930 gzip bytes over master's 104,290. The budget rises to exactly 105,220, andscripts/size-baseline.mdrecords it.ElevenLabs errors. The provider reads
detail.codeanddetail.messageas the lead measured them. The tests build those bodies from that description, not from a captured response.Note
Add selectable transcription provider and model, defaulting to ElevenLabs
providerfield is removed from settings; transcription commands now refuse with a message when the selected provider is unsupported or its key is missing; gzip bundle budget raised from 104,290 to 105,220 bytes in size.mjs.Macroscope summarized c6adf83.
Note
Medium Risk
Touches transcription routing, credential storage, and 2.24.0 migration/import paths—wrong inference would change which API users hit and billing behavior across upgraded libraries.
Overview
Adds ElevenLabs Scribe v3 as a third transcription backend and makes provider + model first-class settings (
transcript.provider,transcript.model,elevenLabsSecretId), replacing the old “speaker diarization → pick OpenAI vs Deepgram” flow.Settings & UI: The Transcripts section now has Transcription provider, per-provider API key rows, Model (recommended vs explicit), and Speaker labels—
Diarization provideris removed. New installs default to ElevenLabs with speaker labels on; unknown providers/models from a newer build stay stored but refuse transcription or fall back to the recommended model with a notice.Runtime: A shared provider registry drives
TranscriptionServiceandtranscribe(); ElevenLabs can transcribe by publicsource_urlor uploaded audio, with diarization gated by settings. Deepgram’sdiarizeflag follows speaker labels; cues omit speakers when labels are off.Migration & sync: Loading 2.24.0 libraries without
transcript.providerinfers the provider from legacydiarization+ linked keys (Deepgram when labels were on with Deepgram, OpenAI when labels were on or an OpenAI key exists, else ElevenLabs). Import/export carries provider/model and ElevenLabs secrets while preservingdiarization.providerfor 2.24.0 compatibility.Tooling: E2E/verify fixtures stub ElevenLabs; bundle gzip budget rises to 105,220 bytes with updated baseline docs.
Reviewed by Cursor Bugbot for commit c6adf83. Bugbot is set up for automated code reviews on this repo. Configure here.