Skip to content

feat(transcription): transcribe with OpenRouter - #540

Merged
chhoumann merged 55 commits into
masterfrom
feat/transcription-openrouter
Nov 16, 2026
Merged

chhoumann merged 55 commits into
masterfrom
feat/transcription-openrouter

Conversation

@chhoumann

@chhoumann chhoumann commented Nov 14, 2026 •

Copy link
Copy Markdown
Owner

Summary

PodNotes is moving transcription to current models from several providers, so that a provider is a credential row, one provider file and a registry line, and no user runs a retired model. This PR is step 4 of that plan: OpenRouter. It stacks on #539.

  • Settings. Transcription provider offers OpenRouter between Deepgram and OpenAI. Choosing it shows the OpenRouter API key row and a Model row with Recommended (Grok STT 2.0), Grok STT 2.0, ElevenLabs Scribe v2 and MAI-Transcribe 2.
  • Requests. Each model gets the request measured to return word times and speakers. Grok STT 2.0 gets base64 JSON routed to xAI with provider.only, which takes diarize from provider.options.xai. A request routed elsewhere answers 200 with no speakers. Scribe v2 and MAI-Transcribe 2 get multipart with diarize beside the model. Scribe v2 gets the address of an episode a saved public feed lists, so nothing is downloaded. With speaker labels off, no request carries diarize.
  • Parts. OpenRouter takes 25 MB a request and gives the model 60 s, so an upload goes in parts of at most 18 MB and 1,800 s, one at a time. Labels from two parts name different speakers, and the notice says OpenRouter transcribed this episode in 3 parts, and speaker labels restart in each, so one person can have several labels. Merging labels across parts is the next PR.
  • Missing speakers. A note saved with speaker labels asked for and none returned says Saved without speaker labels, because Grok STT 2.0 returned none. This holds for every provider.
  • Model freshness. PodNotes reads OpenRouter's public catalog, without the key, when the Model row draws and when an OpenRouter job starts, and keeps it in memory for 24 hours. It only takes models away: an unlisted model is not offered, a model whose expiration_date has come is skipped, and a later end date reads Grok STT 2.0, ends <date>. A chosen model the catalog drops stays selected as <label>, no longer offered, and the job runs the recommended model with a notice. A failed read is not kept and leaves the tested list.
  • Errors. 402 says the account is out of credit, 401 that the key was rejected, 404 that the model is not offered. A 404 whose message names the account's data policy or zero data retention says the model is not offered under the account's OpenRouter privacy settings. Each ends the job at the first part, and none quotes OpenRouter.
  • Notices naming the providers that label speakers now end with OpenRouter.
  • Times in a later part are rounded to the millisecond, so their links read time=1400.112, not time=1400.1119999994578.

For review:

  • The registry owns the catalog. A provider exposes an optional catalog() beside failure(). The registry keeps the reads in a Svelte store, so the Model row redraws when a read lands while it is open, and tests reset it through the same handle.
  • The privacy 404 is its own failure kind, privacy, which ends the job like model-unavailable, because the user's action differs.
  • withdrawnModel sits beside unknownModel on the choice, because each says something different: this version has no such model, or the provider's catalog dropped it.

Not measured with a real key, for the lead's real-key check:

  • The privacy 404 is matched by /data policy|zero data retention|\bzdr\b/i on error.message. No probe saved the real body. The pattern follows OpenRouter's published "No endpoints found matching your data policy (Zero data retention)".
  • Scribe v2 by address sends format: "mp3", as measured on an MP3 episode. An M4A episode by address is unmeasured.
  • Scribe v2 and MAI-Transcribe 2 with speaker labels off send verbose_json without diarize. Whether they still return words is unmeasured.
  • An address Scribe v2 cannot fetch through OpenRouter ends as OpenRouter could not transcribe this audio. with no upload, because OpenRouter's answer for it is unknown.

Not in this PR: merged speaker labels across parts, the weekly model review, and the docs, which describe the released version until the release.

OpenRouter chosen in Settings Scribe v2 note from a public episode by address
Grok STT 2.0 note from a 70 minute MP3 in three parts, with the notice

Verification

Run on Linux with Node 22.23.4, on the final tree c46f215a:

  • npm run lint, npm run format:check, npm run typecheck, npm run build and npm run docs:build pass.
  • npm run test: 124 files, 2,878 tests pass.
  • npm run test:tooling: 8 files, 48 tests pass.
  • npm run test:e2e on Obsidian 1.14.4: 33 files, 257 tests pass, J-TR-09 and J-TR-10 new.
  • PODNOTES_OBSIDIAN_APP=/opt/Obsidian/obsidian npm run test:e2e -- settings-tab journey-resilience on Obsidian 1.13.7: 2 files, 23 tests pass.
  • npm run size: 106,031 gzip bytes, 131 under the new 106,162 budget. The base measures 104,706 in a separate build of 1eef3e6f. The budget rises by the 1,296 bytes OpenRouter measured at a844ca70, and the millisecond fix after it adds 29.
  • npm run plant: 27 planted defects, each caught by the unit test named below, and 2 more caught by npm run test:e2e.

Obsidian 1.14.4 on Linux under Xvfb, in this worktree's isolated vault, with the verify skill's fixture provider answering OpenRouter and its catalog. No real key was used. The fixture feed is http://podnotes.test:24280/feed.xml. The local file is Fixtures/Long Fixture.mp3, 4,200 s of sine at 8 kbps mono from ffmpeg, 4,200,452 bytes.

  • Settings rows. OpenRouter was chosen with the mouse and arrow keys. The row shows the key row and the four Model options above. data.json held provider: "openrouter" and model: "". The network log showed one GET https://openrouter.ai/api/v1/models with no upload. The warning beside the key is Obsidian's own Secrets are not encrypted, because Xvfb has no keyring.
  • Scribe v2, public episode. With ElevenLabs Scribe v2 chosen by keyboard, Transcribe current episode from audio on Fixture Episode 1 sent one POST https://openrouter.ai/api/v1/audio/transcriptions, application/json, 160 bytes, naming http://podnotes.test:24280/episode-1.wav. The fixture server saw no request during the command. The note has **1:** at 0 s and **2:** at 12 s.
  • Grok STT 2.0, long local MP3. On Recommended (Grok STT 2.0), after a plugin reload, the command sent the catalog GET, then three POSTs of 1,867,279, 1,867,027 and 1,866,931 bytes of JSON, one after the other. The note numbers six speakers at time=0, 12, 1400.112, 1412.112, 2800.224 and 2812.224, and the restart notice stayed until clicked. Before the millisecond fix the same run linked time=1400.1119999994578.

dev:errors reported no error. The instance and the fixture server are stopped. The lead verifies with a real key separately.

Planted defects, each caught by the named test through npm run plant
Planted defect In Caught by
Grok without provider.only providers/openrouter.ts Grok STT 2.0, the recommended model, gets even a public episode as base64 JSON, routed to xAI and asked there for speaker labels
Grok sent an address providers/openrouter.ts Grok STT 2.0, the recommended model, gets even a public episode as base64 JSON, routed to xAI and asked there for speaker labels
sounds kept as words providers/openrouter.ts Grok STT 2.0, the recommended model, gets even a public episode as base64 JSON, routed to xAI and asked there for speaker labels
Grok asked for speakers with labels off providers/openrouter.ts with speaker labels off, x-ai/grok-stt-2.0 is asked for none, and the note names no speaker
Scribe v2 and MAI asked for speakers with labels off providers/openrouter.ts with speaker labels off, microsoft/mai-transcribe-2 is asked for none, and the note names no speaker
Scribe v2 and MAI name diarize differently providers/openrouter.ts microsoft/mai-transcribe-2 gets an upload as multipart, asked for speaker labels beside the model
Scribe v2 always uploads providers/openrouter.ts Scribe v2 takes a public episode by its address in one request, and its audio is never loaded
part offset dropped cues.ts a 40 MB episode goes in parts of at most 18 MB and 1,800 s, one at a time, timed to the millisecond from the episode's start
parts of 24 MB providers/openrouter.ts a 40 MB episode goes in parts of at most 18 MB and 1,800 s, one at a time, timed to the millisecond from the episode's start
cue times not rounded cues.ts a 40 MB episode goes in parts of at most 18 MB and 1,800 s, one at a time, timed to the millisecond from the episode's start
labels reused across parts cues.ts labels from two parts never name the same speaker, and the notice says labels restart in each
no restart notice transcribe.ts labels from two parts never name the same speaker, and the notice says labels restart in each
no check for missing speakers transcribe.ts speaker labels asked for and none returned, as from a route that ignores the option, saves the note and says so
OpenRouter's text in the sentence http.ts a 402 is said in PodNotes' words, with none of OpenRouter's, and no other part is sent
privacy 404 read as any 404 providers/openrouter.ts a 404 for a model no route of which meets the account's privacy settings is said in PodNotes' words, with none of OpenRouter's, and no other part is sent
privacy failure does not end the job transcribe.ts a 404 for a model no route of which meets the account's privacy settings is said in PodNotes' words, with none of OpenRouter's, and no other part is sent
unlisted model still offered registry.ts a chosen model the catalog no longer lists runs the recommended one, and the notice says so
no catalog read at job start TranscriptionService.ts a chosen model the catalog no longer lists runs the recommended one, and the notice says so
end date ignored by the filter registry.ts a model whose end date has come is skipped, and the next one is recommended
failed read kept as an empty catalog registry.ts a catalog that cannot be read leaves every model offered, and the next job reads it again
catalog read on every job registry.ts the catalog is read again only once a day has passed
catalog kept for two days registry.ts the catalog is read again only once a day has passed
expiration_date not read providers/openrouter.ts the catalog read as the row draws leaves out a model it does not list and dates one that ends
Model row reads no catalog PodNotesSettingsTab.ts the catalog read as the row draws leaves out a model it does not list and dates one that ends
Model row ignores a read that lands PodNotesSettingsTab.ts the catalog read as the row draws leaves out a model it does not list and dates one that ends
dropped model not marked PodNotesSettingsTab.ts a chosen model the catalog no longer lists stays selected, marked as no longer offered
an unmeasured model offered providers/openrouter.ts choosing it shows its API key row and its three models, Grok STT 2.0 recommended
Scribe v2 always uploads providers/openrouter.ts J-TR-09 a public episode goes to Scribe v2 by its address in one request, and its audio is not fetched (npm run test:e2e)
Grok audio sent as a data URI providers/openrouter.ts J-TR-10 Grok STT 2.0 gets the episode's audio as base64 in one request (npm run test:e2e)

Real OpenRouter key (lead, after review)

Live probes first, then Obsidian 1.14.4, Linux, build of 23a4d86b, palette command, a pass-through request log (no fixture):

  • Grok STT 2.0 on the 54 min, 39 MB local E231, labels on: one catalog GET without the key, then 3 sequential JSON POSTs of 18.1, 17.0 and 16.9 MB. Done in 35 s; 9,585-word note with timestamps and labels; the restart notice stayed up.
  • Scribe v2 on a public NPR News Now episode (https://feeds.npr.org/500005/podcast.xml), labels on: one 520-byte POST naming the address, no download, saved in 5 s with timestamps and speakers. Labels off: one 505-byte POST; the note has timestamps and no speakers.
  • dev:errors: none.

Review fixes (lead)

  • Probes showed Scribe v2 and MAI-Transcribe 2 return verbose_json with no words when neither diarize nor word timestamps are asked for, so a labels-off job failed with "OpenRouter found no speech". Both now always send word timestamps (timestamp_granularities[]=word, or ["word"] in JSON). Plant: drop it (caught by with speaker labels off, elevenlabs/scribe-v2 is still asked for word times, which OpenRouter sends only when asked).
  • An address the provider cannot download is a 400 "Provider could not download input_audio.url; ensure it is publicly reachable and serves a supported audio file" (captured live). It now maps to unfetched, so the job uploads once. Plant: return no failure kind (caught by Scribe v2 uploads the episode once when OpenRouter's provider cannot download its address).
  • The privacy matcher (/data policy|zero data retention|\bzdr\b/i) matches the real ZDR body captured earlier: "0 endpoints out of 1 requested are available matching your guardrail restrictions and data policy ... ZDR violation (account settings)".
  • Checks after the fixes: lint, format, typecheck, build, size (106,065 of 106,162), 2,881 unit tests.

Release impact

  • A new root field openRouterSecretId, empty by default. The key lives in SecretStorage as podnotes-openrouter-api-key. A settings export carries it as secrets.openRouter when keys are included. The schema stays at version 2. feat(transcription): choose a provider and model, with ElevenLabs as the default #539 measured that 2.24.0 keeps elevenLabsSecretId, an unknown root field to it. This field takes the same path, which I inferred and did not run.
  • transcript.provider can now be openrouter. A version without OpenRouter keeps the value and refuses to transcribe with a notice, as feat(transcription): choose a provider and model, with ElevenLabs as the default #539 does for any unknown provider.
  • With OpenRouter chosen, PodNotes sends a public GET to openrouter.ai/api/v1/models?output_modalities=transcription when the Model row draws or a job starts. After a read that works it waits a day. After one that fails it asks again next time.
  • The gzip budget rises from 104,866 to 106,162 bytes.

Note

Add OpenRouter as a transcription provider with model catalog and multipart upload

  • Adds an OpenRouter provider in openrouter.ts with three models: Grok STT 2.0 (default), ElevenLabs Scribe v2, and MAI Transcribe 2. Grok sends base64 audio with xAI routing; the others send JSON with a public audio URL or multipart file audio.
  • Adds a daily-cached model catalog (registry.ts). Models past their end date are excluded, a withdrawn stored model keeps working with a "no longer offered" label, and catalog failures fall back to the built-in list.
  • Long episodes are split into parts within 18 MB / 1,800-second caps, with speaker-label and part-splitting warnings. Word timestamps are rounded to millisecond precision.
  • Adds OpenRouter credentials, settings UI rows, and privacy/unavailable-model failure messages that end the job.
  • Behavioral Change: the settings modelRow now refreshes models on draw and can follow a supplied store via follow(store); transcribeAudio refreshes the provider catalog before resolving credentials; gzip budget rises to 106,562 bytes in size.mjs.

Macroscope summarized df7e41c.


Note

Medium Risk
Touches external API calls, API key storage, and transcription job logic with catalog fallbacks; well-covered by unit and e2e tests but real-key edge cases remain partially unverified per PR notes.

Overview
Adds OpenRouter as a transcription provider with openRouterSecretId credentials, settings (API key + model picker), and verify/e2e fixture support.

The new provider supports Grok STT 2.0 (base64 JSON routed to xAI), ElevenLabs Scribe v2 (public episode URL when under 1,800s), and MAI-Transcribe 2, with 18 MB / 1,800 s chunking, per-part speaker numbering, and user-facing warnings for missing labels, withdrawn models, and multi-part restarts.

A daily-cached public model catalog filters offered models, marks end dates in the UI, falls back when a stored model is delisted, and refreshModels runs before jobs and when the model row draws. Adds a privacy failure kind for OpenRouter ZDR/policy 404s.

Transcript cue times are rounded to milliseconds for seek links. The gzip bundle budget rises to 106,562 bytes.

Reviewed by Cursor Bugbot for commit df7e41c. Bugbot is set up for automated code reviews on this repo. Configure here.

gpt-4o-transcribe-diarize rejects requests with more than 1400 s of audio,
but MP3 chunks were split by size alone. Any MP3 under about 120 kbps put
more than 1400 s in a 20 MB chunk, so every chunk failed.

For diarization, split MP3 at frame boundaries into the fewest equal parts
that stay under both 20 MB and 1400 s. Whisper keeps its size-only chunks.

Fixes #532
A chunk that OpenAI rejected with a status such as 400 or 401 was sent
three times, although each send fails the same way. A long diarized
episode went up three times before the error showed.

Retry only the statuses openai-node retries: 408, 409, 429 and 5xx, plus
network errors.

Refs #532
gpt-4o-transcribe-diarize takes up to 0.58 s per second of audio, measured
over 81 requests, so the 1,072 s parts a 54 min episode got would take about
10 min each, at the request timeout. Parts are now at most 600 s, which the
model answers in under 6 min.

Refs #532
Each test passes on the hand-written code. Together they catch wrong
derivations that no existing test sees: the -1 SecretStorage ID suffix
that released versions accept, the provider an export error names, the
kind a settings key row adopts a secret for, and the key migration
notices.
CREDENTIALS in src/types/Credentials.ts holds each kind's settings
field, export key and label. The credential repository, settings
decoder, settings transfer, settings tab and key migration notice loop
over it. IPodNotesSettings takes its key fields from it, so a kind with
no default in DEFAULT_SETTINGS fails typecheck. The plaintext key map
from before 2.19.0 stays fixed.
The published transcript renderer is the reader of the format that stays
when the diarization folder goes. The default template now lives only in
DEFAULT_SETTINGS, so src/constants.ts imports no service code.
src/services/transcription/ turns an episode into a transcript with any
provider. A provider is one file: its models, its request, and how to read
its answers. OpenAI (gpt-transcribe) and Deepgram (nova-3) are the first two.

The engine owns what providers share:

- audioSource.ts decides whether a provider may be given the episode's
  address. Only an address that a saved public feed lists, that carries no
  credential, and that names a public host passes. All other audio is
  uploaded.
- split.ts is audioChunker.ts with the caps as arguments and a start time
  on each part.
- http.ts sends a request at most three times, and only after a 429, a 5xx,
  or a lost connection. Its failure sentences never include the provider's
  own text, which can quote a private address.
- cues.ts turns words or text into the shapes of a published transcript.

Nothing calls the engine yet. Lint rejects an import of the folder's files
from outside it, except its index.
TranscriptionService keeps its queue, the published-transcript path, the
progress notice, and note saving. It calls the engine for audio.

- OpenAI transcribes with gpt-transcribe. whisper-1 and
  gpt-4o-transcribe-diarize, which OpenAI shuts down on 2027-02-26, are
  gone. A tooling test searches the built main.js for them.
- With speaker labels on and OpenAI selected, the transcript is saved
  without labels. A notice that stays until clicked names the providers
  that label speakers.
- Deepgram transcripts go through renderTranscriptBody, so each paragraph
  has a timestamp. Speakers are numbered in the order they first speak.
- Deepgram gets the address of a public episode and downloads it itself.
- A failure notice stays until clicked. Parts go out one at a time, and a
  401, 402, 404, or 400 stops the job before the next part.

Deletes src/services/diarization/, openaiTranscription.ts, the Whisper
worker pool, and createBinaryChunkFiles. getEpisodeAudioBuffer no longer
returns a basename, because uploads are named for their format only.
net.mjs answers OpenAI with { text } and Deepgram with the words of two
speakers, and logs the content type of each request. fixtures.mjs serves a
feed at priv3.supercast.com, an address that PodNotes keeps private. The
transcripts recipe covers OpenAI in parts, and Deepgram by address and by
upload.
The marketplace review test called TranscriptionService.waitForRetry, which
the engine replaced. An aborted run now rejects through waitForLifecycle, so
the test aborts a pending wait there and expects the same Error.
An aborted job no longer downloads the episode or decodes it into parts.
Until now only its requests were refused.

A 429 or a 5xx that asks to wait over a minute is sent once. A sooner send
would only upload the audio to be refused again.
A provider's name was written twice: once in its credential row and once in
its provider file. The row now owns it, so a provider file holds only its
models and its request. Each model gains the name the settings will show.
The provider was derived from the speaker settings: Deepgram with speaker
labels on, OpenAI otherwise. It is now stored as transcript.provider, with
transcript.model beside it, where "" follows the provider's recommended model.
schemaVersion stays 2 and both fields are additions, so a device on 2.24.0
keeps loading the library and keeps the fields on its own saves.

A library that has no transcript.provider gets the provider it ran with:
Deepgram for speaker labels with Deepgram, with or without its key, and OpenAI
otherwise. The migration reads only the stored document, adds no decode
warning, and gives the same answer on every load until a save stores it.

transcript.diarization.provider belongs to versions up to 2.24.0. This version
reads it for that migration alone, never writes it, keeps it in data.json as
found, and leaves it out of settings exports. Importing a 2.24.0 export that
labels speakers with Deepgram selects Deepgram.

A provider or model ID this version does not have is kept as stored and shown
under that name. An unknown provider refuses to transcribe with a notice. An
unknown model runs the recommended model and the notice says so.

The Transcripts settings now read: Transcription provider, the chosen
provider's API key, Model, Speaker labels, Speaker label format, then the path
and template. Speaker labels is disabled for a model that cannot label
speakers. The Diarization provider row is gone.
ElevenLabs is a credential row, one provider file and a registry line. Its key
reference is the new root field elevenLabsSecretId, stored in SecretStorage as
podnotes-elevenlabs-api-key, and a settings export carries the key as
secrets.elevenLabs. The export envelope stays at version 2.

A public episode goes to ElevenLabs by its address, so nothing is downloaded.
Any other episode is uploaded in one request. Word times and speakers reach
the note, and turning speaker labels off asks for none.

ElevenLabs answers 401 for a used-up quota as for a rejected key, and 400 for
a model the key may not use as for an address it could not download, so the
provider reads detail.code. Only the last of these is followed by an upload.
A request waits 10 minutes for its answer.

A new install starts on ElevenLabs with speaker labels on. A library saved up
to 2.24.0 that holds no OpenAI key and has speaker labels off moves to
ElevenLabs too, because it never set transcription up. A library that never
stored the speaker labels choice keeps them off.

The notice for a model without speaker labels now names ElevenLabs first.
A journey now seeds its transcription provider with its key, because a new
vault is on ElevenLabs. The OpenAI journeys choose OpenAI. A new journey runs
the default: a public episode goes to ElevenLabs by its address, no audio is
fetched, and the note has timestamps and speaker labels. The ElevenLabs stub
answers with the words ElevenLabs returns and records what each request named.

The settings tab test walks the new Transcripts rows, and the key recovery
test links the ElevenLabs key, the one a new vault shows.
The fixture provider answers api.elevenlabs.io with ElevenLabs' words, two
speakers with the second 12 s in, and logs the address a request named in
place of a file. The transcript guide follows the new settings rows and adds
the ElevenLabs runs and a 2.24.0 library check.
…choice

The provider row decided for itself whether the stored provider is one this
version has, and the Speaker label format row read the stored speaker setting.
Both now ask transcriptionChoice, as the key rows, the model row and the
Speaker labels row do. The engine no longer exports a type nothing imports.
The Transcripts tests looked each control up by a CSS selector inside its row
and asserted on a hidden row's inline style. They now query by role and by the
row's name, as a user or a screen reader finds a control, so a hidden row is
one the query does not find.
…ings measure

The transcription engine left main.js at 103,795 gzip bytes. The provider and
model settings, their migration and the ElevenLabs provider add 866, which
takes it to 104,661 and over the 104,000 budget. The budget rises by those 866
bytes to 104,866, so the 205 bytes of headroom stay as they were.
…m library

2.24.0 imports the diarization object whole and fills a missing provider
with OpenAI, so omitting the field moved Deepgram libraries to OpenAI.
Imports here keep the provider this library stored.
A library migrated with OpenAI speaker labels on could not turn them off,
so every transcription repeated the notice.
… one

The tests for a provider and a model a newer PodNotes stored used OpenRouter
and Grok STT 2.0, which this version is about to run. They now use AssemblyAI
and universal-3-6-pro.
OpenRouter is a credential row, one provider file and a registry line. Its
key reference is the new root field openRouterSecretId, stored in
SecretStorage as podnotes-openrouter-api-key, and a settings export carries
the key as secrets.openRouter.

It offers three models, each with the request measured to give word times
and speakers:

- Grok STT 2.0, recommended: base64 JSON with verbose_json and word and
  segment times, routed to xAI with provider.only, which takes diarize from
  provider.options.xai. A request routed elsewhere is answered with no
  speakers.
- ElevenLabs Scribe v2: multipart with diarize beside the model, or JSON with
  input_audio.url for an episode a saved public feed lists.
- MAI-Transcribe 2: multipart with diarize beside the model.

With speaker labels off, no request carries diarize. Sounds Scribe v2 lists
as audio_event items are left out.

OpenRouter takes 25 MB a request and gives the model 60 s, so an upload goes
in parts of at most 18 MB and 1,800 s. Parts of 24 MB were answered in 20 to
31 s.

A 402 says the account is out of credit, a 401 that the key was rejected,
and a 404 that the model is not offered. A 404 whose message names the
account's data policy or zero data retention says the model is not offered
under the account's OpenRouter privacy settings. Each ends the job at the
first part, and none quotes OpenRouter.
…t each part

A note saved with speaker labels asked for and none returned now says so:
"Saved without speaker labels, because Grok STT 2.0 returned none." OpenRouter
answers 200 with no speakers when a request reaches a route that ignores the
speaker option, so this is checked for every provider.

Labels from two parts already name different speakers. An episode sent in
parts now also says that its labels restart in each part, so one person can
have several labels, until labels are merged across parts.
…ll lists

PodNotes reads OpenRouter's public catalog of transcription models, without
the key, when the Model row draws and when an OpenRouter job starts. It keeps
the read in memory for 24 hours. A read that fails is not kept, so the next
one asks again, and the tested list is offered as it is.

The catalog only takes models away. A model it does not list is not offered,
and one whose expiration_date has come is skipped. A model with a later end
date reads "Grok STT 2.0, ends <date>" in the Model row. A catalog that would
leave no model is not followed.

A chosen model the catalog no longer lists stays selected in the Model row as
"<label>, no longer offered". A job runs the first offered model instead and
says "OpenRouter no longer offers ElevenLabs Scribe v2, so PodNotes used Grok
STT 2.0." The row redraws when a read lands while it is open.

A provider gets its catalog through an optional catalog() beside failure(),
so the registry owns the cache and the filter for any provider that has one.
A stub answers OpenRouter's transcriptions with two speakers, the second 12 s
in, and its model catalog with the three models PodNotes offers. It records
what each request named and a hash of the audio it carried, decoded from
base64 for JSON.

J-TR-09 plays a public episode on Scribe v2: one JSON request names the
episode's address, and the episode's audio is never fetched. J-TR-10 plays it
on the recommended Grok STT 2.0: one JSON request carries the audio as base64,
the same bytes the fixture server holds. Both notes have timestamps and
numbered speakers, and the catalog is read once, with no Authorization.

A journey seeds any provider by its credential row, and a model with it.
…eads them

The fixture provider answers openrouter.ai transcriptions with verbose_json
words from two speakers, the second 12 s in, and the model catalog with the
three models PodNotes offers. The log keeps openrouter.ai paths and reads the
address a JSON request names in input_audio.url.

The transcript guide adds the OpenRouter rows, its runs (Scribe v2 by
address, Grok by base64, a long MP3 in parts) and the notices that now name
OpenRouter.
…ame parts

Summing frame durations drifts past the cap: 25,000 frames of 24 ms sum to
600.0000000000218 s, so the per-frame seconds check cut each part a frame
early and a 49,999-frame episode went as 24,999 + 24,999 + 1 frames. Parts
are already counted in frames, which keeps them under maxSeconds, so the
check goes.
The planner fixed a frame count per part and checked the size cap
separately. When a leading tag or a variable bitrate put an equal split over
the cap, the size check cut a part short and the next kept the old frame
count, leaving a one-frame part that cannot be decoded: 43,690 frames of a
320 kbps MP3 with a 512-byte tag went as 21,844 + 21,845 + 1. The planner
now takes the fewest equal splits that fit both caps, adding a part when an
equal split does not fit.
Equal splits put the odd frame first, so a 512-byte tag and 43,689 frames of
a 320 kbps MP3 put the first of two parts 192 bytes over 20 MiB, and the
planner sent three where two fit. Each cut now takes an equal share of the
frames still to place and steps back while the part is over the size cap,
so 21,844 + 21,845 goes in two.
Cuts only moved back from an even split, so where the bitrate rose a part
was added although moving the cut forward fit: 21,847 low-bitrate frames
then 21,845 at 320 kbps and an ID3v1 tag went as three parts, not two. The
planner now finds the fewest parts by filling each as far as it fits, then
places each cut as close to an equal share as the parts before and after it
allow. A property test checks the count against a brute-force minimum on 300
random variable-bitrate streams.
Frame-like bytes inside a leading tag set the first frame's duration, and
counting every part in that frame's length sent a 1,728 s episode of 72 ms
frames up in one request. Parts now fit by the summed duration of their own
frames, and each part's start is the sum of the frames before it.
Audio frames inside a tag's declared size were counted and could be cut
through, sending an undecodable first part, and frame-like bytes in a
trailing tag added a part.
…engine

# Conflicts:
#	src/services/TranscriptionService.test.ts
#	src/services/TranscriptionService.ts
#	src/services/audioChunker.test.ts
#	src/services/audioChunker.ts
#	src/services/diarization/openaiProvider.ts
#	src/services/diarization/types.ts
#	src/services/openaiTranscription.test.ts
#	src/services/openaiTranscription.ts
#	tests/mocks/openaiFailures.ts
…h cap

A file in a format other than MP3 under the size cap went whole whatever its
length, so a long .m4a passed a model's maxSeconds. Its length now comes from
its metadata through a media element, without decoding it, and audio over the
cap is decoded into WAV parts that fit both caps. Audio of an unknown length
still goes whole.
A public feed's enclosure with AWS or Google Cloud signing parameters passed the
feed credential check, so Deepgram got the whole signed address. A query
parameter whose name holds signature, credential or token, or Akamai's hdnts,
now uploads the audio instead.
…nload

A WebView may load no metadata for media it does not play, so the length probe
could wait forever, and an unload could not stop it before a decode. The probe
now reads the length as unknown after 5 s or when the job is aborted, and an
aborted job decodes nothing.
…nscription-providers

# Conflicts:
#	AGENTS.md
#	scripts/size-baseline.md
#	scripts/size.mjs
#	scripts/size.test.ts
#	src/persistence/settingsCodec.ts
#	src/settingsTransfer.ts
#	src/types/Credentials.ts
#	src/ui/settings/PodNotesSettingsTab.ts
Base automatically changed from feat/transcription-providers to master November 16, 2026 09:00
…nscription-openrouter

# Conflicts:
#	AGENTS.md
#	scripts/size-baseline.md
#	scripts/size.mjs
#	scripts/size.test.ts
@chhoumann

Copy link
Copy Markdown
Owner Author

Retargeted to master after #539 merged (9784fd0), and brought up to date (f60f448).

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit f60f448. Configure here.

@chhoumann

Copy link
Copy Markdown
Owner Author

FAIL
Verified head f60f448 by an independent verifier (GPT 6 Astra).

  • Compared origin/master at 9784fd0a with this head in isolated Obsidian 1.14.4 on Linux. Read the diff, all comments/review threads and merged feat(transcription): move transcription onto one engine and current models #538/feat(transcription): choose a provider and model, with ElevenLabs as the default #539. Used fake keys and the fixture provider only, with trusted mouse/keyboard input, palette commands, captured request bodies, notes on disk, data.json and screenshots.
  • All required checks pass through fnm exec --using 22.23.2: npm run lint, npm run format:check, npm run typecheck, npm run build, npm run size, npm run test (2,908 tests), and npm run test:tooling (48 tests). Bundle is 326,347 raw / 106,544 gzip bytes, with zero gzip headroom. That installed Node entry reports v22.23.4.
  • Passed 40 head cases across recommended/explicit Grok, Scribe and MAI, labels on/off, public/private/signed episodes, 70-minute local MP3 and 40-minute local M4A; also ran 20 base cases across its three providers. Head uploads MP3 in three parts and M4A in twelve. Independently decoded all 134 uploaded parts from the main head matrix; each fits 18 MB and 1,800 seconds. Notes have ordered timestamps and distinct speaker numbers across parts.
  • Requests carry the expected model, xAI routing, word-time and speaker options. Scribe sends public addresses and uploads private/signed audio; AWS, Google, Azure, Akamai and custom-token credentials never enter provider bodies. Tracking-only URLs retain address mode. A simulated Scribe download failure falls back to one upload. Missing-speaker and part-restart notices work.
  • Head and base failure probes confirm stop-on-400/401/402/404, at most three sends per part for 429/503/connection loss, and Retry-After waits. Head privacy 404 and missing-speaker notices remain after 16 seconds; provider error text stays out of notices. Failed jobs write no note. Settings survive reload, retain schema 2 and store key references rather than key values.
  • Partial catalog withdrawal works: the stored Scribe choice stays marked “no longer offered” and runs Grok with a warning. Individual expired models disappear, future expiration dates display, catalog requests omit authorization, and a failed catalog read is retried.
  • P2 defect in registry.ts:102: a successful catalog with no supported, unexpired model restores the entire fixed list. Reproduced with HTTP 200 listing Grok expired on 2020-01-01 plus an unsupported replacement, with Scribe/MAI absent. Settings still recommends expired Grok and offers both absent models; the palette command sends three Grok uploads without a withdrawal warning. Also reproduced with all three expired and all three absent. Preserve the empty availability result and refuse transcription before uploading; reserve fallback for failed catalog reads.
  • Evidence and rerunnable drivers are under /home/christian/.t3/worktrees/PodNotes/aa30073c-1a46f260e23f16bc/.obsidian-e2e-artifacts/review540/, starting with verification.md, catalog-defect.mjs and catalog-defect-summary.json. Restored the pinned head and byte-identical bundle, changed no tracked files, captured no runtime errors, and stopped the fixture server and isolated instance (pids: none).

…address

An address goes in one request, so Scribe v2 on OpenRouter got a public episode
of any length by address, past the 1,800 s its parts are capped at for
OpenRouter's 60 s limit. An episode the feed says is longer than the model's
cap is now uploaded in parts. One of unknown length still goes by address.
@chhoumann

Copy link
Copy Markdown
Owner Author

On the catalog finding in the f60f448 verdict: falling back to the fixed list when the catalog would leave no model is intended, and documented on offeredModels: "A catalog that would take every model away is not followed." I'm keeping it, for these reasons:

  • A withdrawn model fails safely. OpenRouter answers a model it no longer serves with 404. That maps to model-unavailable, which shows "OpenRouter does not offer the x-ai/grok-stt-2.0 model to this API key." and ends the job before the next part. Your probe sent three parts because the fixture provider answers 200 for any model, including one the catalog lists as expired.
  • Following an empty catalog has a worse failure. A successful read is kept for 24 h. One malformed or filtered 200 response, or a rename on OpenRouter's side, would turn OpenRouter off in PodNotes for a day, with no request to show what is wrong.
  • A model the catalog lists with an end date still in the future, or that it withdraws while others remain, is handled as you observed: it is hidden, or marked "no longer offered" with a warning.

If OpenRouter's catalog ever drops every model PodNotes knows, the fix is a release with new model IDs. The weekly model review in #541 exists to catch that.

@chhoumann

Copy link
Copy Markdown
Owner Author

PASS
Verified head df7e41c by an independent verifier (GPT 6 Astra).

  • Compared master 9784fd0a and this head in isolated Obsidian 1.14.4 on Linux. Read the diff, comments/review threads, prior FAIL and replies, and merged feat(transcription): move transcription onto one engine and current models #538/feat(transcription): choose a provider and model, with ElevenLabs as the default #539. Used fake keys and fixture providers only, trusted mouse/keyboard input, palette/direct commands, request bodies, notes on disk and screenshots.
  • All requested checks pass through fnm exec --using 22.23.2: npm run lint, npm run format:check, npm run typecheck, npm run build, npm run size, npm run test (2,910 tests), and npm run test:tooling (48 tests). Bundle is 326,389 raw / 106,562 gzip bytes, with zero gzip headroom. The installed Node entry reports v22.23.4.
  • Selected recommended/explicit Grok, Scribe and MAI in Settings, with the OpenRouter key row and labels on/off. All 32 main head cases and 16 public-duration cases passed, alongside 24 baseline cases. Independently decoded 354 head uploads plus 26 baseline uploads; head audio parts fit 18 MB and 1,800 seconds. Long MP3s use three parts and M4As twelve; MP3 parts concatenate to the original bytes. Notes have ordered timestamps and distinct speaker numbers across parts.
  • The URL-length fix works. Scribe sends the 1,752-second episode by address with no download; the 3,800-second episode downloads and sends nineteen uploads, with no URL submission. Both pass with labels on/off. Unknown duration retains address mode. A simulated provider download failure falls back to one upload. Requests carry word-time and model-specific speaker options; restart and missing-speaker notices work.
  • Private-feed and signed enclosures upload without exposing their addresses or credential markers to providers. Tested AWS, Google, Azure, Akamai hdnea/hdnts, custom tokens, and every OpenRouter model with a signed enclosure. Tracking-only URLs still use address mode.
  • Base/head failure probes confirm stop-on-400/401/402/404, at most three sends per part for 429/503/connection loss, and Retry-After handling. A two-second delay waited 2.011 seconds; an over-minute delay caused no immediate retry of that part. Failed jobs write no note. Head privacy-404 and missing-speaker notices remain after sixteen seconds; provider error text stays out of notices.
  • Partial withdrawal preserves the stored model as "no longer offered", runs the recommended model and warns. Expired models disappear, future end dates display, catalog requests omit authorization, and failed reads retry. For the accepted empty-catalog fallback, both empty and entirely unavailable catalogs followed by 404 send exactly one upload, write no note, and show the sticky "OpenRouter does not offer the x-ai/grok-stt-2.0 model to this API key." notice. The author's stated stop path works.
  • Provider/model choices survive reload; data.json remains schema 2 and stores key references. No runtime errors or error-console messages were captured. Restored the pinned head and byte-identical bundle, changed no tracked files, and stopped the fixture server and isolated instance (pids: none). Evidence and rerunnable drivers are under /home/christian/.t3/worktrees/PodNotes/9e00e65b-218463c8eca4582c/.obsidian-e2e-artifacts/review540/, starting with verification.md.

@chhoumann
chhoumann merged commit a152aac into master Nov 16, 2026
18 checks passed
@chhoumann
chhoumann deleted the feat/transcription-openrouter branch November 16, 2026 09:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant