Cartesia: pin dated snapshots and add Sonic 3.6 - #7
Open
esegev-cai wants to merge 16 commits into
Open
Conversation
The arena's four Cartesia voices are Pro Voice Clones. When the Sonic 3.5 clips were generated those voices only had fine-tunes for sonic-3-2026-01-12, so the 3.5 generations were not served by a 3.5-quality voice. All four now carry fine-tunes through sonic-3.6-2026-08-14, so the clips should be re-run. Transport (src/pipeline/transports/cartesia.ts): - Unify the API version pin at 2026-08-14 across synthesize and ttfbPlanFor, which previously sent 2024-11-13 and 2025-04-16 respectively. - Use Authorization: Bearer, the documented scheme, instead of x-api-key. - Send an explicit language on synthesize; it was relying on auto-detection. - Bench TTFB on a stock voice instead of the Clara PVC, matching how every other provider on the table is measured. - Replace the /voices/clone call (Instant Voice Clone, one ~10s clip) with the documented Pro Voice Clone flow: dataset, upload every sample, fine-tune, poll to completion, then read the voice the fine-tune produced. Registry: - Pin Sonic 3.5 to sonic-3.5-2026-05-04 so the entry names exact weights rather than an alias we repoint over time. Goldens re-derived. - Correct two copy claims that named 3.5 the current flagship. - Stage Sonic 3.6 as a NEW_MODELS row, per the clips-before-registration rule. Added postFormNoContent to http.ts: dataset file upload answers 204. check-types clean, 125 tests pass, build succeeds. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The models a PVC serves sit a level below the fine-tune, which is why adapted versions do not consume a plan slot. Makes the data[0] read an explicit assumption rather than an accident. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Working note, not part of the change set. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
3.6 is in preview today and reaches GA 2026-08-27, so the registry copy should not assert general availability yet. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
@esegev-cai is attempting to deploy a commit to the Vapi Team on Vercel. A member of the Team first needs to authorize it. |
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Restore the original file header, changing only the two facts that moved (IVC to PVC, Clara to a stock bench voice). Trim the constant comments to one line each. Inline the single-use FineTuneStatus union. Hoist the five headers() calls in createClone to one, destructure the poll response, and collapse the form and error-detail lines. No behavior change. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The enum gains newer base models, so 'the only one accepted today' goes stale immediately. State the choice and the forward-fill instead. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
createClone polled to completion, so a run held the process open for up to 3 hours per voice. cloneVoices loops the four voices sequentially, making that up to 12 hours for a full re-clone. It now creates the dataset, uploads every sample, starts the fine-tune, and returns. All four kick off in one run and train concurrently, and the maintainer collects the ids from the PVC dashboard and persists them with the existing --record flag. createClone widens to Promise<string | null> for this: null means the clone is still training. Every other transport already returns a string, so none of them change. Drops the poll loop, sleep, both timing constants, the status union, and the two follow-up GETs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Name the variable fineTune so it does not read as a voice id, say what the null return means, and point at the dashboard plus the --record command needed to finish the job. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The dataset file upload answers 204 with no body, so postFormForJson cannot parse it. Rather than add a fourth helper to the shared module, handle it in uploadSample here and leave http.ts untouched. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
createClone was typed Promise<string>, which assumed cloning produces a usable voice id by the time the call returns. That holds for instant clones: POST /voices/clone answers 200 with the voice, so the id is the result. It does not hold for a Pro Voice Clone. POST /fine-tunes answers with a fine-tune id, which identifies the training job, not a voice. The voice id only exists once training finishes, hours later. Returning the fine-tune id instead would be worse than returning nothing: whatever createClone returns is persisted to voices.local.json and sent as voice.id on every synthesis request, so every clip would 400 on a malformed voice. Promise<string | null> lets the Cartesia transport say the honest thing, that no voice id exists yet. cloneVoices prints a pending line instead of recording an id, and the operator persists it later with --record. Every other transport returns a string, which still satisfies the type, so none of them change. Also names the 180s upload window UPLOAD_TIMEOUT_MS, matching how fish.ts keeps its own timeout constants. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Moves 3.6 out of the pipeline staging table and into the registry, directly after the 3.5 entry it supersedes, cloned from 3.5's shape. latencyMs stays null and both copy blocks say the benchmark has not run: latency is measured-only and only a maintainer can bench it. seedLikelyRank is omitted because the suite requires unseeded entries not to carry one. voiceProfile is 27, the next free value, since profiles must be unique. Pinned counts bumped intentionally: BASE_MODEL_ENTRIES and listedModelEntries 24 to 25, arenaModelEntries 22 to 23, VARIANTS 88 to 92 (four source voices times one new model). EXPECTED_MODELS gains a row and cartesia-sonic-36 joins UNSEEDED_MODEL_IDS. Note the entry is active, so it joins the arena on merge. Its clips need to exist first or battles will serve missing audio. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The snapshot was re-minted today. apiModelId, arenaApiId, releaseDate and the EXPECTED_MODELS pin all move, and the fallback clip hash is re-derived from the new identity (arenaApiId feeds the content hash). The 2026-08-14 left in cartesia.ts is the Cartesia-Version API pin, which is a different thing that happened to share the date. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
esegev-cai
changed the title
Registering it active put a model in the arena with no audio behind it, and forced a pinned fallbackClip hash for a file nobody has generated. The registry already has the right state for this: unlisted is documented as excluded from every derived surface with identity kept wired so re-listing is a one-line status flip, and the type makes sample optional there, which is why the fake clip pin is gone. Reverts the pins that only applied while it was active: arenaModelEntries, listedModelEntries, VARIANTS and the EXPECTED_MODELS row all go back. BASE_MODEL_ENTRIES stays 25, and the unlisted assertion now names the entry rather than expecting none. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
createClone, its supporting helpers, the ProviderTransport return-type change and the cloneVoices handling all move to a follow-up PR so this one stays about the API version pin, auth, language, and the registry entries. Also restores the Clara clone as the TTFB bench voice. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
esegev-cai
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cartesia: pin dated snapshots and add Sonic 3.6
Hi, Cartesia here. We reviewed this code to add our latest model Sonic 3.6 to the leaderboard. We identified a number of changes that should be made while reviewing.
Two changes to our transport and registry entries, plus a few small fixes we
hit on the way through. A follow-up PR switches clone creation from an Instant Voice Clone to a
Pro Voice Clone, which is what your four Cartesia voices actually are.
check-types,test(125 pass), andbuildare green.Pinned to dated snapshots
sonic-3.5becomessonic-3.5-2026-05-04. PVCs are pinned to dated snapshots so best practice is to ping to model dated snapshot.Added Sonic 3.6
sonic-3.6-2026-08-27, registered right after 3.5 asstatus: 'unlisted', so it is off everypublic surface until you generate its clips. Flipping it to
'active'is the one-line re-list.latencyMsisnulland the copy says the benchmark has not run, since latency is measured-onlyand only you can bench it.
Drive-bys
synthesizesent API version2024-11-13whilettfbPlanForsent2025-04-16. Both now send2026-08-14.Authorization: Bearerinstead ofx-api-key, matching our docs.synthesizenow sendslanguage. It was relying on auto-detection, which can pick wrong onshort lines.
Before merging
Pinning 3.5 changes
arenaApiId, which your CONTRIBUTING freezes. Every arena clip is saved undera filename generated from the model's id, so the 20 hosted Cartesia 3.5 clips re-derive to new
filenames and need regenerating alongside this. Standings are keyed by variant id too, so its
existing votes would not carry over. Happy to drop that one edit if the timing does not suit.
Sonic 3.6 needs no sequencing: unlisted keeps it off the arena, the leaderboard and the sitemap,
so this is safe to merge whenever.
Happy to jump on a call.