Cartesia: create a Pro Voice Clone instead of an Instant one - #8
Open
esegev-cai wants to merge 16 commits into
Open
Cartesia: create a Pro Voice Clone instead of an Instant one#8esegev-cai wants to merge 16 commits into
esegev-cai wants to merge 16 commits into
Conversation
The arena's four Cartesia voices are Pro Voice Clones. When the Sonic 3.5 clips were generated those voices only had fine-tunes for sonic-3-2026-01-12, so the 3.5 generations were not served by a 3.5-quality voice. All four now carry fine-tunes through sonic-3.6-2026-08-14, so the clips should be re-run. Transport (src/pipeline/transports/cartesia.ts): - Unify the API version pin at 2026-08-14 across synthesize and ttfbPlanFor, which previously sent 2024-11-13 and 2025-04-16 respectively. - Use Authorization: Bearer, the documented scheme, instead of x-api-key. - Send an explicit language on synthesize; it was relying on auto-detection. - Bench TTFB on a stock voice instead of the Clara PVC, matching how every other provider on the table is measured. - Replace the /voices/clone call (Instant Voice Clone, one ~10s clip) with the documented Pro Voice Clone flow: dataset, upload every sample, fine-tune, poll to completion, then read the voice the fine-tune produced. Registry: - Pin Sonic 3.5 to sonic-3.5-2026-05-04 so the entry names exact weights rather than an alias we repoint over time. Goldens re-derived. - Correct two copy claims that named 3.5 the current flagship. - Stage Sonic 3.6 as a NEW_MODELS row, per the clips-before-registration rule. Added postFormNoContent to http.ts: dataset file upload answers 204. check-types clean, 125 tests pass, build succeeds. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The models a PVC serves sit a level below the fine-tune, which is why adapted versions do not consume a plan slot. Makes the data[0] read an explicit assumption rather than an accident. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Working note, not part of the change set. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
3.6 is in preview today and reaches GA 2026-08-27, so the registry copy should not assert general availability yet. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Restore the original file header, changing only the two facts that moved (IVC to PVC, Clara to a stock bench voice). Trim the constant comments to one line each. Inline the single-use FineTuneStatus union. Hoist the five headers() calls in createClone to one, destructure the poll response, and collapse the form and error-detail lines. No behavior change. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The enum gains newer base models, so 'the only one accepted today' goes stale immediately. State the choice and the forward-fill instead. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
createClone polled to completion, so a run held the process open for up to 3 hours per voice. cloneVoices loops the four voices sequentially, making that up to 12 hours for a full re-clone. It now creates the dataset, uploads every sample, starts the fine-tune, and returns. All four kick off in one run and train concurrently, and the maintainer collects the ids from the PVC dashboard and persists them with the existing --record flag. createClone widens to Promise<string | null> for this: null means the clone is still training. Every other transport already returns a string, so none of them change. Drops the poll loop, sleep, both timing constants, the status union, and the two follow-up GETs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Name the variable fineTune so it does not read as a voice id, say what the null return means, and point at the dashboard plus the --record command needed to finish the job. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The dataset file upload answers 204 with no body, so postFormForJson cannot parse it. Rather than add a fourth helper to the shared module, handle it in uploadSample here and leave http.ts untouched. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
createClone was typed Promise<string>, which assumed cloning produces a usable voice id by the time the call returns. That holds for instant clones: POST /voices/clone answers 200 with the voice, so the id is the result. It does not hold for a Pro Voice Clone. POST /fine-tunes answers with a fine-tune id, which identifies the training job, not a voice. The voice id only exists once training finishes, hours later. Returning the fine-tune id instead would be worse than returning nothing: whatever createClone returns is persisted to voices.local.json and sent as voice.id on every synthesis request, so every clip would 400 on a malformed voice. Promise<string | null> lets the Cartesia transport say the honest thing, that no voice id exists yet. cloneVoices prints a pending line instead of recording an id, and the operator persists it later with --record. Every other transport returns a string, which still satisfies the type, so none of them change. Also names the 180s upload window UPLOAD_TIMEOUT_MS, matching how fish.ts keeps its own timeout constants. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Moves 3.6 out of the pipeline staging table and into the registry, directly after the 3.5 entry it supersedes, cloned from 3.5's shape. latencyMs stays null and both copy blocks say the benchmark has not run: latency is measured-only and only a maintainer can bench it. seedLikelyRank is omitted because the suite requires unseeded entries not to carry one. voiceProfile is 27, the next free value, since profiles must be unique. Pinned counts bumped intentionally: BASE_MODEL_ENTRIES and listedModelEntries 24 to 25, arenaModelEntries 22 to 23, VARIANTS 88 to 92 (four source voices times one new model). EXPECTED_MODELS gains a row and cartesia-sonic-36 joins UNSEEDED_MODEL_IDS. Note the entry is active, so it joins the arena on merge. Its clips need to exist first or battles will serve missing audio. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The snapshot was re-minted today. apiModelId, arenaApiId, releaseDate and the EXPECTED_MODELS pin all move, and the fallback clip hash is re-derived from the new identity (arenaApiId feeds the content hash). The 2026-08-14 left in cartesia.ts is the Cartesia-Version API pin, which is a different thing that happened to share the date. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Registering it active put a model in the arena with no audio behind it, and forced a pinned fallbackClip hash for a file nobody has generated. The registry already has the right state for this: unlisted is documented as excluded from every derived surface with identity kept wired so re-listing is a one-line status flip, and the type makes sample optional there, which is why the fake clip pin is gone. Reverts the pins that only applied while it was active: arenaModelEntries, listedModelEntries, VARIANTS and the EXPECTED_MODELS row all go back. BASE_MODEL_ENTRIES stays 25, and the unlisted assertion now names the entry rather than expecting none. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
createClone called /voices/clone, an Instant Voice Clone built from a single ~10 second clip. The arena's four Cartesia voices are Pro Voice Clones, a different and higher-fidelity product, so that code could never have reproduced them. It now creates a dataset, uploads every sample rather than just the first (a PVC trains on the whole dataset and needs 30+ minutes of audio), and starts the fine-tune. It does not wait: POST /fine-tunes returns a training-job id and the voice appears up to 3 hours later, and cloneVoices loops the four voices sequentially, so polling would hold the process open for most of a day. Returning lets all four train concurrently. createClone widens to Promise<string | null> for this. Null means still training, so cloneVoices prints a pending line instead of recording an id and the operator persists it later with --record. Returning the fine-tune id instead would be worse than nothing: whatever createClone returns is written to voices.local.json and sent as voice.id on every synthesis call. Every other transport returns a string, so none of them change. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
@esegev-cai is attempting to deploy a commit to the Vapi Team on Vercel. A member of the Team first needs to authorize it. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cartesia: create a Pro Voice Clone instead of an Instant one
Follow-up to #7, which this builds on. Review that one first; the diff here shows its changes too
until it merges.
createClonecalled/voices/clone, which is an Instant Voice Clone built from a single ~10second clip. Your four Cartesia voices are Pro Voice Clones, a different and higher-fidelity
product, so that code could never have reproduced them. It now creates a dataset, uploads every
sample, and starts a fine-tune.
It no longer returns a voice id, because there is not one yet
POST /fine-tunesreturns a training-job id; the voice appears up to 3 hours later. SincecloneVoicesloops the four voices sequentially, waiting would hold the process open for most ofa day, so it returns instead and all four train concurrently. The command prints each fine-tune
id and where to watch it. Once training completes you paste the voice id in with your existing
--recordflag.That is why
ProviderTransport.createClonewidens toPromise<string | null>. Null means stilltraining, so
cloneVoicesprints a pending line rather than recording an id. Returning thefine-tune id instead would be worse than returning nothing, since whatever
createClonereturnsis written to
voices.local.jsonand sent asvoice.idon every synthesis request. Every othertransport returns a string, which still satisfies the type, so none of them change.
Notes
sampleFiles[0], since a PVC trains on the whole datasetand needs 30+ minutes of audio.
204with no body, so it does not go throughpostFormForJson,which would throw parsing an empty response. That is handled locally in
uploadSampleratherthan adding a fourth helper to
http.ts.the IVC-shaped call it replaces.
check-types,test(125 pass), andbuildare green.