One edit history for the agent's work and your own - #33
Draft
jlocala1 wants to merge 27 commits into
Draft
Conversation
Flip client imports from the local ui-staging/ folder to the promoted @mieweb/ui components (MediaPlayer, MediaEditor, TranscriptView + transcript schema), delete the staging folder, and bump the ui submodule pointer to the media-editor-components branch (mieweb/ui#313). - client/src/App.tsx: MediaPlayer/MediaEditor from @mieweb/ui/components/* - client/src/types.ts: transcript schema re-exported from @mieweb/ui/components/TranscriptView - remove client/src/ui-staging/ (now lives in @mieweb/ui) - bump ui submodule add2596 -> a445544
FileUpload becomes a dashed-border Card dropzone with Button, Alert, and SpinnerWithLabel; PulseCamButton becomes a Card with the QR flow in a ui Modal; BrandSelector swaps the native <select> for ui Select. Emoji icons replaced with lucide-react; component SCSS deleted.
Header, featured-pulse cards (with thumbnail fallback), feature tiles, and the loading view move to Card/Alert/SpinnerWithLabel + semantic tokens; the API-key and featured dialogs move to ui Modal + Input + Button. Dead landing/modal SCSS removed (App.scss 940 -> 469 lines).
Adds a second transcription provider alongside AssemblyAI: whisper.cpp running on the server - free, offline, no API key. Registers when WHISPER_MODEL_PATH is set (model file must exist); appears in the existing provider dropdown automatically. - media -> 16kHz mono WAV via ffmpeg, then whisper-cli with --max-len 1 --split-on-word for word-level segments - normalizes to the shared Transcript schema with per-word confidence (mean token probability, special tokens excluded) - new alwaysAsync provider flag: whisper jobs always use the async polling flow so slow local transcription never holds an HTTP request - README setup section, .env.example entries, models dir gitignored Verified: 2:43 real video -> 375 words with timestamps through the async path; result cached; both providers listed in /api/providers.
The provider dropdown only rendered before a transcript existed, so there was no way to re-transcribe with a different provider - invisible with one provider, a real gap with two. Show the same dropdown in the viewing state next to Re-transcribe, which already uses the selected provider and skips the cache.
…EL_PATH Each model registers as its own provider (Whisper (base.en), Whisper (large-v3-turbo-q5_0), ...) so the existing provider dropdown doubles as a model picker - no client changes. Distinct provider ids also keep cached transcripts separate per model.
Adds POST /api/artipod/:id/export — builds the cut list from the edit list (mirrors buildPlaybackSegments in @mieweb/ui so exports match play-as-edited), renders with ffmpeg trim/concat via a filter script, and writes export.mp4 (or export.m4a for audio-only sources) into the artipod folder. Async job + status polling, same pattern as transcription. Export button in the toolbar downloads the result; a 401 opens the API-key dialog. Speed markers are not baked into exports yet — they are not persisted server-side; documented as follow-up.
whisper.cpp emits contiguous word timestamps, so the editor never saw a pause on locally transcribed uploads (zero silence chips). After transcription we run ffmpeg silencedetect on the already-extracted WAV (signal-level ground truth, no model) and trim word boundaries that overlap detected silences, opening real inter-word gaps. The editor's existing chip synthesis then works unchanged. Verified on the benchmark clip: base.en went from 0 gaps to 20 gaps >=400ms, including the 4.0s pause AssemblyAI independently reports.
Export now opens a small modal (ui Modal/Input/Button, so it follows the brand theme) prefilled with '<original-name>-edited'; the chosen name is applied to the browser download via the anchor download attribute. The server-side file stays export.mp4/.m4a so media detection is unaffected. The toolbar button is the @mieweb/ui Button with a loading state instead of a raw styled button.
Speed: markers and the default speed now persist in edits.json (PUT merges missing fields so a speed-only save cannot clobber undo history), rehydrate into the editor via the new MediaEditor props, and bake into renders — segments split wherever the effective speed changes, video via setpts, audio via atempo (clamped to atempo's 0.5-2 range, which is the editor's own speed range). Verified: 2s@2x + 3s@1x source renders 4.03s; a marker mid-run splits the segment (3s source -> 2.97s). Captions: the export plan maps every spoken word onto the exported timeline (cut- and speed-aware); cues group to ~64 chars broken on dead air, written as an export.srt sidecar on every export and burned into the video via the subtitles filter when the new dialog checkbox is on. Hosts whose ffmpeg lacks libass render unburned instead of failing (the dev box's ffmpeg has it; some Homebrew builds do not). Requires @mieweb/ui with MediaEditor speed-state props (mieweb/ui#343).
POST /api/artipod/:id/agent-edit (requireAuth, async job + poll, mirroring
export). The client sends transcript words; the server rebuilds the editor's
silence-inserted baseline (initEditableWords in @mieweb/ui), procedurally
deletes the silences, then an LLM picks content deletions (fillers, false
starts, repeats) from a numbered word list, returning validated JSON
{delete, summary} — in-range unique ints, capped at 50% of the spoken words,
conservative when unsure. The proposal is written to edits.json with the
prior editor state pushed as a single undo snapshot and speedMarkers /
defaultSpeed preserved: the human reviews it in the editor and reverts with
one Cmd-Z. Never auto-exports.
The LLM is env-pluggable (AGENT_PROVIDER=anthropic|openai, AGENT_API_BASE,
AGENT_API_KEY, AGENT_MODEL) via global fetch — no SDK dependency, so the
runtime tar and `npm ci --omit=dev` are untouched. Anthropic Messages API or
any OpenAI-compatible base (local Ollama is the zero-key dev fallback). With
nothing configured the endpoint fails with a clear 503, so the dev box
degrades gracefully until a key is seeded into shared/env.
Client: an AI-edit button in the viewing toolbar (Export-button pattern — ui
Button, loading state, 401 opens the key dialog) polls, reloads the proposal,
and remounts MediaEditor via a key epoch so the strikethroughs and undo
appear.
First slice of the brand-overlay layer. A BrandLowerThird React component built from @mieweb/ui (Card + brand tokens + the PulseClip mark) is the swappable brand surface — every color is a primary-* / white token, so switching brands via BrandSelector, or dropping in real MIE assets later, restyles it without touching the export/render code. The client rasterizes the component to a transparent PNG (html-to-image) and sends it with the export; the server scales it to the video width and composites it bottom-left for the opening ~5s via one ffmpeg overlay (enable + shortest), after the optional caption burn. Audio-only exports and unreadable video sizes skip it; a rasterization hiccup exports without it. The export dialog gains an "Add MIE title bar" toggle + title field. Verified end-to-end on a real 2160x3840 phone capture: the MIE-green band with title and byline composited over the video.
The ✨ button ran one fixed pass — strip fillers, false starts, repeats,
and dead air — with no way to say what you actually wanted. The server
already accepted an `instructions` field and threaded it into the prompt;
the client never sent one. This wires up that path.
Clicking ✨ now opens a dialog (Export-dialog pattern: ui Modal +
Textarea + Button, theme-aware) with an optional free-text field. Blank
keeps the old behaviour exactly, so the one-click cleanup is unchanged.
Instructions alone weren't enough to make it work, though — two things
downstream fought them:
- The system prompt said "delete ONLY fillers... NEVER delete words that
carry meaning." Every useful direction ("cut this to 60 seconds", "drop
the pricing tangent") requires cutting meaningful content, so the model
was being told to ignore the request. The prompt now splits: the
conservative cleanup rules apply when no direction is given, and a
directed pass where the editor's instruction outranks them otherwise.
- The 50% deletion cap silently truncated any real edit. A directed pass
gets 90% — still a rail against a model returning every index, but it
no longer blocks the feature.
The directed prompt also gets the recording's duration and a words-per-
minute hint so length targets mean something.
Also bumps the default AGENT_MODEL from claude-opus-4-8 to claude-opus-5
(same price, better judgment on exactly this kind of call).
Verified against a local Ollama (qwen2.5:7b-instruct, no key), driving
the real UI in a browser: typed an instruction, submitted, and the job
came back having deleted 247 of 275 words — exactly the directed 90% cap,
which the cleanup path would have held at 137, so the instruction
provably reached the model. The editor remounted on the new epoch and
showed the proposal with a one-⌘Z undo snapshot. The no-instruction pass
was checked separately and stayed on the conservative cap. Both CI build
steps (build:server, build:client) and both typechecks are clean.
Untested: the Anthropic provider branch, including the new model default
— every run so far has gone through the OpenAI-compatible path.
qwen2.5:7b, asked to strip fillers from a 283-word transcript, returned 281 consecutive indices — it emitted `2, 3, 4, …` and never stopped. Small models fall into that attractor easily: once a few integers are on the tape, the next integer is always the likeliest token. The old code capped the runaway at 50% and applied it, so a counting bug became a silent 141-word cut through the middle of a sentence, and the editor showed it as a considered edit. The validator checked that indices were in range, unique, and under a cap — nothing about shape. But shape is exactly what separates an edit from a range: real filler cleanup comes out in ones and twos, so one unbroken 15+ word run is a paragraph of speech, not disfluency. Two rules now reject rather than truncate: - Cleanup pass: longest run >= 15 words. A directed pass is exempt — "drop the part about pricing" is supposed to cut one long block. - Either pass: proposal exceeds the cap AND >=90% of it is a single run. Truncating that doesn't rescue the edit, it just picks an arbitrary place to stop counting. Failing is the right outcome. The job errors, the status endpoint 500s with the message, and the client already surfaces `status.message` in the alert and flips the button to "AI edit failed" — so the person sees "the agent returned 281 consecutive words to delete" instead of a mangled timeline they have to notice on their own. Verified against the same Ollama setup that produced the original runaway: the cleanup pass now fails with that message instead of writing a 141-word cut. build:server clean.
The agent could only return a list of words to delete, and its prompt said
so outright: "you can only remove words — you cannot add or reorder them."
Meanwhile export.ts walks the edit list in array order and bakes per-segment
speed with setpts/atempo, so deleting, reordering, and re-timing were all
renderable already. The renderer was never the limit; the agent's vocabulary
was. It now speaks the same three verbs a person does:
{"op":"delete","from":N,"to":M}
{"op":"speed","from":N,"to":M,"rate":1.5}
{"op":"move","from":N,"to":M,"before":K}
An operation list rather than parallel arrays, because ops compose — move a
section and speed it up — and adding a verb later is one more case, not
another top-level field.
Every index refers to the ORIGINAL numbering and the server resolves them
all against one snapshot. Models are unreliable at renumbering after their
own edits, so they are never asked to; ops are applied by entry identity,
not position, and speed markers are derived only after the ordering settles
(they key off final array positions, so they cannot be computed sooner).
Validation rejects rather than repairs — out-of-range spans, unknown ops,
rates outside the renderer's 0.5–2, overlapping same-kind ops, a move
targeting its own interior, moving what it also deletes, or deleting past
90% of the track. A half-understood edit applied silently is worse than a
clear failure the person can retry.
Filler cleanup is gone as a model call. The ✂️ modal already does it
deterministically, and whether fillers exist at all depends on the ASR —
whisper base.en strips them upstream, which is why both capable models
returned zero deletions when asked. An instruction is now required; silence
removal stays procedural and free.
Checkpoints replace the word-only undo snapshot. Each run stores the
COMPLETE state (words, speed markers, default speed) labelled with the
instruction that produced it, and POST /edits/restore rolls back to any of
them. This is not a nicety: the old snapshot captured words alone, so the
moment ops could change speed and order, a rollback would have restored old
words beside new markers — a mixed state that never existed. Restoring
doesn't truncate the history, so a rollback is itself reversible. The
editor's ⌘Z is untouched; these are commits, not undo.
Verified end to end. 21 unit tests on the applier with no model involved,
covering original-index stability across a move+delete and speed markers
following moved words. Live: "cut this to about 40 seconds, keep the
Whisper opening, drop the AssemblyAI story, play the fixes at 1.5x" →
claude-sonnet-5 returned two deletes and a speed, 211/283 words cut, one
marker at the right word; the free gpt-oss-120b on Groq produced an
equivalent edit from the same instruction. Browser: the proposal remounts
into the editor with a 1.5x chip inline, History lists both checkpoints
with the model's summary, and restore works in both directions.
build:server and build:client clean.
The Anthropic branch capped max_tokens at 2048, sized for the JSON the model returns. Current Claude models think by default and thinking is billed and budgeted as output, so on the longer op-list prompt Sonnet 5 spent the entire ceiling reasoning and returned a response with no text block at all. The endpoint failed with "Anthropic API returned no text content", which describes the symptom and hides the cause. Raises the ceiling to 8192 and, when a response still comes back empty, reports stop_reason — so a truncated turn says it ran out of budget instead of looking like a broken API. Found by exporting an agent edit for the first time: two runs died here before any video rendered.
Two gaps that made "agent" the wrong word for this. It never saw the consequence of its own edit, and it never remembered making one. **It can check its work now.** Length is the thing a single-shot model reliably misjudges — it estimates from a word count instead of measuring — so it now declares a `targetSeconds` when the instruction names a length. The server applies the ops, computes the real duration through buildExportPlan (the same planner the renderer uses, so the number is the rendered length rather than an approximation), and hands a miss back: "that came to 78s, you were aiming for 60, revise." Bounded at 2 calls by default, AGENT_MAX_ROUNDS raises it to 5. The loop only runs when there is something measurable to check. No target declared, no second call, no extra cost — and a length miss is objectively verifiable, unlike "is this a good edit", so the feedback is real rather than the model grading itself. **It remembers.** Checkpoints already stored the ops and instruction for every run; nothing read them back. The previous turn now goes into the prompt, so "actually that's too short, make it 50 seconds" resolves against what it actually did. This is nearly free because every run already re-plans from the original transcript — a follow-up revises the plan rather than patching the result, so there is no renumbering and no compounding drift. Checkpoints also store durationMs now, so the model is told how long its own last edit ran. A validation failure stays fatal rather than triggering a revision. A malformed plan means the model misunderstood the task; asking again mostly spends another call. Also gates POST /edits/restore behind requireAuth, matching agent-edit. Both no-op when SECRET_KEY is unset, so local dev is unaffected, but neither should be open on a box where a request costs money or burns a shared rate limit. Verified: 17 new unit tests stubbing the model — stops at one call with no target, does not revise a good answer, revises and keeps the corrected plan, gives up at the cap while still returning a proposal, honours AGENT_MAX_ROUNDS, and fails without a second call on a bad plan. Live on gpt-oss-120b: "cut to about 30 seconds" used both rounds and landed at 25s; the follow-up "that's too short, make it 50" read the stored 24.9s, aimed at 50 and reached 41s. Direction is right, convergence in two rounds is not exact — it is a proposal for review, so a near miss is trimmable by hand. 21 op tests still pass; build:server and build:client clean.
Three failures in a row, all the same mistake: every max_tokens I picked was sized for the JSON the model returns, and current models spend output tokens *thinking* before they write it. - The OpenAI path had no ceiling at all, so the provider default applied. gpt-oss-120b came back with an empty `content` and the endpoint reported "LLM API returned no message content" — accurate and useless. - Raising it to 8192 there made it worse. Groq's free tier meters the RESERVATION, not the usage: prompt + max_tokens = 10,035 against an 8,000/minute cap, so every request was rejected before the model ran. - Anthropic's 8192 was too low for the same reason as the first: Sonnet spent the entire budget reasoning and returned no text block. The two providers need opposite treatment, which is why fixing one kept breaking the other. Anthropic gets 32000 so thinking fits; OpenAI-compatible gets 4096 so the reservation fits inside a free tier's per-minute budget. Both read AGENT_MAX_OUTPUT_TOKENS to override. Also: reasoning models can put the answer in `message.reasoning` with `content` empty, so the OpenAI path now falls back to it — extractJson pulls the object out of either. And an empty reply reports finish_reason instead of a bare "no content", so the next one of these is diagnosable from the error alone. Verified against both providers on the same instruction: gpt-oss-120b returned move/speed/delete/delete, claude-sonnet-5 returned move/delete/speed/delete. 38 unit tests still pass.
It is captured from whatever edits.json held when the agent first ran on an artipod — which may already carry hand edits or stripped silences. On the demo clip that meant restoring "Original" left 20 silence chips cut, which reads as the rollback having failed. The state is right; the label was describing something it never was.
The agent is the only endpoint that does a read-modify-write on edits.json across an await: it reads the file, spends up to a minute in the model, then writes back. Two runs on the same pulse both read the same state, both write, and the loser's checkpoint is gone — the person sees an edit they didn't ask for and a history entry that silently disappeared. A per-artipod lock now holds the pulse for the life of a run; a second request gets 409 with the job id that holds it. Rejecting beats queueing here: the caller learns their run didn't happen, which a silent overwrite never tells them, and a queued second edit would be planned against state the first one was about to replace anyway. Released in a finally so a failed run frees the pulse instead of wedging it, and only if this job still owns the lock — a run finishing late must not release a lock that a later run has already taken. Client prefers `message` over `error` on a failed POST, so the 409 surfaces as "An AI edit is already running on this pulse" rather than "Busy". Verified against a stub that holds the connection open: first run accepted, second rejected 409 naming the holder, lock released after both success and failure, and an unknown artipod still 404s rather than 409ing. 7 assertions, plus the existing 38.
Agent runs were already checkpointed and rollback-able by name. Hand edits
were not saved as anything, so half the work done to a pulse had no history
at all. They share one timeline now.
Deliberately one list rather than two. They are edits to the same document, so
two lists would mean two competing notions of "current" and restoring in one
would silently invalidate the other's position. Kind is a badge and a filter.
Hand edits coalesce into bursts rather than one version per click: keep editing
and the entry grows and relabels itself; pause 30s, or hit a 2-minute ceiling,
and the next edit starts a new one. Labels are generated from the state diff
("Removed 12 words - Reordered 2 words - Sped up 1 region") because asking
people to name every version means versions never get named.
Undo and redo are a cursor moving along that timeline. Restoring never
truncates, so stepping back and forward is symmetric; only a NEW edit made
while rewound abandons the versions ahead. Cmd+Z steps the timeline when the
current version is an AI run - one run can delete two hundred words and undoing
it a word at a time is not undoing it - or when the editor has no word-level
steps left. In between it falls through untouched, so fine-grained undo still
works. Cmd+Y and Cmd+Shift+Z redo, which the editor never had.
Three things the diff had to get right:
- Deletion is a FLAG, not a removal, so the diff is mostly a comparison of
flags. That is also why the untouched original can be reconstructed from any
later state to seed checkpoint 0 for a pulse that has only been hand-edited.
- Moves are detected by longest-increasing-subsequence over the words common to
both sides, so reordering one sentence marks that sentence and not everything
it displaced. Verified against a real run: the words it marks are exactly the
closing line that run's instruction asked to move to the front.
- Speed markers are POSITIONAL (an index into editedWords), so the same marker
list means different things either side of a reorder. Comparing them directly
reports phantom changes. Both sides are resolved to a per-word rate and
compared by word identity, then counted in contiguous regions - one marker
over a long passage is one thing someone did, not ninety-one.
Word identity is originalIndex plus an occurrence ordinal: pasted words keep
the index they were copied from and split entries all carry -1, so the index
alone is not unique.
Also here, because the feature does not work without them:
- Editing was gated client-side on an API key while the server has PUT /edits
in the open participation tier, so hand edits were never saved at all on an
unlocked instance.
- /edits/restore joins that same tier. Undo and redo run through it and a key
requirement would break them on a locked instance while protecting nothing -
anyone who can reach it can already rewrite the same state through PUT /edits.
- GET /edits was returning a full copy of the edit list for every checkpoint,
713KB on a three-minute video, on every page load. It returns metadata now
and diffs are fetched per version on demand. 713KB -> 42KB.
- A new transcript rebuilds the whole edit list and the editor saves that
rebuild on mount. It is not an edit anyone made and its words are indexed
against a different baseline, so recording it invented a version nobody made.
Suppressed until someone actually touches the editor.
67 unit tests in agent-test-history.mjs covering the diff, burst coalescing,
truncate-forward, eviction and the cursor.
The version Undo/Redo sat in the page header while the editor kept its own word-level Undo down in the stats bar. Two Undos in two places, and the one you wanted depended on what you had just done — which is not something anyone should have to reason about. They are one control now, in the editor toolbar, using the host hooks added in mieweb/ui#354: word-level steps are spent first and the same button carries on into the version timeline once they run out. Exactly the layering ⌘Z already had, now visible instead of implied. Redo moves there too, and both show their shortcut in a kbd like Del, Cut and Paste do. The header keeps only the History toggle, which opens a panel rather than performing an edit.
The consolidated Undo needs the MediaEditor props from mieweb/ui#354, which is not merged yet. Both this URL and the pin revert to mieweb/ui once it lands.
Author
|
Follow-up: the Undo/Redo pair has moved into the editor toolbar, next to the ✂️ and the speed control, rather than sitting in the page header. The reason it needed moving: They are one control now. Word-level steps are spent first, and the same button carries on into the version timeline once they run out — the layering ⌘Z already had, made visible. Redo moved too, and both now show their shortcut in a That needed two new props on
Both are live on https://pulseclip-dev2.os.mieweb.org if you want to click them. |
A permanently greyed Redo on a pulse that has never been edited reads as a broken button rather than an inapplicable one. Gated on the same condition as the History button, and paired with mieweb/ui#354 keeping Undo and Redo on screen together so neither disappears out from under the other.
Author
|
Bug found on dev2 and fixed: Redo was permanently greyed on a pulse with no history, with no Undo beside it — which reads as a broken button rather than an inapplicable one. Two causes, one each side:
Verified live on dev2 across all three states: no history → neither button · at the tip → Undo enabled naming its target, Redo greyed · at the original → Undo greyed, Redo enabled naming its target. Round trip both ways. ui submodule pin moved to |
⌘Y is Chrome's History shortcut on macOS, so pressing it opened a browser tab rather than redoing. Still accepted for anyone arriving from Windows, but no longer advertised. Picks up the word-level redo added in mieweb/ui#354, which is what makes Redo live after an ordinary word undo.
Author
|
Second round of fixes, both from testing on dev2. Redo genuinely did not work, and the reason was deeper than the gating.
⌘Y was the wrong key on macOS. It is Chrome's own History shortcut, so pressing it opened a browser tab instead of redoing. The button advertises ⇧⌘Z now; ⌘Y is still accepted for anyone arriving from Windows, but nothing tells a Mac user to press it. Verified live on dev2, the exact case from the report:
Round trip both ways. ui submodule pin moved to |
The agent has always run on whatever key the server holds. That key is billed to whoever runs the server and rate-limited across everyone at once — on Groq's free tier the binding limit is 8,000 tokens a MINUTE, and because the reservation is metered rather than the usage, one edit of a long enough transcript can exceed it with nobody else even using it. Anyone doing real work should be able to point this at their own account. The AI-edit dialog now says which account is about to pay, and offers to swap it: presets for Groq, Anthropic and OpenAI, or any OpenAI-compatible base. The key lives in localStorage, travels with the request that uses it, and is never logged, never written into edits.json, and never echoed back. The app key stops being required when a caller brings their own provider. It was only ever there because an LLM call spends money or burns a shared rate limit, unlike CPU work — and that reasoning does not apply to someone spending their own. Nothing else this endpoint touches is newly reachable: the artipod and its edits are already writable through the open edit routes. A caller-supplied base URL means THIS server makes the outbound request, from inside whatever the box can reach — the classic SSRF shape. So the base must be https, and loopback, link-local, cloud-metadata and RFC1918 hosts are refused. A hostname that RESOLVES into those ranges still gets through; closing that needs resolution at connect time, which the platform fetch does not expose. The floor is deliberate and commented as such. An invalid provider is rejected rather than ignored, and rejected loudly. The quiet failure — falling back to the shared key someone was specifically trying not to spend — is the one that costs money without telling anyone. The route also answers a bad provider with the real reason: replying "valid API key required" to a mistyped base URL sends someone off to fix the wrong thing. The ✨ button no longer hides when the server has no LLM. It hid because it would only have failed; now it leads somewhere useful, and on an instance with nothing configured the dialog opens straight into provider setup. Without that there is no way to discover bring-your-own-key on exactly the instances that need it. 33 tests in agent-test-byo.mjs — validation, rejection messages, and every private range. Verified live: a caller-supplied model reached the provider (a deliberately bogus one came back naming itself, proving the override), and a real run returned real ops. The 429 in between was the shared-tier limit doing exactly what this change exists to route around.
Author
|
Adds bring-your-own-key, so the agent can run on the caller's LLM account rather than the server's. The shared key is billed to whoever runs the server and rate-limited across everyone at once. On Groq's free tier the binding limit is 8,000 tokens per minute, and because the reservation is metered rather than the usage, a single edit of a long enough transcript can exceed it with nobody else using it at all. (I hit a real 429 while testing this, which was a fair demonstration.) The AI-edit dialog now names the account about to pay and offers to swap it — presets for Groq, Anthropic and OpenAI, or any OpenAI-compatible base. The key lives in localStorage, travels with the request that uses it, and is never logged, never written into The app key stops being required when a caller brings their own provider. It was only ever there because an LLM call spends money or burns a shared rate limit, unlike CPU work — and that reasoning does not apply to someone spending their own. Nothing else becomes reachable: the artipod and its edits are already writable through the open edit routes. A caller-supplied base URL means this server makes the outbound request, from inside whatever the box can reach — the classic SSRF shape. So the base must be https, and loopback, link-local, cloud-metadata and RFC1918 hosts are refused. A hostname that resolves into those ranges still gets through; closing that needs resolution at connect time, which the platform fetch does not expose. The floor is deliberate and commented as such — worth a second opinion. Two smaller calls worth flagging:
33 tests in |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Agent runs were already checkpointed and rollback-able by name. Hand edits were not saved as anything, so half the work done to a pulse had no history at all. They share one timeline now, with undo, redo, and a diff.
One list, not two
They are edits to the same document. Two lists would mean two competing notions of "current", and restoring in one would silently invalidate the other's position. Kind is a badge and a filter instead — ✨ AI, ✋ by hand, ◎ original.
The panel sits beside the editor rather than in a modal: comparing a version against what is on screen is the point, and a modal hides the thing you are comparing against.
Hand edits become versions, in bursts
A version per click would bury the agent runs people come back for. A burst of edits coalesces into one entry — keep editing and it grows and relabels itself; pause 30s, or hit a 2-minute ceiling, and the next edit starts a new one.
Labels are generated from the state diff — "Removed 12 words · Reordered 2 words · Sped up 1 region" — because asking people to name every version means versions never get named. Rename any of them; renaming also closes that entry to further amendment.
Undo and redo
A
historyIndexcursor inedits.json. Restoring never truncates, so stepping back and forward is symmetric; only a new edit made while rewound abandons the versions ahead, which is what redo means everywhere else.⌘Zsteps the timeline when the current version is an AI run — one run can delete two hundred words, and undoing it a word at a time is not undoing it — or when the editor has no word-level steps left. In between it falls through untouched, so fine-grained undo still works.⌘Yand⌘⇧Zredo, which the editor never had.Three things the diff had to get right
wordIndexintoeditedWords), so the same marker list means different things either side of a reorder, and comparing them directly reports phantom changes. Both sides are resolved to a per-word rate and compared by identity, then counted in contiguous regions — one marker over a long passage is one thing someone did, not ninety-one words.Word identity is
originalIndexplus an occurrence ordinal: pasted words keep the index they were copied from and split entries all carry-1, so the index alone is not unique.In the diff: bright red = cut in this version, grey = already cut (context, not news), green = brought back, purple ring = moved, dotted underline = re-timed. Purple deliberately is not a brand token — a moved word is often also a restored word, and the two have to stay tellable apart.
Also here, because the feature does not work without them
PUT /editsin the open participation tier. Hand edits were never saved at all on an unlocked instance./edits/restorejoins that same tier. Undo and redo run through it, and a key requirement would break them on a locked instance while protecting nothing — anyone who can reach it can already rewrite the same state throughPUT /edits. Happy to revert this if you would rather it stayed closed.GET /editswas returning a full copy of the edit list for every checkpoint — 713KB on a three-minute video, on every page load. Metadata only now, diffs fetched per version on demand. 713KB → 42KB.Testing
67 unit tests in
agent-test-history.mjs(the diff, burst coalescing, truncate-forward, eviction, the cursor). The existing 21 op-applier and 17 agent-loop tests still pass.Verified in a real browser against a pulse with 8 genuine agent runs: hand edit → version appears with a generated label; second edit after the window → its own version;
⌘Z8→7 on an AI run;⌘Zon a hand edit correctly does not intercept;⌘Y7→8.Known limitation
Re-transcribing a pulse that already has history leaves the old versions indexed against a different baseline. Phantom versions are suppressed, but the first real edit afterwards will produce a noisy diff. Fixing it properly means either dropping history on re-transcribe or marking a baseline boundary — a product call, not done here.