feat(preview): retain the latest denoise frame server-side and replay it on reconnect - #215
Merged
Conversation
… it on reconnect
Progress frames travelled over the socket as base64 data URLs and existed
nowhere else. A client whose socket was dropped while its tab was hidden, or a
page that reloaded mid-run, had nothing to show until the next denoising step,
and nothing at all for a run that finished while it was away.
Phase 2 of the preview restructuring (additive, no schema removals):
- New `progress_previews` service on InvocationServices: an in-memory,
lock-guarded map of the last emitted `invocation_progress` event per running
queue item. Cleared when the session ends, whatever the outcome. No files, no
TTL sweeps: a process restart empties it, which is the intended epoch.
- `InvocationProgressEvent.revision`: monotonic per queue item and session on
image-bearing frames. The client drops any frame at or below the revision it
has shown, so a replay can never move the preview backwards.
- `signal_progress` throttles image-bearing frames to one per 100 ms per item
(final and indeterminate frames always pass), before the JPEG encode.
- `subscribe_queue` replays the user's stored frames to the subscribing socket.
- `GET /queue/{queue_id}/previews` returns the caller's frames as
`ProgressPreviewDTO` (a dedicated model: exposing the event as a response
model gave it a second OpenAPI schema variant that made every field optional
in the generated client types).
- webv2: the queue coordinator fetches the snapshot when the tab becomes
visible and after re-adopting runs on reload, feeding each payload through
the socket handler's revision gate. The gate reopens when an item goes back
to pending/waiting.
- A frame is dropped from the store as soon as its node completes or errors,
so a replay never shows a finished denoise as running; the revision counter
survives `clear` (bounded) so a workflow-call parent resuming with the same
item and session keeps issuing higher revisions than any a client holds;
the processor-error path clears the item too.
- The SD/SDXL preview extension now reports steps completed rather than the
index of the step just run, so its last frame reads 100% like the FLUX and
SD3 callbacks and the throttle never drops it as a mid-run frame.
Legacy `web` schema regenerated (additive only); its typecheck passes.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DoHBJ9QPWfq1qMTfzdzuke
lstein
requested review from
JPPhoto,
Pfannkuchensack and
blessedcoolant
as code owners
September 6, 2026 17:13
…ew store The openapi-checks workflow diffs `invokeai/frontend/web/openapi.json` (separate from the typegen check's `schema.ts`); regenerated for the new previews route and the `revision` field. The `ProgressImage` component only reorders keys. `test_socket_privilege_revocation.py` binds `ApiDependencies` to hand-built services stubs; the subscribe replay now reads `progress_previews` from them. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DoHBJ9QPWfq1qMTfzdzuke
4 tasks
With two slots live — a long video render next to a quick image batch — the single-frame preview read the store-wide "latest" frame and matched it to the followed placeholder. The latest belonged to whichever slot stepped last, and releasing that slot when its batch finished nulled it outright. The video slot's own frame was still stored per slot (the gallery cell kept showing it), but the preview rendered an empty card until the video's next step, minutes away. Reloading fixed it only because reconcile re-fed the frame. LivePreview now reads its followed slot's frame directly, and the store's latest falls back to the most recently updated remaining slot instead of null so the header toggle and the editor's Current Image node do not blank either. The store-wide matcher is gone. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DoHBJ9QPWfq1qMTfzdzuke (cherry picked from commit 0f36ead)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Phase 2 of the preview restructuring, stacked on #214 (base branch
fix/preview-coordinator; merge that first, then retarget this tomain).Progress frames travelled over the socket as base64 data URLs and existed nowhere else. A client whose socket was dropped while its tab was hidden, or a page reloaded mid-run, had nothing to show until the next denoising step. This adds the two server-side pieces JPPhoto's proposal got right, without the ephemeral file store or URL-based delivery: the server keeps the last emitted frame per running item in memory with a monotonic revision, and hands it out on reconnect.
What changes
Backend (additive)
invokeai/app/services/progress_previews/:MemoryProgressPreviews, one entry per running queue item, lock-guarded (multi-GPU workers). Cleared in_on_after_run_sessionwhatever the outcome. Registered onInvocationServicesasprogress_previews, default-constructed when omitted so existing test constructors keep working.InvocationProgressEvent.revision: int | None, monotonic per item + session on image-bearing frames.emit_invocation_progresstakes it and returns the event.signal_progress: image frames reserve a revision from the store; frames inside 100 ms of the previous one for the same item are dropped before the JPEG encode (finalpercentage >= 1and indeterminate frames always pass). Imageless progress is untouched.subscribe_queuereplays the user's stored frames to the subscribing socket only; a socket that never lost them drops the duplicates via the revision gate.GET /api/v1/queue/{queue_id}/previews→list[ProgressPreviewDTO], owner-scoped (progress is personal UI, so even admins see only their own). A dedicated DTO rather than the event model: using the event as a response model made FastAPI emit a second schema variant that turned every field optional in the legacy client's generated types._on_after_run_node/_on_node_error), so a replay never re-marks a finished denoise node as running. The revision counter survivesclear(bounded to 4096 frameless counters), so a workflow-call parent that suspends onwaitingand resumes with the same item + session keeps issuing higher revisions than any a client holds. The processor-error path clears the item as well.PreviewExtnow reports steps completed (step_index + 1) rather than the 0-based index, so its last frame reads 100% like the FLUX/SD3 callbacks do and is never throttled as a mid-run frame. Side effect: the SD progress bar reaches 100% at the last step.webschema.tsregenerated; diff is purely additive and itstscpasses.webv2
QueueBackendPort.readProgressPreviews,getProgressPreviewsin the server API, andcoordinator.refreshProgressPreviews(): fetched when the tab becomes visible (alongside the sweep) and afterreconcile()re-adopts runs on reload. Each payload goes through the same handler as the socket event, where the revision gate from fix(preview): keep the last denoise frame through completion, reconnect and tab hiding #214 drops anything the live stream already delivered.Contract (from the Discord thread)
Test plan
ruff check/ruff format --checkcleansignal_progresswiring, socket replay on subscribe, REST route scoping (admin sees only own; 401 unauthenticated)tests/app/services, session-queue router tests, socket tests,tests/test_session_queue.py,tests/test_nodes.pypnpm lint,pnpm test(7597), coordinator tests for the snapshot on visibility and after reloadweb:tsc --noEmitagainst the regenerated schema🤖 Generated with Claude Code
https://claude.ai/code/session_01DoHBJ9QPWfq1qMTfzdzuke