Repository navigation
[lexical-y] Bug Fix: A delete keeps every format item it leaves (D86), and a client keeps its own format items from garbage collection while they are undoable (D88) - #9679
Conversation
…osed D86) ## Description Proposed for review: DECISIONS.md D86 is `OPEN` (the number may shift at landing). A delete collapsed the run of deleted content and format items it left (`Cursor#collapseGap`, D24's bound per format key): it kept a few items per key and deleted the rest. The ends it deleted were sometimes the ones a concurrent writer's restore, or an undo, relied on, and the value then ran on past its end: a comment over the following words and line (#9575), a comment's restore after a collaborator's plain text inside it (#9609), a comment its author undid (a #9620 case), the concurrent restore cases of #9640. Keeping the right ends inside the collapse (#9617) needed to know which write relies on which item, which the document does not store. This removes the collapse: a delete keeps every format item it leaves. Yjs's own format cleanup stays off (D24). `Cursor` loses `collapseGap` and the gap tracking it used (`gapStart`, `gapFormats`); comments that pointed at it now point at `Cursor#delete`. Cost: each formatted run (a format, a comment, a NodeState value) whose text is deleted keeps its two items, its opener and its closer. Plain typing and deleting add none, and neither does typing inside a formatted run. y.md gets a Limits entry (Deleted formatted text), and its Write rules paragraph and Undo paragraph lose the collapse's limits. Tests changed where they encoded the collapse, each with the reason in a comment: - `regressions/Formats.test.ts`, F24A-1, and `CursorFamily.test.ts`: words of a format, comment or set member typed and deleted again at one place leave exactly two items each (40, 80, 120 after 20, 40, 60 words) instead of a bound. - `Cursor.test.ts`: a format change then a delete on one cursor keeps all five items after the range. - `Races.test.ts`, #9575: the comment ends where it ended (was `test.fails`); the cell that pinned the spread "as documented" is gone. - `regressions/CollapseKeepsEnds.test.ts`, #9609 (12 cells), `regressions/KeepEndsSessions.test.ts` (4694), `regressions/UndoRestoredRunEnds.test.ts` (#9620, 2 cells) and `regressions/UndoRestoredRunEndExtents.test.ts` (the 10 concurrent empty restore cells): were `test.fails`, pass. - `CollapseKeepsEndsProperty.test.ts` is removed: it compared every session with a run of the same session with the collapse turned off, which is now the only run. ## Test plan Unit tests of lexical-y, lexical-mark, lexical, lexical-selection, lexical-list, lexical-table, lexical-html, lexical-extension and lexical-playground: 24,546 passed, 395 expected fail, 0 failed (`DisposeMemory.test.ts`'s heap bound failed once under a 16-worker run and passes alone and in reruns, here and on main). `pnpm run ci-check` passes. `MATRIX_FULL=1` matrix: see the body. Measured (the coordinator's runs and mine, same seeds, against main): - A three-client view fuzz, 15,000 seeds: 0 seeds where this is wrong and main is right; 631 where main spreads a value and this does not. - Growth (measured on 1853707 with the collapse turned off, the same rule): F24A-1 above; a comment added and its word deleted 3,000 times at one place: 6,002 live format items (main 6), 41 ms per edit (main 27); adding and removing a comment 1,000 times: 1 live item (as main); 1,000 keystrokes inside a commented run: none added. - Emily: x1.003 at 10 cycles, x1.26 at 150. Jot: x1.01 on its default profile; a concentrated recolor 438 items (main 345).
…to the collapse ## Description KeepEndsSessions' 3645 cell spied on the removed `Cursor#collapseGap` inside `test.fails`, so it passed by throwing before its comparison. It now asserts what the session shows: c9 on every character of "wxq4xq56abcd". Test comments that pointed at `Cursor#collapseGap` now point at `Cursor#delete` or say the collapse is gone (D86); no test references `collapseGap`, `gapStart` or `gapFormats` any more. ## Test plan KeepEndsSessions passes (3645 as a plain test). Units of lexical-y, lexical-mark, lexical, lexical-selection, lexical-list, lexical-table, lexical-html, lexical-extension and lexical-playground: 24,547 passed, 394 expected fail, 0 failed. `pnpm run ci-check` passes.
## Description D86 (a delete keeps every format item it leaves) moves from `OPEN` to `DECIDED` (10-31): the linear growth with deleted formatted text is accepted over the bound and its spreads. The row moves from Open to Storage model, next to D24, whose bound per format key it removes; D24 says so. ## Test plan Docs only: DECISIONS.md rows D24 and D86; `prettier --check` passes.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
AI review status for this pull request.
|
lexical
@lexical/a11y
@lexical/clipboard
@lexical/code
@lexical/code-core
@lexical/code-prism
@lexical/code-shiki
@lexical/compiler
@lexical/devtools-core
@lexical/dragon
@lexical/eslint-plugin
@lexical/extension
@lexical/fast-check
@lexical/file
@lexical/hashtag
@lexical/headless
@lexical/history
@lexical/html
@lexical/internal
@lexical/link
@lexical/list
@lexical/mark
@lexical/markdown
@lexical/mdast
@lexical/offset
@lexical/overflow
@lexical/plain-text
@lexical/react
@lexical/rich-text
@lexical/selection
@lexical/table
@lexical/tailwind
@lexical/text
@lexical/utils
@lexical/y
@lexical/yjs
commit: |
…keep the delete-next-to-typing convergence property ## Description Review follow-ups for D86: - DECISIONS.md: D51 (the #9575 spread) is replaced by D86, whose cells pass; D65 no longer argues from the growth bound D86 removed and says which #9609 cells still fail; D86 names its corpus measurement without an internal name. `Cursor#delete` cites D86. - `DeleteNextToTyping.test.ts` keeps the randomized sessions of the removed `CollapseKeepsEndsProperty.test.ts` (one peer types inside a commented word while the other deletes around it, taking turns and at the same time) with the checks that never depended on the collapse: no errors, every peer and a fresh reader agree. - Test names and comments that described the collapse in the present tense now say it is gone (D86). - The changeset says documents keep two format items per formatted run whose text is deleted. ## Test plan `DeleteNextToTyping.test.ts`: 4 passed (60 sessions typing ‖ deleting and 20 taking turns per client id order). The renamed tests pass. `prettier --check`, `eslint --no-cache` on the changed files and the changeset check pass.
9817eb6 to
9148d75
Compare
9148d75 to
a6f5aaf
Compare
a6f5aaf to
71bd7f4
Compare
71bd7f4 to
2b39655
Compare
2b39655 to
31df93c
Compare
There was a problem hiding this comment.
This looks good to me. Keeping a client's own format items readable while it can still undo fixes #9623 under Yjs's default garbage collection, and the Cursor changes for #9599 and #9671 bring the binding closer to what Yjs's own formatText does. The new regression tests assert both peers and a fresh reader, and they cover both client id orders with gc on and off. The suggestions below are optional and non-blocking.
Optional suggestions
- Guard the rule-3 copy end (
packages/lexical-y/src/internal/Cursor.ts:1244-1249). The new code takeschangeEnd(doc, next)and writes a null after it throughinsertAfter, which always usesthis.parent(line 791). It never checks that the end is in the sameY.Nodeor to the right of the copy.restatedEnd(lines 767-777) gets that guarantee by walking right from the cursor.changeEndis a clock-order guess, so a check such asend.parent === this.parentplus a rightward scan would keep a misidentified end from writing a null somewhere else. In the existing suites the end is always ahead in the same node, so this is hardening rather than a reproduced bug. - Document D88's local cost in y.md's Limits.
packages/lexical-y/AGENTS.mdasks for accepted limits to appear in y.md's Limits (packages/lexical-website/docs/collaboration/y.md) as well as DECISIONS.md. The new D88 row accepts that a client keeps the format items collaborators delete while it can undo, and that those items are in any state it encodes until it reloads. For example, after 100 rounds of one client bolding and another clearing formats, the first client's encoded state is 2,603 bytes against 229 for the second. One sentence next to "Deleted formatted text" would cover it. While there, "against 27 ms before" (line 1108) could say what "before" means. - Pin the gc-on failures of the new lane. With
LEXICAL_Y_GC=1, three cells ofregressions/Formats.test.tsF30A-1 ("italic over "b ", italic removed from "b"; the removal undone ‖ "b" italic again", ids 1,2,3 / 3,2,1 / 2,3,1) still fail. They also fail without this change, so this is not a regression.packages/lexical-y/AGENTS.mdasks for a known failure to be atest.failsnaming its issue, which would keep the lane usable. - Test plan output. AGENTS.md (Commits and Pull Requests) asks the Test plan for the real Before and After output of the commands run. The gate results are still "to be added", and the description does not mention
OwnFormatItemsRelease.test.tsorUndoSpreadSession2810.test.ts. - Small test nit. In
OwnFormatItemsRelease.test.ts, "an undo that changes nothing and empties both stacks releases them" callsundoManager.clear()after the undo (line 183), so the clear's own release would pass the test even without the undo path. Dropping that line makes the test check what its name says.
| if (value !== null && next.id.client !== this.tr.doc.clientID) { | ||
| const end = changeEnd(this.tr.doc, next); | ||
| if (end !== null && !end.deleted && valueOf(end) === null) { |
There was a problem hiding this comment.
Optional: changeEnd is a clock-order guess, and insertAfter below always uses this.parent (line 791). Unlike restatedEnd (lines 767-777), which finds its end by walking right from the cursor, nothing here checks that end is in this Y.Node and to the right of the copy. Something like end.parent === this.parent plus a rightward scan would keep a misidentified end (a later write of that client elsewhere) from getting a stray null.
There was a problem hiding this comment.
Not taken in this PR: in every suite and fuzz run the copied opener's end is ahead of the copy in the same node, so this would be hardening without a failing case, and any change to where rule 5 writes its null needs the full property and soak gates again. Noted as a follow-up (a same-parent check plus a rightward scan, like restatedEnd).
| formatted text is typed and deleted over and over. | ||
| formatted text is typed and deleted over and over, where edits also get | ||
| slower (a comment added and its word deleted 3,000 times at one place: | ||
| 41 ms per edit, against 27 ms before). |
There was a problem hiding this comment.
Optional: "before" could name what the 27 ms is measured against. This bullet's neighbourhood would also be a good place for D88's local cost, which packages/lexical-y/AGENTS.md asks to list under Limits: a client keeps the format items collaborators delete while it can undo the change that wrote them, and those items are in any state it encodes until it reloads.
There was a problem hiding this comment.
Done in d1807fb: the 27 ms now says what it is measured against, and Limits has a "Deleted format items a client can undo" entry with D88's local cost (about 25 bytes each; 2,603 vs 229 bytes after 100 rounds; a gc-on server or relay drops them). D88 links to it.
| format(b.editor, false); | ||
| sync(); | ||
| a.binding.undo(); | ||
| a.binding.undoManager!.clear(); |
There was a problem hiding this comment.
Nit: with this clear() the test passes through the stack-cleared release even if the undo path did not release. Without it the undo path does release (both stacks end empty and the kept items go to zero), so dropping this line makes the test check what its name says.
There was a problem hiding this comment.
Done in d1807fb: the clear() is gone, so the cell passes only through the undo path (B removed the bold, so A's undo changes nothing and drops its step).
| * Peers' Yjs garbage collection: off unless a test asks for it, or | ||
| * `LEXICAL_Y_GC=1` runs every harness test with it on (as products run). | ||
| */ | ||
| export const WITH_GC = process.env.LEXICAL_Y_GC === '1'; |
There was a problem hiding this comment.
Optional: with LEXICAL_Y_GC=1, three F30A-1 cells in regressions/Formats.test.ts still fail ("italic over "b " … "b" italic again", ids 1,2,3 / 3,2,1 / 2,3,1). They fail without this PR too, but packages/lexical-y/AGENTS.md asks for known failures to be test.fails naming an issue, which would keep this lane green.
There was a problem hiding this comment.
Agreed it should be pinned; the F30A-1 gc-on failure has no issue yet and fails the same way on main. It goes into the test-only follow-up that re-points this package's test.fails markers (filed issue + exact-predicate pin under LEXICAL_Y_GC=1), so this PR stays scoped.
31df93c to
04ac260
Compare
04ac260 to
337568c
Compare
…llection while they are undoable ## Description With Yjs's default garbage collection, which products use, an undo could not read the format items of its own change after a collaborator deleted them: they were collected, so the undo guards (`isKeptByUndo`, `changeEnd`, `closeRestoredRuns`) took them for other content, and the value an undo brought back ran on over the rest of the document (#9623). The unit harness built every peer with gc off, so most tests never saw it. - `Binding`: the binding sets Yjs's own `Item.keep` on the format items its transactions and its undo manager's insert, and on the end of each run they delete. It releases them, between transactions, when both undo stacks are empty (after a clear, an undo that leaves nothing, or a write they did not capture), and when it is disposed or its document destroyed (a queued release keeps the stronger request; a listener call under way at dispose keeps nothing; ownership is per listening lifetime, so a binding started again keeps what its new steps need). An item another binding or an `UndoManager` step also keeps stays kept. Releasing collects nothing, as the `UndoManager` collects nothing when it releases what its steps kept: a format item a collaborator deleted while kept keeps its deleted content in this client's document until it reloads. Nothing new is written. `YjsContract` checks `Item#keep` and `Transaction#local`. - `Cursor#runOpener`: a restore's scan stops at content of another client, so a client's earlier end and later opener with a collaborator's text between them are not one restored run (#9599). - `Cursor` rule 3: a range end that writes its own copy of another client's opener also ends it where that opener's run ended, so undoing that client's change leaves the text after the range alone. - `Cursor`: a format write removes the null and the restore that this client's own earlier insert left inside its range, as Yjs's `formatText` does, so a collaborator's typing no longer relies on items the insert's undo deletes (#9671). - Tests: `regressions/OwnFormatItems.test.ts` runs the #9623, #9599 and #9671 repros and a restated-opener case, gc on and off, every client id order. `regressions/OwnFormatItemsRelease.test.ts` checks the release: clears, an undo that empties both stacks, writes the stacks do not capture, dispose, a new binding on the same document, another binding's undo and an app `UndoManager` tracking a provider's updates after this one releases (also from inside that `UndoManager`'s capture), dispose or destroy while a release waits, dispose during the first capture, a restart before the old release ran, repeated cycles, and what a relay stores. `regressions/UndoSpreadSession2810.test.ts` replays a downstream app's undo-fuzz session where, with gc on, the parent commit put a comment on four paragraphs and main did not. The harness takes a `gc` option and `LEXICAL_Y_GC=1` runs every harness test with gc on. Pins that now pass are plain tests: `UndoRestoredRunEndExtents` (117 cells), `UndoRestoredRunEnds` (20; the D64 cells stay pinned), `UndoSpreadFuzz` (5 seeds; its documents use gc on), `UndoCommentRemovalSpread` (4), `RunSequenceUndo` seed 87. - DECISIONS.md: D88 (own format items kept from gc while undoable), D78 decided (the #9599 class fixed). y.md's Limits states D88's local cost. - Docs for D86 (#9678 review follow-ups): y.md's #9564 entry no longer says a delete drops the item that ended the comment, and the Deleted formatted text limit says edits at such a place get slower; two comments rewrapped and a test title updated. ## Test plan `OwnFormatItems.test.ts`: 72 passed; on the parent commit 44 fail (the issue cells with gc on for #9623; #9599, #9671 and the restated-opener color cells with gc on and off). `OwnFormatItemsRelease.test.ts`: 24 passed; without the release paths 5 fail, and collecting on release fails the other binding's and the app `UndoManager`'s undo with gc on, as does a release inside that `UndoManager`'s capture; the 4 dispose and destroy cells fail without the queued-release handling, and the restart cell (gc on) with ownership per binding instead of per lifetime. `UndoSpreadSession2810.test.ts`: 4 passed; on the parent commit the 2 gc-on cells fail (main passes all 4). The pins above flip from expected-fail to pass and nothing else changes in the lexical-y unit suite.
a8c8bf8 to
d1807fb
Compare
There was a problem hiding this comment.
This looks good to me. Keeping a client's own format items readable while it can still undo fixes #9623 under Yjs's default garbage collection, and the Cursor changes for #9599 and #9671 follow what Yjs's own formatText does. The new regression tests assert both peers and a fresh reader, in both client id orders with gc on and off, and they fail on the parent commit. The suggestions below are optional and non-blocking.
Optional suggestions
-
Release a container the
UndoManagerkept first (packages/lexical-y/src/internal/Binding.ts:1233-1237). An item that is already kept when the binding first keeps it is recorded asexternal(lines 1147-1151) and is never unkept on release (lines 1182-1184). When that earlier keeper is the binding's ownUndoManager, itsclear()unkeeps the item, this loop keeps it again, and it then stays kept until the document reloads. For example: delete a character inside a box container, then bold a word in it, then clear both stacks. The box's container item is still kept, and if a collaborator then deletes the box, this client's state is 267 bytes against the collaborator's 169; with the two steps in the other order both are 169. D88 says the binding releases what it kept once both stacks are empty. Marking the entry non-external before keeping it again fixes this, and the release and spread tests still pass:for (const it of kept) { if (!it.keep) { const entry = keepOwners.get(it); if (entry !== undefined) { entry.external = false; } it.keep = true; } }
-
Paste the Test plan's output. AGENTS.md (Commits and Pull Requests) asks for
### Beforeand### Aftersections with the command and its real output. The plan currently says the gates are still running, and it leaves outOwnFormatItemsRelease.test.tsandUndoSpreadSession2810.test.ts. -
Nit: D88's row in
packages/lexical-y/DECISIONS.mdgives thegcFilteralternative twice; one of the two sentences can go.
Thanks for agreeing to pin the three F30A-1 cells that fail with LEXICAL_Y_GC=1 (they fail the same way without this change) as test.fails with an issue, and for tracking the rule-3 copy-end guard as a follow-up.
| for (const it of kept) { | ||
| if (!it.keep) { | ||
| it.keep = true; | ||
| } | ||
| } |
There was a problem hiding this comment.
Optional: an item this binding first saw already kept is recorded as external (lines 1147-1151) and never unkept (lines 1182-1184). When the earlier keeper is this UndoManager, its clear() unkeeps the item, this loop keeps it again, and it stays kept until reload. Repro: delete a character inside a box container, bold a word in it, clear both stacks; the box's container item is still kept, and after a collaborator deletes the box this client's state is 267 bytes against 169 (169 for both with the steps swapped). Marking the entry non-external here fixes it:
for (const it of kept) {
if (!it.keep) {
const entry = keepOwners.get(it);
if (entry !== undefined) {
entry.external = false;
}
it.keep = true;
}
}There was a problem hiding this comment.
Agreed, and thanks for the repro. It stays inside the in-session retention D88 accepts (a kept flag on a container until reload, in this client only), so I am not re-gating this PR for it; it goes into the next lexical-y follow-up together with the rule-3 copy-end guard, with your box-container case as its test.
| | D23 | **Garbage collection stays on**, as Yjs defaults; headless results are compacted as clients store them. Only temporary documents turn it off. Turning it off for history is an app's choice, on every client and server. | A server that skips compaction while clients compact is a trap, and undo and insert attribution work with GC on. | `DECIDED` | 10-23 | [#9437], [y-server.md][y-server] | | ||
| | D24 | **Yjs's format cleanup is off** in `@lexical/y` containers and on every relay that loads a state (`createYDoc`). A delete keeps the ends of each deleted formatted run (within a bound per format key until D86 removed the bound: a delete keeps every format item it leaves). Not reported upstream. | The cleanup deletes format items a concurrent writer may still rely on. Upstream already knows that skipping it is the app's trade-off. | `DECIDED` | 10-28 | [#9400], [y.md: Servers][y-servers] | | ||
| | D86 | **A delete keeps every format item it leaves (no collapse).** The collapse of the run of deleted content and format items a delete leaves (`Cursor#collapseGap`, D24's bound per format key) is removed; D24's cleanup-off stays. | The collapse dropped ends that a concurrent writer's restore or an undo relied on, and the value ran on: #9575, #9609, a #9620 cell, the concurrent restore cells of #9640, a KeepEndsSessions seed. Every attempt to keep the right ends in the collapse (#9617) needed to know which write relies on which item. Measured against main, same seeds: a three-client view fuzz of 15,000 seeds: 0 seeds wrong where main is right, 631 where main spreads a value and this does not. Cost: each formatted run whose text is deleted keeps its two items, none per keystroke or plain delete (F24A-1: 40, 80, 120 after 20, 40, 60 words typed and deleted at one place); a comment added and its word deleted 3,000 times at one place: 6,002 live format items (main 6); a downstream document library's corpus of 189 documents x1.003 at 10 cycles, x1.26 at 150; Jot x1.01 on its default profile, a concentrated recolor 438 items (main 345). Accepted: the linear growth with deleted formatted text, over the bound and its spreads. | `DECIDED` | 10-31 | [#9575], [#9609] | | ||
| | D88 | **A client keeps its own format items from garbage collection while they are undoable (memory only).** The binding sets Yjs's `Item.keep` on the format items its own transactions and its undo manager's insert, and on the end of each run they delete; it releases them, between transactions, when both undo stacks are empty (after a clear, an undo that leaves nothing, or a write they did not capture), and when it is disposed or its document destroyed. An item another binding or an `UndoManager` step also keeps stays kept. Releasing collects nothing, as Yjs's `UndoManager` collects nothing when it releases what its steps kept: any `UndoManager` on the document may still restore the item. | An undo reads its change's format items, and the end each gave a run it deleted, after a collaborator deleted them (`isKeptByUndo`, `changeEnd`, `closeRestoredRuns`). With Yjs's default garbage collection, which products use, those were collected, the undo could not tell them apart from other content, and the restored value ran on over the rest of the document ([#9623]; a two-client undo fuzz: 221 of 3,000 seeds to 0). Nothing new is written and other clients are unchanged. The cost is local: a format item a collaborator deletes while this client can undo the change that wrote it keeps its deleted content in this client's document, and in a state it encodes, until the document reloads or a relay that loads its state with gc on compacts it: about 25 bytes per such deletion, in this client only (+0.6% to 0.8% of the client's state in a realistic 4,500-edit session). Accepted over a stored change (a `gcFilter` keeping every format item: +1.9% to 7.6% bytes on long notes, in every peer and relay) and over collecting on release, which was tried and dropped: it destroyed what another `UndoManager` on the same document still needed. A `gcFilter` keeping every format item fixed the same cases, but changes what every peer and relay stores (+1.9% to 7.6% bytes on long notes). | `DECIDED` | 10-31 | [#9623], [y.md: Limits][limits] | |
There was a problem hiding this comment.
Nit: this row gives the gcFilter alternative twice ("Accepted over a stored change (a gcFilter keeping every format item ...)" and "A gcFilter keeping every format item fixed the same cases ..."); one of the two sentences can go.
There was a problem hiding this comment.
Right, the second gcFilter sentence is redundant; trimmed in the same follow-up (docs only).
## Description Two markers in `@lexical/y`'s tests named nothing a reader could follow, and one lane failed on a known gc-only case: - `UndoRunEnds.test.ts`: its 14 `test.fails` groups (81 cells) named only #9561, a pull request parked as a draft. Each now names the open issue that tracks it: #9681 (an undo of text typed inside a run deletes that text's run ends, so a collaborator's plain text next to it takes the run's value; D74) or #9682 (an undo of a run's value leaves it after a restore the undo has seen; D64, documented in y.md's Limits, Undo). The file's header says so and that #9561 fixes both. - `regressions/Formats.test.ts`, F30A-1: the italic case fails with gc on (`LEXICAL_Y_GC=1`, which #9679 added), for 3 client id orders, the same on main before #9679. It is now a `test.fails` naming #9683 with gc on only, so the gc-on lane is green. - `regressions/PasteRemadeLine.test.ts` names #9587, which GitHub had closed by mistake (#9598's "Not fixed: #9587" reads as a closing keyword); the repro still fails on main, and the issue is reopened. No change to the test. No behavior change; tests only. ## Test plan ### Before ``` $ LEXICAL_Y_GC=1 pnpm exec vitest run --project unit packages/lexical-y/src/__tests__/unit/regressions/Formats.test.ts -t F30A-1 × italic over "b ", italic removed from "b"; the removal undone ‖ "b" italic again (ids 1,2,3) × italic over "b ", italic removed from "b"; the removal undone ‖ "b" italic again (ids 3,2,1) × italic over "b ", italic removed from "b"; the removal undone ‖ "b" italic again (ids 2,3,1) AssertionError: expected '"a" "b"[I] " cd ef" / "next"' to be '"a" "b "[I] "cd ef" / "next"' Tests 3 failed | 8 passed | 187 skipped (198) ``` `UndoRunEnds.test.ts` with every `test.fails` turned into `test`: 81 failed, 98 passed, on gc off and gc on (each of the 81 is one of the two issues' shapes; the issues carry standalone repros that fail on main). ### After ``` $ pnpm exec vitest run --project unit packages/lexical-y/src/__tests__/unit/UndoRunEnds.test.ts packages/lexical-y/src/__tests__/unit/regressions/Formats.test.ts Tests 292 passed | 85 expected fail (377) $ LEXICAL_Y_GC=1 pnpm exec vitest run --project unit packages/lexical-y/src/__tests__/unit/UndoRunEnds.test.ts packages/lexical-y/src/__tests__/unit/regressions/Formats.test.ts Tests 289 passed | 88 expected fail (377) ``` Whole `@lexical/y` unit project, gc off and on (`LEXICAL_Y_GC=1`): ``` Tests 19022 passed | 295 expected fail | 130 skipped (19447) Tests 19019 passed | 298 expected fail | 130 skipped (19447) ``` One run of four with gc off failed `DisposeMemory.test.ts` (15.7 MB held, bound 8 MB) with 24 workers; it passed the other three and 18 of 18 alone or 12 at a time, and the same full-suite run fails it on main before #9679 (cfcd3c4, 9.4 MB), so it is a heap bound that full-suite load can exceed, unrelated to this change.
## Description Two markers in `@lexical/y`'s tests named nothing a reader could follow, and one lane failed on a known gc-only case: - `UndoRunEnds.test.ts`: its 14 `test.fails` groups (81 cells) named only #9561, a pull request parked as a draft. Each now names the open issue that tracks it: #9681 (an undo of text typed inside a run deletes that text's run ends, so a collaborator's plain text next to it takes the run's value; D74) or #9682 (an undo of a run's value leaves it after a restore the undo has seen; D64, documented in y.md's Limits, Undo). The file's header says so and that #9561 fixes both. - `regressions/Formats.test.ts`, F30A-1: the italic case fails with gc on (`LEXICAL_Y_GC=1`, which #9679 added), for 3 client id orders, the same on main before #9679. It is now a `test.fails` naming #9683 with gc on only, so the gc-on lane is green. - `regressions/PasteRemadeLine.test.ts` names #9587, which GitHub had closed by mistake (#9598's "Not fixed: #9587" reads as a closing keyword); the repro still fails on main, and the issue is reopened. No change to the test. No behavior change; tests only. ## Test plan ### Before ``` $ LEXICAL_Y_GC=1 pnpm exec vitest run --project unit packages/lexical-y/src/__tests__/unit/regressions/Formats.test.ts -t F30A-1 × italic over "b ", italic removed from "b"; the removal undone ‖ "b" italic again (ids 1,2,3) × italic over "b ", italic removed from "b"; the removal undone ‖ "b" italic again (ids 3,2,1) × italic over "b ", italic removed from "b"; the removal undone ‖ "b" italic again (ids 2,3,1) AssertionError: expected '"a" "b"[I] " cd ef" / "next"' to be '"a" "b "[I] "cd ef" / "next"' Tests 3 failed | 8 passed | 187 skipped (198) ``` `UndoRunEnds.test.ts` with every `test.fails` turned into `test`: 81 failed, 98 passed, on gc off and gc on (each of the 81 is one of the two issues' shapes; the issues carry standalone repros that fail on main). ### After ``` $ pnpm exec vitest run --project unit packages/lexical-y/src/__tests__/unit/UndoRunEnds.test.ts packages/lexical-y/src/__tests__/unit/regressions/Formats.test.ts Tests 292 passed | 85 expected fail (377) $ LEXICAL_Y_GC=1 pnpm exec vitest run --project unit packages/lexical-y/src/__tests__/unit/UndoRunEnds.test.ts packages/lexical-y/src/__tests__/unit/regressions/Formats.test.ts Tests 289 passed | 88 expected fail (377) ``` Whole `@lexical/y` unit project (`vitest run --project unit packages/lexical-y/`), gc off and on (`LEXICAL_Y_GC=1`): ``` Tests 1 failed | 18528 passed | 291 expected fail | 130 skipped (18950) Tests 18526 passed | 294 expected fail | 130 skipped (18950) ``` The one failure is `DisposeMemory.test.ts` (heap held above its 8 MB bound) with 24 workers. It passes alone and 12 at a time, and the same full-suite run fails it on main before #9679 (cfcd3c4, 9.4 MB): a heap bound that full-suite load can exceed, unrelated to this change.
…scope D95 ## Description Review follow-ups, tests and docs only: - The `it.fails` / `test.fails` cells name their open issue in the test title (#9689, #9690); the #9671/#9599 rule references stay as context. - D95 says none of the narrower variants tried kept the gains, and lists stored provenance and dropping the restore rule as options under consideration rather than the only ones. - `FormatRuleTradeSessions.test.ts`: the cells pass before the rules with gc off and on (the header said gc on failed there too; with the binding sources from before #9679 and today's harness all ten trade cells pass under both settings). ## Test plan ### Before Titles without the issue numbers. ### After ``` $ LEXICAL_Y_GC=0 vitest --run --project unit packages/lexical-y/src/__tests__/unit/regressions/{FormatRuleTrade,FormatRuleTradeSessions,UndoRestoredRunSharedEnd,UndoRestoredRunOverDeletedText}.test.ts Tests 52 passed | 42 expected fail (94) $ LEXICAL_Y_GC=1 (same files) Tests 52 passed | 42 expected fail (94) ```
…ents; link #9679 in D95 ## Description Review follow-ups, tests and docs only: sessions 107991 and 111874 pass `gc: false` (the setting they passed with before the rules; 110073 already pins `gc: true`), and the header says so; three comments in `FormatRuleTrade.test.ts` now match their ops; D95 links #9679. ## Test plan ### Before The two sessions took gc from `LEXICAL_Y_GC`. ### After ``` $ LEXICAL_Y_GC=0 vitest --run --project unit packages/lexical-y/src/__tests__/unit/regressions/{FormatRuleTrade,FormatRuleTradeSessions,UndoRestoredRunSharedEnd,UndoRestoredRunOverDeletedText}.test.ts Tests 52 passed | 42 expected fail (94) $ LEXICAL_Y_GC=1 (same files) Tests 52 passed | 42 expected fail (94) ```
…does not repeat what it stored ## Description Stacked on #9696. When a local write throws part way, the binding rewrites the document from the editor and reports `@lexical/y/recovered-write`. That rewrite wrote a changed line through the keystroke fast path (`$writeTextEdit`), which writes the text edits Lexical recorded against the previous editor state. Where the failed write had already stored such an edit, the rewrite stored it again: the other client showed the inserted text twice, and the author's editor no longer matched its own document. The rewrite now skips the fast path and diffs each line against the document. The failure is not reachable through a known path; it is measured by injecting a throw (the second item a cursor writes) and affects every tree, including the commit before #9679. ## Test plan ### Before ``` $ vitest --run --project unit packages/lexical-y/src/__tests__/unit/regressions/RecoveredWriteRepeats.test.ts AssertionError: expected 'paragraph("one INSERT INSERT alpha be…' to be 'paragraph("one INSERT alpha beta") / …' // Object.is equality Tests 2 failed (2) ``` ### After ``` $ LEXICAL_Y_GC=0 vitest --run --project unit packages/lexical-y/src/__tests__/unit/regressions/RecoveredWriteRepeats.test.ts Tests 2 passed (2) $ LEXICAL_Y_GC=1 (same file) Tests 2 passed (2) $ LEXICAL_Y_GC=0 vitest --run --project unit packages/lexical-y Test Files 1 failed | 241 passed | 2 skipped (244) Tests 1 failed | 19232 passed | 304 expected fail | 130 skipped (19667) $ LEXICAL_Y_GC=1 vitest --run --project unit packages/lexical-y Test Files 1 failed | 241 passed | 2 skipped (244) Tests 1 failed | 19229 passed | 307 expected fail | 130 skipped (19667) $ pnpm run ci-check exit 0 ``` The one failure in each full run is a memory-loop test (Headless gc off, DisposeMemory gc on) that exceeded its heap bound under load from other runs on the machine; both pass alone.
…does not repeat what it stored ## Description Stacked on #9696. When a local write throws part way, the binding rewrites the document from the editor and reports `@lexical/y/recovered-write`. That rewrite wrote a changed line through the keystroke fast path (`$writeTextEdit`), which writes the text edits Lexical recorded against the previous editor state. Where the failed write had already stored such an edit, the rewrite stored it again: the other client showed the inserted text twice, and the author's editor no longer matched its own document. The rewrite now skips the fast path and diffs each line against the document. The failure is not reachable through a known path; it is measured by injecting a throw (the second item a cursor writes) and affects every tree, including the commit before #9679. ## Test plan ### Before ``` $ vitest --run --project unit packages/lexical-y/src/__tests__/unit/regressions/RecoveredWriteRepeats.test.ts AssertionError: expected 'paragraph("one INSERT INSERT alpha be…' to be 'paragraph("one INSERT alpha beta") / …' // Object.is equality Tests 2 failed (2) ``` ### After ``` $ LEXICAL_Y_GC=0 vitest --run --project unit packages/lexical-y/src/__tests__/unit/regressions/RecoveredWriteRepeats.test.ts Tests 2 passed (2) $ LEXICAL_Y_GC=1 (same file) Tests 2 passed (2) $ LEXICAL_Y_GC=0 vitest --run --project unit packages/lexical-y Test Files 1 failed | 241 passed | 2 skipped (244) Tests 1 failed | 19232 passed | 304 expected fail | 130 skipped (19667) $ LEXICAL_Y_GC=1 vitest --run --project unit packages/lexical-y Test Files 1 failed | 241 passed | 2 skipped (244) Tests 1 failed | 19229 passed | 307 expected fail | 130 skipped (19667) $ pnpm run ci-check exit 0 ``` The one failure in each full run is a memory-loop test (Headless gc off, DisposeMemory gc on) that exceeded its heap bound under load from other runs on the machine; both pass alone.
…9694 (an editor stopped matching its own document in long sessions) Reverts #9679 whole, on top of current `main` (keeps #9684 and #9685). With #9679, a downstream app's long-note sessions end with an editor that no longer matches its own Yjs document (15 of 133 sessions; 0 of 132 on the commit before #9679). Reverting only part of it does not work: the delete rule (D86) alone is worse on a recorded session (2810), and the divergence was traced to one of its hunks (the end a restated run gets at a range's end, which could be linked into another container; #9693 fixes that at the source). This PR is the fallback if #9693 is not taken. Closes #9694 if merged instead of #9693. What this brings back, as before #9679: - A delete collapses the run of deleted content and format items it leaves to a bounded number per format key (D24's bound): the cases it fixed (#9575, #9609, a #9620 cell, #9640's concurrent restore cells) fail again and are pinned with their issues. - No format items kept from garbage collection: with gc on, an undo can again put a value back past its end (#9623). The harness keeps its `LEXICAL_Y_GC` switch (added by #9679), and the cells that fail only with gc on are pinned as `test.fails` with gc on, naming #9623. - The restore rules for #9671 and the restore scan for #9599 (D78) are gone: those cases are open again. `DECISIONS.md` keeps D86 and D88 (now `REVERTED`, with the reason) and reopens D78, rather than rewriting them. `y.md` goes back to the behavior before #9679. The two changesets of #9679 are removed: it was not part of a release, so there is nothing to announce. `UndoSpreadSession2810.test.ts` (added by #9679) is kept: it passes without D86 and D88 as well. `main` 60022ae: the downstream app's long-note sessions diverge in 15 of 133. ``` $ LEXICAL_Y_GC=0 vitest --run --project unit Test Files 1 failed | 648 passed | 4 skipped (653) Tests 1 failed | 28071 passed | 481 expected fail | 135 skipped (28688) $ LEXICAL_Y_GC=1 vitest --run --project unit Test Files 2 failed | 647 passed | 4 skipped (653) Tests 2 failed | 28055 passed | 496 expected fail | 135 skipped (28688) $ pnpm run ci-check exit 0 ``` The failures are load flakes with 40 workers: `DisposeMemory.test.ts` (heap bound; gc off and on) and `MarkdownEscaping.test.ts` (5 s timeout, gc on). Each passes alone (two runs each).
…ome guards, D95 moot, gc-on #9623 cells pinned, D51 back to an accepted limit ## Description Rebased on `main` 064b18e (#9688, #9691). Without #9679's rules: - The #9690 cells (`FormatRuleTrade.test.ts`, `FormatRuleTradeSessions.test.ts`) pass: they become plain tests, guards against bringing those rules back as they were. D95 is marked moot (`REVERTED`). - `UndoRestoredRunOverDeletedText.test.ts`: with gc on, the sequential order of six shapes spreads again (the undo cannot find items gc collected): `it.fails` with gc on, titled "(#9623 with gc on)". The #9689 cells are unchanged. - `UndoInvisibleSteps.test.ts` (#9688): the cell that needed a delete to keep the comment removal's null passes again with the collapse; it becomes a plain test, with its comment updated. - D51 goes back to `ACCEPTED LIMIT` (its status before #9679), keeping the D86 sentence and the revert note. ## Test plan ### Before On 064b18e with the revert alone, `FormatRuleTrade*` and one `UndoInvisibleSteps` cell failed as unexpected passes, and twelve gc-on cells of `UndoRestoredRunOverDeletedText` failed. ### After ``` $ LEXICAL_Y_GC=0 vitest --run --project unit packages/lexical-y/src/__tests__/unit/regressions/{UndoInvisibleSteps,UndoRestoredRunOverDeletedText,FormatRuleTrade,FormatRuleTradeSessions}.test.ts Tests 35 passed | 46 expected fail (81) $ LEXICAL_Y_GC=1 (same files) Tests 35 passed | 46 expected fail (81) ```
…does not repeat what it stored ## Description Stacked on #9696. When a local write throws part way, the binding rewrites the document from the editor and reports `@lexical/y/recovered-write`. That rewrite wrote a changed line through the keystroke fast path (`$writeTextEdit`), which writes the text edits Lexical recorded against the previous editor state. Where the failed write had already stored such an edit, the rewrite stored it again: the other client showed the inserted text twice, and the author's editor no longer matched its own document. The rewrite now skips the fast path and diffs each line against the document. The failure is not reachable through a known path; it is measured by injecting a throw (the second item a cursor writes) and affects every tree, including the commit before #9679. ## Test plan ### Before ``` $ vitest --run --project unit packages/lexical-y/src/__tests__/unit/regressions/RecoveredWriteRepeats.test.ts AssertionError: expected 'paragraph("one INSERT INSERT alpha be…' to be 'paragraph("one INSERT alpha beta") / …' // Object.is equality Tests 2 failed (2) ``` ### After ``` $ LEXICAL_Y_GC=0 vitest --run --project unit packages/lexical-y/src/__tests__/unit/regressions/RecoveredWriteRepeats.test.ts Tests 2 passed (2) $ LEXICAL_Y_GC=1 (same file) Tests 2 passed (2) $ LEXICAL_Y_GC=0 vitest --run --project unit packages/lexical-y Test Files 1 failed | 241 passed | 2 skipped (244) Tests 1 failed | 19232 passed | 304 expected fail | 130 skipped (19667) $ LEXICAL_Y_GC=1 vitest --run --project unit packages/lexical-y Test Files 1 failed | 241 passed | 2 skipped (244) Tests 1 failed | 19229 passed | 307 expected fail | 130 skipped (19667) $ pnpm run ci-check exit 0 ``` The one failure in each full run is a memory-loop test (Headless gc off, DisposeMemory gc on) that exceeded its heap bound under load from other runs on the machine; both pass alone.
Lands together with #9678 (D86, a delete keeps every format item it leaves): this PR is stacked on it and the two were gated as one landing tree, because #9678 alone is worse than main on one gc-on seed that this change fixes. It is retargeted to
mainso both land as one commit; #9678 is closed as landed here.Stacked on #9678 (D86, no collapse): review only the top commit. The two land together, #9678 first (on its own #9678 is worse than main on one gc-on session that this PR fixes; see below).
Description
DECISIONS.md D88,
DECIDED(10-31); D78DECIDED(10-31).With Yjs's default garbage collection, which products use, an undo could
not read the format items of its own change after a collaborator deleted
them: they were collected, so the undo guards (
isKeptByUndo,changeEnd,closeRestoredRuns) took them for other content, and the value an undobrought back ran on over the rest of the document (#9623). The unit harness
built every peer with gc off, so most tests never saw it.
Binding: the binding sets Yjs's ownItem.keepon the format items itstransactions and its undo manager's insert, and on the end of each run
they delete. It releases them, between transactions, when both undo
stacks are empty (after a clear, an undo that leaves nothing, or a write
they did not capture), and when it is disposed or its document destroyed.
Ownership is per listening lifetime and shared across bindings on one
document; an item another binding or an
UndoManagerstep also keepsstays kept. Releasing collects nothing, as the
UndoManagercollectsnothing when it releases what its steps kept. Nothing new is written.
Cursor#runOpener: a restore's scan stops at content of another client,so a client's earlier end and later opener with a collaborator's text
between them are not one restored run (Bug: [lexical-y] Undoing your own formats spreads a format onto the next word after a collaborator typed plain text inside your first run #9599).
Cursorrule 3: a range end that writes its own copy of another client'sopener also ends it where that opener's run ended, so undoing that
client's change leaves the text after the range alone.
Cursor: a format write removes the null and the restore that thisclient's own earlier insert left inside its range, as Yjs's
formatTextdoes (Bug: [lexical-y] Undoing text you typed into a format run and then formatted takes the format off the untouched rest of the run when a collaborator typed after it, with no concurrent edits #9671).
cost), and the [lexical-y] Bug Fix: A delete keeps every format item it leaves (D86) #9678 review follow-ups (the Bug: [lexical-y] A comment spreads onto later words when a client types plain text into it offline and a collaborator's delete drops the null that ended it #9564 entry, the edit cost of
deleted formatted text).
The accepted cost (D88): a format item a collaborator deletes while
this client can still undo the change that wrote it keeps its deleted
content in this client's document, and in states this client encodes,
until the document reloads or a relay that loads its state with gc on
compacts it: about 25 bytes per such deletion, in this client only (100 /
1,000 / 3,000 rounds of one client bolding a word and another clearing it:
2,603 / 25,077 / 75,077 bytes vs 229 / 231 / 231 without this PR; realistic
sessions +0.6% to 0.8%). Collecting on release was tried and dropped: it
destroyed what another
UndoManageron the same document still needed.Tests:
regressions/OwnFormatItems.test.ts: the Bug: [lexical-y] A comment comes back over the rest of the document when a collaborator undoes removing it after its author undid adding it #9623, Bug: [lexical-y] Undoing your own formats spreads a format onto the next word after a collaborator typed plain text inside your first run #9599 and Bug: [lexical-y] Undoing text you typed into a format run and then formatted takes the format off the untouched rest of the run when a collaborator typed after it, with no concurrent edits #9671 reprosand a restated-opener case, gc on and off, every client id order.
regressions/UndoSpreadSession2810.test.ts: a downstream app'sundo-fuzz session where [lexical-y] Bug Fix: A delete keeps every format item it leaves (D86) #9678 alone, with gc on, put a comment on four
paragraphs and main did not.
regressions/OwnFormatItemsRelease.test.ts: the release (clears, an undothat empties both stacks, uncaptured writes, dispose, destroy, a restart,
another binding's and an app
UndoManager's undo after a release, alsofrom inside that
UndoManager's capture), repeated cycles, what a relaystores.
UndoRestoredRunEndExtents(117cells),
UndoRestoredRunEnds(20; the D64 cells stay pinned),UndoSpreadFuzz(5 seeds; its documents use gc on, so it is the gc-ongate),
UndoCommentRemovalSpread(4),RunSequenceUndoseed 87.gcoption, andLEXICAL_Y_GC=1runs every harnesstest with gc on.
Test plan
Landing tree: main cfcd3c4 + #9678 + this PR. Code identical from
3a66cb0 to d1807fb (the last commits change docs and one test). Every
property, fuzz and soak lane ran with gc on and off.
Before (#9678 alone; main where named)
on). Main: 492 / 511.
Run-sequence fuzz, 1,000 seeds: 8. Comment-run fuzz, 500: 0.
OwnFormatItems44 of 72 fail;UndoSpreadSession28102 of 4fail (gc on; main passes);
OwnFormatItemsRelease: n/a.(main 350 / 433).
After
where main or [lexical-y] Bug Fix: A delete keeps every format item it leaves (D86) #9678 is right, gc on and off (identical on every head
since 9817eb6).
comment-run fuzz 0.
pnpm run ci-checkpasses (d1807fb). Units, all packages, gc off andgc on: no failure that [lexical-y] Bug Fix: A delete keeps every format item it leaves (D86) #9678 does not have (gc on: the same F30A-1 cells;
147 pins flip). MATRIX_FULL gc off and on: the same 11 legacy-binding
failures.
OwnFormatItems72 pass,UndoSpreadSession28104 pass,OwnFormatItemsRelease24 pass.tree): about 92,000 sessions across this PR's heads, 0 failing only here;
L1C and L3: 0 only here.
on local builds: identical to main. Corpus growth after 10 / 40 / 150
cycles: format items x1.005 / x1.052 / x1.261, bytes x1.002 / x1.016 /
x1.066 (main = 1).
identical to main. Undo-spread fuzz: plain 0 / 3,000, macro 9 / 2,000
(all 9 also fail on main and [lexical-y] Bug Fix: A delete keeps every format item it leaves (D86) #9678). Growth runs (long notes, 2k to 20k
words, and a concentrated hot spot): no errors, divergence or fresh-read
mismatch; bytes on long notes x0.97 to x1.00 of main, x1.05 to x1.18 at
the concentrated hot spot (the retention above).
lifecycle; the last (3a66cb0, code identical to this head) is clean.