feat(rust): a DOM for @lexical/html, with DOMParser and jsdom's inline style declaration

rust/DESIGN.md

@@ -64,7 +64,7 @@ depends on as the upstream package depends on them:
 | `text`      | `@lexical/text`        |                            |
 | `hashtag`   | `@lexical/hashtag`     | `text` (upstream also lists `extension` and `utils`, for `HashtagExtension` and `mergeRegister`, which aren't ported) |
 | `history`   | `@lexical/history`     | (upstream lists `utils`, but uses nothing from it) |
-| `html`      | `@lexical/html`        | `selection`                |
+| `html`      | `@lexical/html`, with `dom` (a DOM in place of the browser's, which the nodes' DOM hooks use) | `selection` |
 | `markdown`  | `@lexical/markdown`    | `rich-text`, `list`, `link`, `code` |
 | `file`      | `@lexical/file`        |                            |
 | `yjs`       | `@lexical/yjs`         | `selection`, `yrs`         |
@@ -139,6 +139,38 @@ Names may change as packages land; this table is updated with them.
   commits it at the next `read` or discrete update, which JS also does when
   one comes first. Update closures that may be queued are `'static`.
 
+### The DOM
+
+`createDOM`, `exportDOM`, `importDOM` and HTML export and import need a
+DOM, which a browser or (in upstream's unit tests) jsdom gives JS. The
+`html` feature brings one, `lexical_editor::dom`, ported from the Swift
+port's (`swift/Sources/Lexical/DOM`, `swift/DESIGN.md`, "The DOM is a small
+model of jsdom's"), since the vectors record what jsdom does:
+
+- **One node type.** `DomNode` is a shared handle (`Rc`) compared by
+  identity, as JS compares node objects, standing for `Node` and every
+  interface Lexical reads through it (`Element`, `Text`, `Comment`,
+  `DocumentFragment`, `Document`); an element's methods answer as empty
+  values on other nodes. A node holds its children; its parent and
+  document are weak links. `get_document()` is one `document` per thread,
+  as JS has one global `document`.
+- **The inline style declaration normalizes as jsdom's does** for what
+  Lexical writes (colors to `rgb()`, trimmed numbers, `0px`, lower-case
+  keywords, `calc()` of a product, CSS-wide keywords), and is read from the
+  `style` attribute lazily and written back as `cssText`, as jsdom's is.
+  The accepted properties and how each normalizes are a table generated
+  from the `css/properties` vectors (`rust/scripts/generate-dom-tables.mjs`,
+  which also writes the entity table from `html/entities`); the runner
+  checks both the tables and the declaration's behavior against them.
+  Unlike jsdom it keeps most values it can't parse, collapsing their
+  whitespace, as Swift's does; it rejects a shadow of nothing but
+  identifiers, as jsdom does.
+- **`DOMParser` is a WHATWG tokenizer and tree builder**, checked against
+  jsdom's by the `html/parse` and `html/entities` vectors. As in Swift it
+  leaves out templates, framesets, the `<select>` and `<ruby>` insertion
+  modes and the case adjustments of SVG and MathML names; their content
+  parses as body content.
+
 ## Conformance
 
 `cargo run -p lexical-conformance` replays every file in
@@ -183,7 +215,7 @@ Each milestone ends with green CI and a report to the Space.
 | P | Desktop playgrounds (`apps/`): `lexical-playground-rune` on Rune, then `lexical-playground-gpui` on GPUI, sharing `lexical-playground-core`; they grow with each milestone below | the vectors, replayed in the app; a session exported from the app replays in JS (`replay.test.ts`); in CI, a GPUI session typed in a real window replays in JS | both apps run on macOS with typing, Return, deletion, formatting, input methods and the Parity panel; they grow with each milestone |
 | M2 | `rich-text` (headings, quotes, and the command handlers a text surface dispatches: typing, Enter, Backspace, formatting) and `history` (undo), which the playgrounds need first; then the node packages `list`, `link`, `code`, `overflow`, `hashtag`, with `selection` and `utils` | `serialization/*`, `property/documents` (JSON and text), `entities/hashtag-*`, `scripts/history` | done (every vector it names passes): `history` is done (`scripts/history` passes); `rich-text`'s nodes and `registerRichText`'s model command handlers are done (`scripts/rich-text` and `scripts/rich-text-commands` pass, with the heading and quote cases of `property/documents`); `plain-text`'s model command handlers are done (`scripts/plain-text` passes); `list`'s nodes, `registerList` and the model part of `registerCheckList` are done (`scripts/list` and `serialization/list` pass, with the list cases of `property/documents`); `link`'s nodes, `$toggleLink` and `registerLink`'s model handlers are done (`scripts/link` and `serialization/link` pass, with the link cases of `property/documents` that need no other package), `registerAutoLink` and its matchers are done behind `autolink` (`scripts/autolink` and `entities/autolink-match` pass); `selection`'s `$setBlocksType` is done (`scripts/set-blocks-type` passes); `code`'s nodes, `FlatStructureUtils.ts`, `CodeExtension`'s transform and `registerCodeIndentation`'s model handlers are done (`scripts/code` and `serialization/code` pass, with the code cases of `property/documents` that need no other package); `overflow`'s `OverflowNode` is done (`scripts/overflow` and `serialization/overflow` pass, with the overflow cases of `property/documents` that need no other package); `hashtag`'s `HashtagNode`, its matcher (generated character tables, no `regex`) and `registerLexicalHashtag`, with `text`'s `registerLexicalTextEntity`, are done (`entities/hashtag-match`, `entities/hashtag-documents` and `entities/hashtag-typing` pass, with the last waiting cases of `property/documents`), so M2's node packages are done |
 | M3 | The commands `property/sequences` dispatches | `property/sequences` | |
-| M4 | `html`: a DOM abstraction over an HTML parser, CSS declarations, export and import | `html/*`, `css/*`, `property/documents` (HTML) | |
+| M4 | `html`: a DOM abstraction over an HTML parser, CSS declarations, export and import | `html/*`, `css/*`, `property/documents` (HTML) | in progress: the DOM (`dom`: nodes, `DOMParser`, the serializer, the inline style declaration and its tables) and `getStyleObjectFromCSS` are done (`html/parse`, `html/entities`, `css/properties` and `css/values` pass); next, the core nodes' DOM hooks and `@lexical/html`'s export and import |
 | M5 | `markdown` | `markdown/import-export`, `property/documents` (Markdown) | |
 | M6 | `yjs`: a port of `@lexical/yjs` on `yrs` (behind the feature), with new JS vectors for the binding (sevki/lexical-rs's `lexical-yjs` and jpcamara/yrby's `lexical-yjs-html` only as cross-checks) | the new collab vectors | |
 | M7 | Releases: the mirror, changesets, a name tmm settles | | |

rust/PARITY.md

@@ -46,6 +46,8 @@ updated with each milestone.
 | `@lexical/extension`'s decorators (the `extension` feature) | `packages/lexical-extension/src/HorizontalRuleExtension.ts` (`HorizontalRuleNode`: a block decorator whose text is `\n`; `$createHorizontalRuleNode`, `$isHorizontalRuleNode`), `DecoratorTextExtension.ts` (`DecoratorTextNode`: inline, `__isInlineFormattable`, `getFormat`, `hasFormat`, `setFormat`, `toggleFormat`, its `format` as a flat NodeState; `$isDecoratorTextNode`), with `LexicalNode`'s generated JSON | `extension` | `scripts/decorator` |
 | `registerLexicalTextEntity` (the `text` feature) | `packages/lexical-text/src/registerLexicalTextEntity.ts`: `EntityMatch`, `registerLexicalTextEntity` (the `TextNode` transform and `$reverseNodeTransform`, `$replaceWithSimpleText`, `$wasTurnedIntoText`, `getFirstMatchAfter`, `isNextTextEntity`, `isTextAfterOutsideSource`, `isEntityAfter`, `$checkNextSibling`, `$moveSelectionIntoEntity`, `$wouldMerge`, `MAX_MATCHES_PASSED`, `MAX_TURNED_INTO_TEXT`, the per-update `markedInUpdate` and `turnedIntoTextInUpdate`); from the core `INTERNAL_getActiveEditorState` | `text`, `updates` | `entities/hashtag-documents`, `entities/hashtag-typing` |
 | `$setBlocksType` (the `selection` feature) | `packages/lexical-selection/src/range-selection.ts` (`$setBlocksType`, with its default `$afterCreateElement`, `$copyBlockFormatIndent`, and `$isPointAtBlockEdge`), through `LexicalNode.replace` and `ListItemNode.replace` | `selection` | `scripts/set-blocks-type`, the block-decorator case of `scripts/decorator` |
+| The DOM (the `html` feature) | what a browser or jsdom gives JS: `Node` (`nodeType`, `nodeName`, `parentNode`, `childNodes`, siblings, `textContent`, `insertBefore`, `appendChild`, `append`, `prepend`, `removeChild`, `replaceChild`, `remove`, `replaceWith`, `replaceChildren`, `contains`, `cloneNode`), `Element` (attributes in order, `id`, `dir`, `className`, `classList`, `style`, `innerHTML`, `outerHTML`, `getElementsByTagName`), `Text`, `Comment`, `DocumentFragment`, `Document` (`createElement` and the other factories, `head`, `body`), `DOMTokenList`, `CSSStyleDeclaration` (`setProperty`, `getPropertyValue`, `getPropertyPriority`, `removeProperty`, `cssText`, `item`) with jsdom's normalization, `DOMParser.parseFromString`, the HTML fragment serialization algorithm; ported from the Swift port's `DOM/` | `dom` | `html/parse`, `html/entities`, `css/properties`, `css/values` |
+| Inline CSS text | `utils/setDOMStyle.ts` (`getStyleObjectFromCSS`, with JS's property order; with the `html` feature `setDOMStyleObject` and `setDOMStyleFromCSS`) | `set_dom_style` | through `css/values`; unit tests against JS's output |
 | UTF-16 offsets | `String#length`, `String#slice` | `utf16` | through the text methods; unit tests against Node |
 | Upstream version | `meta.json` | `UPSTREAM_VERSION` | `meta` |
 

rust/conformance/Cargo.toml

@@ -9,7 +9,7 @@ license.workspace = true
 repository.workspace = true
 
 [dependencies]
-lexical = { workspace = true, features = ["autolink", "code", "extension", "hashtag", "history", "link", "list", "overflow", "plain-text", "rich-text"] }
+lexical = { workspace = true, features = ["autolink", "code", "extension", "hashtag", "history", "html", "link", "list", "overflow", "plain-text", "rich-text"] }
 
 [lints]
 workspace = true

rust/conformance/src/suites/css.rs

@@ -0,0 +1,78 @@
+use crate::case::{Case, Outcome, same};
+use lexical::dom::{CssPropertyTraits, css_property_names, css_property_traits, get_document};
+use lexical::json::Value;
+
+/// What `element.style.setProperty(name, value)` reads back, as
+/// `css.test.ts`'s `probe` asks jsdom.
+fn probe(name: &str, value: &str) -> String {
+    let element = get_document().create_element("div");
+    element.style().set_property(name, value, "");
+    element.style().get_property_value(name)
+}
+
+fn bool_field(case: &Value, key: &str) -> Result<bool, String> {
+    case.field(key)?
+        .as_bool()
+        .ok_or_else(|| format!("`{key}` is not a boolean"))
+}
+
+/// `css/properties`: each property an inline style accepts in jsdom is
+/// accepted, and normalizes hex colors, a bare `0`, an identifier's case and
+/// an unparsed value as jsdom's does; the generated table
+/// (`rust/scripts/generate-dom-tables.mjs`) holds the same properties.
+pub fn check_property(case: &Value, file: &Value) -> Result<Outcome, String> {
+    let name = case.str("name")?;
+    let color = bool_field(case, "color")?;
+    let zero_length = bool_field(case, "zeroLength")?;
+    let preserve_case = bool_field(case, "preserveCase")?;
+    let verbatim = bool_field(case, "verbatim")?;
+    let bits = u8::from(color)
+        | u8::from(zero_length) << 1
+        | u8::from(preserve_case) << 2
+        | u8::from(verbatim) << 3;
+    same(
+        "table traits",
+        css_property_traits(name),
+        Some(CssPropertyTraits(bits)),
+    )?;
+    same(
+        "properties in the table",
+        css_property_names().count(),
+        file.arr("cases")?.len(),
+    )?;
+    same(
+        "accepts `initial`",
+        probe(name, "initial").is_empty(),
+        false,
+    )?;
+    same(
+        "color",
+        probe(name, "#f00") == "rgb(255, 0, 0)"
+            || probe(name, "1px solid #f00") == "1px solid rgb(255, 0, 0)",
+        color,
+    )?;
+    same("preserveCase", probe(name, "Foo") == "Foo", preserve_case)?;
+    same(
+        "verbatim",
+        probe(name, "1PX 1px RED") == "1PX 1px RED",
+        verbatim,
+    )?;
+    same("zeroLength", probe(name, "0") == "0px", zero_length)?;
+    Ok(Outcome::Pass)
+}
+
+/// `css/values`: inline CSS text serializes back as the `cssText` jsdom
+/// gives, in the declaration and in the `style` attribute.
+pub fn check_value(case: &Value, _file: &Value) -> Result<Outcome, String> {
+    let css = case.str("css")?;
+    let css_text = case.str("cssText")?;
+    let element = get_document().create_element("span");
+    element.style().set_css_text(css);
+    same("cssText", element.style().css_text().as_str(), css_text)?;
+    same(
+        "style attribute",
+        element.get_attribute("style").as_deref(),
+        Some(css_text),
+    )?;
+    Ok(Outcome::Pass)
+}

rust/conformance/src/suites/html.rs

@@ -0,0 +1,48 @@
+use crate::case::{Case, Outcome, same};
+use lexical::dom::{DomNode, is_legacy_entity_name, named_entity, parse_from_string};
+use lexical::json::Value;
+
+/// `html/parse`: `DOMParser` builds the same document as jsdom's, compared
+/// by the `innerHTML` of its head and body.
+pub fn check_parse(case: &Value, _file: &Value) -> Result<Outcome, String> {
+    let document = parse_from_string(case.str("html")?);
+    let inner = |e: Option<DomNode>| e.map(|e| e.inner_html());
+    same(
+        "head",
+        inner(document.head()).as_deref(),
+        Some(case.str("head")?),
+    )?;
+    same(
+        "body",
+        inner(document.body()).as_deref(),
+        Some(case.str("body")?),
+    )?;
+    Ok(Outcome::Pass)
+}
+
+/// `html/entities`: each named character reference decodes to the text
+/// jsdom gives, with and (for a legacy name) without its semicolon, and the
+/// generated table (`rust/scripts/generate-dom-tables.mjs`) agrees.
+pub fn check_entity(case: &Value, file: &Value) -> Result<Outcome, String> {
+    let name = case.str("name")?;
+    let value = case.str("value")?;
+    let legacy = case.field("legacy")?.as_bool().ok_or("legacy")?;
+    same("table value", named_entity(name), Some(value))?;
+    same("table legacy", is_legacy_entity_name(name), legacy)?;
+    let count = file.arr("cases")?.len();
+    let table = lexical::dom::named_entity_count();
+    same("names in the table", table, count)?;
+    // Without its semicolon, a name that isn't legacy may still start with
+    // one that is (`&centerdot` is `&cent` + "erdot").
+    let (html, expected) = if legacy {
+        (
+            format!("<p>&{name};|&{name}</p>"),
+            format!("{value}|{value}"),
+        )
+    } else {
+        (format!("<p>&{name};</p>"), value.to_owned())
+    };
+    let body = parse_from_string(&html).body().ok_or("no body")?;
+    same("textContent", body.text_content(), expected)?;
+    Ok(Outcome::Pass)
+}

rust/conformance/src/suites/mod.rs

@@ -1,7 +1,9 @@
 //! One suite per vector file the port replays. A file with no suite is
 //! reported as not implemented.
 
+mod css;
 mod entities;
+mod html;
 mod meta;
 mod property;
 mod serialization;
@@ -15,6 +17,26 @@ pub fn all() -> Vec<Suite> {
             variant: None,
             check: meta::check,
         },
+        Suite {
+            file: "css/properties",
+            variant: None,
+            check: css::check_property,
+        },
+        Suite {
+            file: "css/values",
+            variant: None,
+            check: css::check_value,
+        },
+        Suite {
+            file: "html/entities",
+            variant: None,
+            check: html::check_entity,
+        },
+        Suite {
+            file: "html/parse",
+            variant: None,
+            check: html::check_parse,
+        },
         Suite {
             file: "entities/autolink-match",
             variant: None,

rust/lexical-editor/Cargo.toml

@@ -58,6 +58,10 @@ hashtag = ["text"]
 # DecoratorTextNode (an inline one). The rest of the package, the extension
 # API, isn't ported.
 extension = []
+# @lexical/html: a DOM in place of the browser's (`dom`: nodes, an HTML
+# parser and serializer, and an inline style declaration that normalizes as
+# jsdom's does), with the nodes' DOM hooks and HTML export and import.
+html = ["selection"]
 
 [dependencies]
 # Without `perf-literal`, which would add aho-corasick and memchr.