Skip to content

feat(rust): @lexical/markdown import, export and transformers - #152

Open
tmm wants to merge 1 commit into
rust-decorator-nodefrom
rust-markdown
Open

tmm wants to merge 1 commit into
rust-decorator-nodefrom
rust-markdown

Conversation

@tmm

@tmm tmm commented Nov 14, 2026

Copy link
Copy Markdown
Member

Description

The Rust port had no @lexical/markdown. A new markdown feature (on
rich-text, list, link and code) ports it file by file from
packages/lexical-markdown/src:

  • MarkdownTransformers.ts: the four transformer types, the built-in
    transformers (HEADING, QUOTE, CODE, the lists, CHECK_LIST, the
    text formats, LINK, TRANSFORMERS), the markdown NodeStates,
    normalizeMarkdown and isTableRowDivider.
  • MarkdownImport.ts, importTextTransformers.ts,
    importTextFormatTransformer.ts, importTextMatchTransformer.ts.
  • MarkdownExport.ts, with selection export.
  • utils.ts, and convert_from_markdown_string,
    convert_to_markdown_string and convert_selection_to_markdown_string.

Upstream's transformers are JS RegExps whose sources are public API,
and the regex crate has no lookaround or backreferences, so the module
has its own small backtracking JS RegExp engine over UTF-16 code units.
That keeps the default build free of third-party crates. Its Unicode
tables (\p{P}, \p{S}, \p{L}, \p{N}, \p{Zs}, and the i flag's
case folding) are generated from Node by
rust/scripts/generate-reg-exp-tables.mjs. PARITY.md lists its two
approximations, which no upstream pattern reaches.

The conformance runner replays markdown/import-export (149 cases). It
uses upstream's five custom transformers, ported as Swift's runner ports
them, and also checks the Markdown field of property/documents.

Not ported: registerMarkdownShortcuts (next), and
$generateNodesFromMarkdownString, which needs ArtificialNode. The
mdast converter is not ported either, as in Swift. The importHtml field
waits for @lexical/html. Export cases read JS's state from
exportState instead of building it from HTML.

Test plan

Before

$ cargo run -p lexical-conformance -- markdown
| `markdown/import-export` | – | – | – | – | – | not implemented |

After

$ cargo run -p lexical-conformance
| `markdown/import-export` | 149 | 149 | 0 | 0 | 0 | pass |
| `property/documents` (JSON, text and Markdown) | 121 | 121 | 0 | 0 | 0 | pass |
| **Total** | 637 | 637 | 0 | 0 | 0 | |

To confirm the checks run, I made each comparison fail on purpose. 147
cases failed on the imported state and 119 on the exported markdown,
which is every import and export case.

cargo fmt --all --check, cargo clippy --workspace --all-targets --all-features --locked -- -D warnings, cargo test --workspace --all-features --locked and RUSTDOCFLAGS='-D warnings' cargo doc --workspace --all-features --no-deps --locked pass. So does
cargo +1.88 check -p lexical-editor --all-features --locked.
cargo tree -p lexical-editor --features markdown lists no dependencies.

## Description

The Rust port had no `@lexical/markdown`. A new `markdown` feature (on
`rich-text`, `list`, `link` and `code`) ports it file by file from
`packages/lexical-markdown/src`:

- `MarkdownTransformers.ts`: the four transformer types, the built-in
  transformers (`HEADING`, `QUOTE`, `CODE`, the lists, `CHECK_LIST`, the
  text formats, `LINK`, `TRANSFORMERS`), the markdown NodeStates,
  `normalizeMarkdown` and `isTableRowDivider`.
- `MarkdownImport.ts`, `importTextTransformers.ts`,
  `importTextFormatTransformer.ts`, `importTextMatchTransformer.ts`.
- `MarkdownExport.ts`, with selection export.
- `utils.ts`, and `convert_from_markdown_string`,
  `convert_to_markdown_string` and `convert_selection_to_markdown_string`.

Upstream's transformers are JS `RegExp`s whose sources are public API,
and the `regex` crate has no lookaround or backreferences, so the module
has its own small backtracking JS `RegExp` engine over UTF-16 code units.
That keeps the default build free of third-party crates. Its Unicode
tables (`\p{P}`, `\p{S}`, `\p{L}`, `\p{N}`, `\p{Zs}`, and the `i` flag's
case folding) are generated from Node by
`rust/scripts/generate-reg-exp-tables.mjs`. PARITY.md lists its two
approximations, which no upstream pattern reaches.

The conformance runner replays `markdown/import-export` (149 cases). It
uses upstream's five custom transformers, ported as Swift's runner ports
them, and also checks the Markdown field of `property/documents`.

Not ported: `registerMarkdownShortcuts` (next), and
`$generateNodesFromMarkdownString`, which needs `ArtificialNode`. The
mdast converter is not ported either, as in Swift. The `importHtml` field
waits for `@lexical/html`. Export cases read JS's state from
`exportState` instead of building it from HTML.

## Test plan

### Before

```
$ cargo run -p lexical-conformance -- markdown
| `markdown/import-export` | – | – | – | – | – | not implemented |
```

### After

```
$ cargo run -p lexical-conformance
| `markdown/import-export` | 149 | 149 | 0 | 0 | 0 | pass |
| `property/documents` (JSON, text and Markdown) | 121 | 121 | 0 | 0 | 0 | pass |
| **Total** | 637 | 637 | 0 | 0 | 0 | |
```

To confirm the checks run, I made each comparison fail on purpose. 147
cases failed on the imported state and 119 on the exported markdown,
which is every import and export case.

`cargo fmt --all --check`, `cargo clippy --workspace --all-targets
--all-features --locked -- -D warnings`, `cargo test --workspace
--all-features --locked` and `RUSTDOCFLAGS='-D warnings' cargo doc
--workspace --all-features --no-deps --locked` pass. So does
`cargo +1.88 check -p lexical-editor --all-features --locked`.
`cargo tree -p lexical-editor --features markdown` lists no dependencies.

Amp-Thread-ID: https://ampcode.com/threads/T-01a1daf1-150e-709b-aaed-996443a64459

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants