Skip to content

fix(docx): render w:sym glyphs from the symbol-font code pages - #211

Open
rahul05ranjan wants to merge 3 commits into
firecrawl:mainfrom
rahul05ranjan:docx-sym-glyph-tables
Open

rahul05ranjan wants to merge 3 commits into
firecrawl:mainfrom
rahul05ranjan:docx-sym-glyph-tables

Conversation

@rahul05ranjan

Copy link
Copy Markdown

Problem

walk_run_content has no arm for w:sym, so a character drawn from a symbol font is dropped entirely (#193): the legacy bullets of old list templates vanish (the runs around them glue into a double space), and Symbol-font operators (p ≤ q) silently corrupt the sentence. The RTF and binary-DOC readers already decode the same characters; only the DOCX run walker has the gap.

One correction to the issue's repro, which the source tables settle: Wingdings F0B7 is the one-o'clock clock glyph (U+1F550), not the round bullet — the round bullet of Word's legacy lists is Symbol F0B7 (→ •). Both fonts' code pages are covered here, so both legacy list styles get their markers back.

Change

  • src/formats/docx/symbols.rs: the code pages of Wingdings, Wingdings 2, Wingdings 3, Webdings and Symbol, mapped to Unicode text. Source: docx4j's SymbolMapper.java (Apache-2.0), which records Alan Wood's symbol-font tables. Every entry was cross-checked against pandoc's independent table (validation only, nothing taken from it): 1003 shared entries are identical; the differences are docx4j's deliberate corrections (Symbol's bracket codes map to real math characters instead of Adobe private-use codepoints, Wingdings F0D8 → U+27A2 ➢, Ohm → U+03A9).
  • glyph() accepts both forms Word and converters write: the code byte and the U+F000..F0FF private-use form, with case-insensitive font names. A value too large to be a code byte loads as the character itself, but only for the five symbol fonts — the issue's converters-that-store-Unicode form (w:font="Symbol" w:char="2264" → ≤) — and private-use and control values never load. Unknown font/code pairs log at debug level instead of vanishing silently; w:sym of a non-symbol font (Arial, …) is dropped exactly as before.
  • content.rs gains the sym arm, emitting the mapped text with the run's style.

This generalizes #177's symbols module — same module, same arm placement, same rejection rules for malformed values. Where #177's four pinned checkbox codepoints differ from the source tables, the tables win on two glyphs (both sources agree Wingdings 0xFE → U+1F5F9 and Wingdings 2 0xA3 → U+2B1C, which still render as a checked and an empty box) and agree on the other two.

Verification

No Rust toolchain on this host, so honestly:

  • The table module is generated deterministically from the Apache-2.0 source; the generator cross-checked all entries against pandoc's table (result above) and asserted code ranges, key uniqueness and no lone surrogates.
  • The glyph() logic was simulated line by line in Python against the same tables, running every scenario of the three new Rust regression tests: all 23 cases match the expected outputs (code-page forms, case-insensitivity, the four operators, the Unicode-valued form, private-use and malformed rejection, non-symbol fonts staying dropped).
  • The Rust tests themselves follow the existing docx_parts style and run under CI's cargo test; they fail on HEAD (beforeafter where the glyph should be) and pass with the table.
  • cargo fmt/clippy not run locally (no toolchain); formatting follows the codebase's style throughout.

Fixes #193

rahul05ranjan and others added 3 commits October 27, 2026 15:53
The run walker dropped w:sym whole, so legacy bullets and
Symbol-font operators vanished from the text (firecrawl#193). Add the
code-page tables of Wingdings, Wingdings 2, Wingdings 3, Webdings
and Symbol and an arm that emits the mapped Unicode text.

Co-authored-by: rahul05ranjan <11793455+rahul05ranjan@users.noreply.github.com>
Co-authored-by: rahul05ranjan <11793455+rahul05ranjan@users.noreply.github.com>
Source: docx4j's SymbolMapper (Apache-2.0), cross-checked entry by
entry against pandoc's independent table.

Co-authored-by: rahul05ranjan <11793455+rahul05ranjan@users.noreply.github.com>

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 3 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

View guided diff | Re-trigger cubic

@rahul05ranjan

Copy link
Copy Markdown
Author

Hi team — this fix for rendering w:sym glyphs from symbol-font code pages is ready for review. The cubic AI review passed, and all tests are green. Let me know if anything needs adjusting!

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

docx: w:sym glyphs beyond checkboxes (Wingdings bullets, Symbol-font operators) are silently dropped

1 participant