Skip to content

Keep Unicode letters and numbers in slugs, so a Korean heading names its file in Korean instead of untitled - #20

Merged
danhstevens merged 2 commits into
ledgesh:mainfrom
oliverne:fix/unicode-note-filenames
Oct 18, 2026
Merged

danhstevens merged 2 commits into
ledgesh:mainfrom
oliverne:fix/unicode-note-filenames

Conversation

@oliverne

Copy link
Copy Markdown
Contributor

This is a bug fix rather than a feature, so per CONTRIBUTING's "Sending a
change" I sent the PR directly instead of opening an issue first. If you would
rather discuss the design before reviewing, I am happy to move it to an issue.

What changed

A note's filename comes from its first-line H1 through slugify in
src/shared/slug.ts. That function kept only [a-z0-9] and turned everything
else into a separator, so a heading in any non-Latin script slugged to nothing
and the note landed as untitled.md. A Korean heading now names its file
한국어-제목.md, and a Korean workspace display name creates a Korean folder.

Three changes in slugify:

  • The allowed set is \p{L}\p{N} instead of [a-z0-9], so letters and numbers
    in any script survive. Dots, separators, colons and control characters are
    still removed, so the "safe by construction" property is unchanged: the
    emitted slug can never contain a path character or start with a dot.
  • After the existing NFD + combining-mark fold, the result is recomposed with
    normalize("NFC"). Without this, Korean text that arrives decomposed (NFD,
    which is what macOS input methods and the filesystem produce) would keep its
    jamo and be written as a name no other platform matches.
  • Truncation at MAX_SLUG now counts code points rather than UTF-16 units, so a
    long CJK heading is not cut in the middle of a surrogate pair. The comment
    documents the limit as 60 code points, at most 240 UTF-8 bytes, which keeps
    room under the 255-byte filename limit for the enumerated -2 suffix and the
    extension.

createManaged in src/bun/workspaces.ts uses the same slugify, so workspace
folders inherit this with no separate code path. Both the desktop view and the
iOS/Android clients load src/shared/slug.ts, so all three name files the same
way.

Why

A note titled # 한국어 제목 was saved as untitled.md, and every following
Korean-titled note enumerated untitled-2.md, untitled-3.md and so on. The
title was still in the file, but the file on disk and the note browser had lost
the name the user wrote.

How verified

bunx tsc --noEmit    # clean
bun test             # 4504 pass, 0 fail (153 files)

Tests land in this change:

  • src/shared/slug.test.ts: Korean and mixed-language headings, NFD input
    producing the same composed slug, unsafe path characters still removed
    (../../한국어/제목\초안:메모\0 becomes 한국어-제목-초안-메모), and a long
    CJK heading truncated without splitting a letter and staying under 255 bytes.
    The old assertion that 日本語 slugs to null is replaced by one that a pure
    emoji heading (🎉, which is neither \p{L} nor \p{N}) does.
  • src/bun/notes.fs.test.ts: two Korean headings create 한국어-제목.md and
    한국어-제목-2.md without clobbering, and an untitled note can be retitled to
    a Korean heading that moves the file.
  • src/bun/workspaces.fs.test.ts: a Korean display name creates a Korean folder.

No UI behavior changed, so bun run test:e2e does not apply here and there is no
native seam to check live.

Docs rules changed

This is called out per CONTRIBUTING's "Sending a change": two normative pages
stated [a-z0-9-] as the slug invariant, and they now say Unicode letters,
numbers and hyphens.

  • docs/contributor/architecture.md — the trust-boundary bullet ("the view never
    names a file") and the filesystem-invariants subsection on the folder name
    being the only validated name.
  • docs/contributor/remote.md — the note on why .server is dotted, which
    relied on managed workspace slugs never being able to collide with it. The
    reasoning still holds (a slug cannot start with a dot), but the stated range
    was wrong.

docs/contributor/interactions.md and the user manual needed no edit: the
filename rule is not user-facing text there.

oliverne and others added 2 commits October 17, 2026 08:41
…its file

한국어-제목.md instead of falling back to untitled

slugify strips everything outside \p{L}\p{N} rather than [a-z0-9], so a heading
in any non-Latin script survives as a filename instead of collapsing into
separators. Hangul that arrives decomposed (NFD, as macOS input methods produce)
is recomposed to NFC after accent folding, so the file keeps composed syllables
and resolves to one name on every platform. Truncation counts code points
instead of UTF-16 units, so a long CJK heading is not cut mid-letter and stays
inside the 255-byte filename limit; MAX_SLUG is now documented as 60 code
points, which is at most 240 UTF-8 bytes.

The note and workspace names come from the same shared slug.ts that the iOS and
Android clients load, so all three platforms name files the same way.

docs/contributor/architecture.md (the trust boundary and the filesystem
invariants) and remote.md (why .server is dotted) stated "[a-z0-9-]" as the
slug invariant; both now say Unicode letters, numbers and hyphens.
note cannot overwrite a decomposed Korean filename on APFS, keep
combining marks outside Latin script, and cap slugs at 200 UTF-8 bytes
so a save's temp file fits on ext4
@danhstevens

Copy link
Copy Markdown
Collaborator

Thanks for this! The diagnosis and the shape of the change were right. Reviewing it turned up three problems that Unicode filenames introduce, so I pushed a commit on top rather than sending you another round:

  1. Overwrite on macOS. Finder writes Korean names decomposed (NFD), and APFS treats NFD and NFC spellings as the same file. uniqueName compared raw readdir names, so a new # 회의록 note next to a Finder-made 회의록.md replaced it without warning. It now compares names in NFC.
  2. Mark stripping. Removing every combining mark was harmless when only ASCII survived, but now it changed the kept letters: ガイド became カイト and Hindi lost its vowel signs. Accents are now stripped only from Latin letters.
  3. Length. The longest name built from a slug is the save’s temp file (..md.tmp--). With 60 four-byte letters that is over 255 bytes, which ext4 refuses. Slugs are now capped at 200 UTF-8 bytes too.

@danhstevens
danhstevens merged commit 7ce8169 into ledgesh:main Oct 18, 2026
@oliverne

Copy link
Copy Markdown
Contributor Author

Thanks for the thorough review and the additional fixes!

@oliverne
oliverne deleted the fix/unicode-note-filenames branch October 21, 2026 06:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants