Skip to content

feat: let the reader use more than one dictionary - #31

Merged
widgetii merged 1 commit into
mainfrom
feat/m6-dictionary-catalogue
Oct 14, 2026
Merged

widgetii merged 1 commit into
mainfrom
feat/m6-dictionary-catalogue

Conversation

@widgetii

Copy link
Copy Markdown
Member

Until now the app opened exactly one file, the bundled CC-CEDICT, so an imported BKRS container was a 493 MB file nothing read. This is the catalogue, the merge, and the three bugs that only appeared once the app actually had two dictionaries open.

Found by scanning, not by remembering where things were

Every .hanreaderdict in the dictionaries directory is installed, plus the bundled one. No stored path, so no path to go stale — which on iOS it would, since the container directory's UUID changes on every install and restore. The predecessor stored absolute audio paths and broke exactly this way.

A dictionary also describes itself: name, languages, licence and attribution are read from its own meta each time it is listed, never copied into a registry, so the acknowledgements cannot show a licence the file contradicts. The reader's own choices — order and permissions — live in the library database, because that is the half that cannot be regenerated and the half that belongs in a backup.

A file that is present but will not open is listed, not hidden. Showing a silently shorter list is how "my Russian definitions disappeared" becomes unanswerable.

Grouped by reading, never deduplicated, always attributed

了 is le in one dictionary and liǎo in another; without grouping a reader sees four entries with no indication which are the same word said the same way. Two dictionaries defining one word are two answers, not one with a duplicate — merging them would mean choosing whose wording survives, which is a licensing problem as much as a usability one. Each answer carries which dictionary it came from, because CC BY-SA attribution has to reach the reader where they read the gloss.

Three permissions, not one switch

A reader can turn off CC-CEDICT's English definitions and still need it: 77% of BKRS entries carry no reading, so its single-character headwords are what put pinyin on the page in both language modes. One "enabled" switch would let them turn off the thing their ruby depends on.

The third permission is the segmentation word list — opt-in, and off by default for a dictionary whose headwords are not words. BKRS contributes 1,806,893 lexemes: a quarter of a gigabyte held for the life of the process, and a word list in which almost every run of Han characters is a word, which is the condition under which maximum-matching produces confident nonsense.

A dictionary nobody has ordered sorts by its gloss language against the reader's own, so a Russian-locale reader sees БКРС first without touching a setting, and an English-locale reader importing the same file does not.

Three bugs that only running it could find

A finished container could not be opened at all. It was left in write-ahead logging mode, and a database whose header says WAL needs a shared-memory file that a read-only connection cannot create. An imported dictionary failed with unable to open database file and simply did not appear — while every writer-side test passed, because a writer opens for writing. Journal mode is an import-time choice for crash safety; a finished dictionary is read-only data.

Adding BKRS changed 打's reading from dǎ to dá on screen. The composer ranks a reading partly by how many senses carry it — sound within one dictionary, meaningless across two, and especially for BKRS, where one entry per reading shares a single sense run by design, so all of 打's readings have 57 senses and the tie fell to alphabetical order. A dictionary with more senses per entry would simply outvote a better answer from the dictionary the reader put first. Readings now come from the first reading-providing dictionary that has the word.

许谎 displayed as xǔ; hǔang. Measured over the 784,004 cards that have a reading: 1,750 lines separate readings with ;, and 287 carry a register note between two readings — zhǔ; [i]в сочет. также[/i] zhù — of which some have no separator at all (nāi [i]гуанчжоуск. диал. также[/i] lāi), so splitting could never find the second reading however many separators it knew.

A reading is now a maximal run of Latin-alphabet words. That takes nāi … lāi as two, leaves the Russian between them out, and needs no list of this dictionary's abbreviations:

readings recovered from the 1,825 awkward lines
split on , only the note becomes a reading
split on , and ;, markup stripped 3,810, and 292 lines still show Russian
maximal Latin runs 4,114, none of them prose

The whole set goes from 3,442,623 entries to 3,445,226.

Also: a sense with nothing in it is no longer chosen as an entry's one-line summary. A BKRS card opens with a division heading whose whole text is a Roman numeral and a part of speech, both lifted out of the gloss — so 爱's summary rendered as a blank line with 32 senses behind it.

Verified against the real thing

The 496 MB container installed where the app looks for it:

installed (preferring en):
  0  CC-CEDICT [cc-cedict] en  glosses=true readings=true lexicon=true   107,619 entries
  2  大БКРС - 250920 [dabkrs-250920] ru  glosses=true readings=true lexicon=false  3,445,226 entries

爱: 1 group [ài]        cc-cedict 4 senses + dabkrs 32
了: 3 groups [le] [liǎo] [liào]   — liào is BKRS's, CC-CEDICT has no such reading
一: 1 group [yī]        cc-cedict 10 + dabkrs 57
打: 2 groups [dá] [dǎ]  cc-cedict 18 + dabkrs 114

Both reading defects were found by looking at the running app and are fixed there — 遭打 reads zāo dǎ again and 许谎 reads xǔhuǎng.

Not yet verified visually: the grouped, attributed detail panel itself. Driving a click needs accessibility permission this session does not have, so the rendering is covered by the data path and by tests rather than by a screenshot. That is what the fidelity XCUITests in M10 are for.

28 new tests. Full suite green, lint clean, no warnings under -warnings-as-errors.

🤖 Generated with Claude Code

Until now the app opened exactly one file, the bundled CC-CEDICT, so an
imported BKRS container was a 493 MB file nothing read. This is the catalogue,
the merge, and the three bugs that only appeared once the app actually had two
dictionaries open.

**Found by scanning, not by remembering where things were.** Every
`.hanreaderdict` in the dictionaries directory is installed, plus the bundled
one. No stored path, so no path to go stale — which on iOS it would, since the
container directory's UUID changes on every install and restore. And a
dictionary describes itself: name, languages, licence and attribution come
from its own `meta` each time it is listed, so the acknowledgements cannot
show a licence the file contradicts. The reader's own choices — order and
permissions — live in the *library* database, because that is the half that
cannot be regenerated and the half that belongs in a backup.

**Answers are grouped by reading and never deduplicated.** 了 is `le` in one
dictionary and `liǎo` in another; without grouping a reader sees four entries
with no indication which are the same word said the same way. Two dictionaries
defining one word are two answers, not one with a duplicate: merging them
would mean choosing whose wording survives, which is a licensing problem as
much as a usability one. Each answer is labelled with where it came from,
because CC BY-SA attribution has to reach the reader where they read the
gloss.

**Three permissions, not one switch.** A reader can turn off CC-CEDICT's
English definitions and still need it: 77% of BKRS entries carry no reading,
so its single-character headwords are what put pinyin on the page in both
language modes. One "enabled" switch would let them turn off the thing their
ruby depends on. The third permission is the segmentation word list, opt-in
and off by default for a dictionary whose headwords are not words — BKRS
contributes 1,806,893 lexemes, which is a quarter of a gigabyte held for the
life of the process and a word list in which almost every run of Han
characters is a word. A dictionary nobody has ordered sorts by its gloss
language against the reader's own, so a Russian-locale reader sees БКРС first
without touching a setting.

### Three bugs that only running it could find

**A finished container could not be opened.** It was left in write-ahead
logging mode, and a database whose header says WAL needs a shared-memory file
that a *read-only* connection cannot create. So an imported dictionary failed
with "unable to open database file" and simply did not appear in the reader's
list, while every writer-side test passed — because a writer opens for
writing. Journal mode is an import-time choice for crash safety; a finished
dictionary is read-only data.

**Adding BKRS changed 打's reading from `dǎ` to `dá` on screen.** The composer
ranks a reading partly by how many senses carry it, which is sound within one
dictionary and meaningless across two — and especially for BKRS, where one
entry per reading shares a single sense run by design, so all of 打's readings
have 57 senses and the tie fell to alphabetical order. A dictionary with more
senses per entry would simply outvote a better answer from the dictionary the
reader put first. Readings now come from the first reading-providing
dictionary that has the word, so the reader's ordering decides and each
dictionary's ranking applies to its own entries.

**`许谎` displayed as `xǔ; hǔang`.** 1,750 reading lines separate their
readings with a semicolon rather than a comma, and 287 carry a register note
between two readings — `zhǔ; [i]в сочет. также[/i] zhù` — of which some have
no separator at all. A reading is now a maximal run of Latin-alphabet words,
which takes `nāi … lāi` as two readings, leaves the Russian between them out,
and needs no list of this dictionary's abbreviations. It recovers 4,114
readings from those 1,825 awkward lines where splitting on separators alone
recovers 3,810 and shows `в сочет. также zhù` above a word for the rest. The
whole set goes from 3,442,623 entries to 3,445,226.

Also: a sense with nothing in it is no longer chosen as an entry's one-line
summary. A BKRS card opens with a division heading whose whole text is a Roman
numeral and a part of speech, both lifted out of the gloss, so 爱's summary
rendered as a blank line with 32 senses behind it.

Verified against the real 496 MB container installed where the app looks for
it: both dictionaries open and merge, 爱 gets CC-CEDICT's 4 senses alongside
BKRS's 32, 了 gains the third reading `liào` that CC-CEDICT does not have, and
一 and 打 get 57 each. The two reading defects were found by looking at the
running app and are fixed there.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@widgetii
widgetii enabled auto-merge (squash) October 14, 2026 09:05
@widgetii
widgetii merged commit 3eb211b into main Oct 14, 2026
8 checks passed
@widgetii
widgetii deleted the feat/m6-dictionary-catalogue branch October 14, 2026 09:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant