Repository navigation
Rebuild on a real AST parser with pluggable dialect catalogues - #1
Merged
Merged
Conversation
The 2013 implementation ran a list of regexes over the raw source text, so it
rewrote the characters `if`, `in`, `=` and `boolean` wherever they appeared —
including inside strings and comments — and emitted JavaScript with words swapped
in rather than sentences. It also only ran in a browser, via a synchronous
XMLHttpRequest for a hardcoded sample, and called eval() on its own config keys.
Replace it with three stages, each testable on its own:
source -> @babel/parser -> phrase tree -> text
The middle stage is the point: describe() emits a language-neutral tree of
{ id, values, children } phrases and never any prose. All wording lives in
catalogue files, so adding a dialect is a data change and never a code change.
Coverage now spans the modern language — classes, async/await, generators,
destructuring, optional chaining, modules, JSX — and TypeScript's type syntax.
Well-known APIs get dedicated messages, so console.log(x) reads as "log `x` in
the console" rather than a generic call description. Unrecognised nodes quote
their source instead of throwing.
Seven catalogues: en-GB is the complete base; pa (Gurmukhi) is a full
translation; pa-Latn, sco, pcm, en-IE and en-x-pirate set "extends" and override
only what they want to say differently, so Pirate is 48 messages, not 230.
Messages are ICU, so plurals and optional clauses follow each language's rules.
Each catalogue names its own list conjunction rather than trusting Intl.ListFormat,
which falls back to English on runtimes with trimmed ICU data.
Stack is Vite 8 + Vitest 4 + TypeScript, with in-source tests enabled so small
tests sit next to the code they cover. Two tests keep the catalogues honest in
both directions: every message the walker can emit must exist in en-GB, and every
message in every dialect must be a key the walker actually emits.
Also adds a CLI, a web playground, CI across Node 20/22/24, generated key
reference docs, and an MIT LICENSE (the source header said "GPL" with no version
and there was no licence file).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016Se5ZqRETMYysztpcjM7CX
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rebuilds jstranslator around
@babel/parserand small per-dialect translation files, on a Vite 8 / Vitest 4 / TypeScript stack.Why
The 2013 version ran a list of
String.replacecalls over the raw source text. It had no idea what any of it meant, so it rewrote characters wherever it found them — inside strings, inside comments, inside variable names — and the output was still basically JavaScript with words swapped:It also only ran in a browser via a synchronous
XMLHttpRequestfor a hardcodedsample.js, and calledeval()on its own config keys. The repo's own To Do list already named the fix: switch to a real parser, cover modern syntax, and support pluggable translations.Same input, now:
How it works
Three stages, each testable on its own:
The middle stage is the point.
describe()emits a language-neutral tree of{ id, values, children }phrases and never any prose — so all wording lives in catalogue files, and adding a dialect is a data change, never a code change. It is also useful on its own:--jsongives you machine-readable structure.Coverage now spans the modern language — classes,
async/await, generators, destructuring, optional chaining, modules, JSX — plus TypeScript's type syntax. Well-known APIs get dedicated messages, which is where most of the accuracy comes from:console.log(x)is "logxin the console", not "askconsoletologwithx". Anything the walker doesn't recognise quotes its source rather than throwing.The dialects
en-GBpaen-GBpa-Latnpascoen-GBpcmen-GBen-IEen-GBen-x-pirateen-GBconsole.log(list[i]), seven ways:en-GBioflistin the console.scoioot olistintae the console.en-IEioflistup in the console.pcmifor insidelistfor console.en-x-pirateioflistfrom the crow's nest.paOnly
en-GBhas to be complete; the rest set"extends"and override just what they want to say differently. Messages are ICU MessageFormat, so plurals and optional clauses follow each language's rules instead of being glued together with string concatenation.One subtlety worth flagging: each catalogue names its own list conjunction rather than relying on
Intl.ListFormat. Runtimes ship varying ICU subsets, and one without list patterns forpasilently returns English "and" — very visible mid-Punjabi-sentence. A test enforces it.On messagevisor
Evaluated properly — installed
@messagevisor/cli+sdk, scaffolded a project, ran a build. It is a good system, and the README now says so.It wasn't adopted because of the authoring shape: it stores one file per message key with every locale inside, which suits translators working key by key. Here the unit of work is a dialect — someone sits down and writes Scots — so adding one would be a diff across ~230 files instead of adding one. The messages are ordinary ICU strings and the inheritance model is the same idea, so porting later is mechanical if this ever grows a translation team.
What else is in here
jstranslator file.ts --locale sco,--json,--locales, stdin.esprima.html; paste code, pick a dialect, translates as you type. Verified in a real browser.docs/architecture.md,docs/writing-a-dialect.md, and a generateddocs/message-keys.md(npm run docs:keys).exports,files,bin,prepublishOnly. Nothing has been published.jstranslator.js,esprima.htmland the stubindex.htmlare removed;sample.jsmoves toexamples/and is still translated in the test snapshots.Testing
80 tests pass; typecheck and both builds are clean. Small tests live in-source next to what they cover (
if (import.meta.vitest)) and are stripped from the build; broader ones are intest/.Two tests keep the catalogues honest in both directions: every message the walker can emit must exist in
en-GB, and every message in every dialect must be a key the walker actually emits. So a new phrase can't ship without wording, and a typo in a dialect override fails the build rather than silently doing nothing. A kitchen-sink file containing every supported construct is run through all seven locales with no unrendered placeholders.One thing to check
Licence.
jstranslator.jssaid "Licensed under the GPL" with no version, and there was noLICENSEfile. You asked for MIT or Apache-2.0, so this adds an MITLICENSEand sets"license": "MIT". Worth a look before merging, since it's a relicence of your own earlier work.The non-English dialects are hand-written and would benefit from a native speaker's eye — corrections are one JSON file each.
Generated by Claude Code