Skip to content

Rebuild on a real AST parser with pluggable dialect catalogues - #1

Merged
jasdeepkhalsa merged 1 commit into
masterfrom
claude/modernize-js-ts-parser-19wn4y
Aug 7, 2026
Merged

jasdeepkhalsa merged 1 commit into
masterfrom
claude/modernize-js-ts-parser-19wn4y

Conversation

@jasdeepkhalsa

@jasdeepkhalsa jasdeepkhalsa commented Aug 6, 2026 •

Copy link
Copy Markdown
Owner

Rebuilds jstranslator around @babel/parser and small per-dialect translation files, on a Vite 8 / Vitest 4 / TypeScript stack.

Why

The 2013 version ran a list of String.replace calls over the raw source text. It had no idea what any of it meant, so it rewrote characters wherever it found them — inside strings, inside comments, inside variable names — and the output was still basically JavaScript with words swapped:

variable true or false (boolean) #equals# true;
if the following condition is true:  (true or false (boolean) #is absolutely equal to# true) {

It also only ran in a browser via a synchronous XMLHttpRequest for a hardcoded sample.js, and called eval() on its own config keys. The repo's own To Do list already named the fix: switch to a real parser, cover modern syntax, and support pluggable translations.

Same input, now:

Make an old-style variable `boolean`, starting as true.
If `boolean` is exactly true:
  Log true in the console.

How it works

Three stages, each testable on its own:

source ──parse──► syntax tree ──describe──► phrase tree ──render──► text
       @babel/parser            no language yet        catalogue applied here

The middle stage is the point. describe() emits a language-neutral tree of { id, values, children } phrases and never any prose — so all wording lives in catalogue files, and adding a dialect is a data change, never a code change. It is also useful on its own: --json gives you machine-readable structure.

Coverage now spans the modern language — classes, async/await, generators, destructuring, optional chaining, modules, JSX — plus TypeScript's type syntax. Well-known APIs get dedicated messages, which is where most of the accuracy comes from: console.log(x) is "log x in the console", not "ask console to log with x". Anything the walker doesn't recognise quotes its source rather than throwing.

The dialects

Locale Language Messages Inherits
en-GB British English (simplified) 230 —
pa ਪੰਜਾਬੀ — Punjabi, Gurmukhi 230 en-GB
pa-Latn Punjabi, romanised 222 pa
sco Scots 73 en-GB
pcm Naijá — Nigerian Pidgin 70 en-GB
en-IE Irish English 53 en-GB
en-x-pirate Pirate 48 en-GB

console.log(list[i]), seven ways:

Locale Output
en-GB Log item i of list in the console.
sco Shout thing i oot o list intae the console.
en-IE Stick item i of list up in the console.
pcm Put number i for inside list for console.
en-x-pirate Cry item i of list from the crow's nest.
pa `list` ਦੀ `i` ਵਾਲੀ ਚੀਜ਼ ਨੂੰ ਕੰਸੋਲ ਵਿੱਚ ਲਿਖੋ।

Only en-GB has to be complete; the rest set "extends" and override just what they want to say differently. Messages are ICU MessageFormat, so plurals and optional clauses follow each language's rules instead of being glued together with string concatenation.

One subtlety worth flagging: each catalogue names its own list conjunction rather than relying on Intl.ListFormat. Runtimes ship varying ICU subsets, and one without list patterns for pa silently returns English "and" — very visible mid-Punjabi-sentence. A test enforces it.

On messagevisor

Evaluated properly — installed @messagevisor/cli + sdk, scaffolded a project, ran a build. It is a good system, and the README now says so.

It wasn't adopted because of the authoring shape: it stores one file per message key with every locale inside, which suits translators working key by key. Here the unit of work is a dialect — someone sits down and writes Scots — so adding one would be a diff across ~230 files instead of adding one. The messages are ordinary ICU strings and the inheritance model is the same idea, so porting later is mechanical if this ever grows a translation team.

What else is in here

  • CLI — jstranslator file.ts --locale sco, --json, --locales, stdin.
  • Web playground — replaces esprima.html; paste code, pick a dialect, translates as you type. Verified in a real browser.
  • CI — typecheck, tests, both builds, across Node 20.19 / 22 / 24.
  • Docs — rewritten README, plus docs/architecture.md, docs/writing-a-dialect.md, and a generated docs/message-keys.md (npm run docs:keys).
  • npm publish config — exports, files, bin, prepublishOnly. Nothing has been published.

jstranslator.js, esprima.html and the stub index.html are removed; sample.js moves to examples/ and is still translated in the test snapshots.

Testing

80 tests pass; typecheck and both builds are clean. Small tests live in-source next to what they cover (if (import.meta.vitest)) and are stripped from the build; broader ones are in test/.

Two tests keep the catalogues honest in both directions: every message the walker can emit must exist in en-GB, and every message in every dialect must be a key the walker actually emits. So a new phrase can't ship without wording, and a typo in a dialect override fails the build rather than silently doing nothing. A kitchen-sink file containing every supported construct is run through all seven locales with no unrendered placeholders.

One thing to check

Licence. jstranslator.js said "Licensed under the GPL" with no version, and there was no LICENSE file. You asked for MIT or Apache-2.0, so this adds an MIT LICENSE and sets "license": "MIT". Worth a look before merging, since it's a relicence of your own earlier work.

The non-English dialects are hand-written and would benefit from a native speaker's eye — corrections are one JSON file each.


Generated by Claude Code

The 2013 implementation ran a list of regexes over the raw source text, so it
rewrote the characters `if`, `in`, `=` and `boolean` wherever they appeared —
including inside strings and comments — and emitted JavaScript with words swapped
in rather than sentences. It also only ran in a browser, via a synchronous
XMLHttpRequest for a hardcoded sample, and called eval() on its own config keys.

Replace it with three stages, each testable on its own:

  source -> @babel/parser -> phrase tree -> text

The middle stage is the point: describe() emits a language-neutral tree of
{ id, values, children } phrases and never any prose. All wording lives in
catalogue files, so adding a dialect is a data change and never a code change.

Coverage now spans the modern language — classes, async/await, generators,
destructuring, optional chaining, modules, JSX — and TypeScript's type syntax.
Well-known APIs get dedicated messages, so console.log(x) reads as "log `x` in
the console" rather than a generic call description. Unrecognised nodes quote
their source instead of throwing.

Seven catalogues: en-GB is the complete base; pa (Gurmukhi) is a full
translation; pa-Latn, sco, pcm, en-IE and en-x-pirate set "extends" and override
only what they want to say differently, so Pirate is 48 messages, not 230.
Messages are ICU, so plurals and optional clauses follow each language's rules.
Each catalogue names its own list conjunction rather than trusting Intl.ListFormat,
which falls back to English on runtimes with trimmed ICU data.

Stack is Vite 8 + Vitest 4 + TypeScript, with in-source tests enabled so small
tests sit next to the code they cover. Two tests keep the catalogues honest in
both directions: every message the walker can emit must exist in en-GB, and every
message in every dialect must be a key the walker actually emits.

Also adds a CLI, a web playground, CI across Node 20/22/24, generated key
reference docs, and an MIT LICENSE (the source header said "GPL" with no version
and there was no licence file).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016Se5ZqRETMYysztpcjM7CX
@jasdeepkhalsa
jasdeepkhalsa merged commit cd5c227 into master Aug 7, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants