Skip to content

Commit ae268ae

Browse files
authored
fix(i18n): close the bracket the scan opens (#3077) (#3078)
Fixes #3077 Owner: Dev-3 Reviewer: Birfy Written-by: Dev-3 Bot: FlyBot Dev-3 Claim-Id: 171de2f559ab43f2aa9b4d358a4fc085 Base: f210567 — the live `main` this branch was cut from (`#3076`'s merge commit), read with `git ls-remote up HEAD` right before the cut and again before the push; no rebase Pushed head: cca7f38 Tree: 1bd6027 Tier: T3 — `scripts/i18n-usage.mjs` and its tests. No gate file is touched: `policy/gates.json` lists `.github/scripts/{merge-gate,review-gate,pass-label,risk-gate,policy-sign,enqueue-guard,enqueue-rules,gate-check}.mjs`, `scripts/self-test-catalog.mjs`, the four workflows and `policy/` — this PR edits none of them, so `policy-sign` stays green and `test` runs for real. Ran: red-first twice on `f2105675` with the three new tests and `scripts/i18n-usage.mjs` at its base blob; at the tip: the five i18n suites 39/39, the three new tests 3/3, `i18n-usage --missing` 167, bare draws 20, template draws 43, `i18n-usage --missing --write … --force` no change, `build-client --check` exit 0, `ds-bot-patches check` 3 lines + `canonical` no-op, `changelog.mjs check` 7 fragments, `arch-docs: OK (0 changed file(s))` Not run: `oxlint` (no oxlint binary in this checkout or in the harness dir, with `DSH_HARNESS` set) and `npm run verify` as one command (no `node_modules` in this worktree beyond a linked `core/node_modules`); every step was run individually, listed below. No client bundle is touched, so no rebuild is owed. ## What changed `tokens()` in `scripts/i18n-usage.mjs` emitted a punct token for `(` and `[` but not for `]` (`:195` only did `calls.pop()` and stepped over it). Every depth that started at a bracket therefore never returned to zero, and the walk stopped splitting `? :` and `||` at depth 0 — so a returned literal beside an indexed read (`return list[0] || 'None yet'`) was swallowed together with the rest of the file. `missingCopy()` on that snippet returned `[]`; without `[0]` the same literal is reported. - **The tokenizer closes what it opens.** `]` now emits `{ type: 'punct', value: ']' }` the way `)` and `[` do, before the `calls.pop()`, so the depth bookkeeping (`:325-332`, `:436-440`, `:461-470`, `:540-550`) sees the token it was written against. - **That alone flips an existing pin off by accident.** With the bracket closed, the run that used to overrun to end-of-file no longer does, and one already-pinned sentence stops being found by overrun: `plugins/social/client.js`'s `FriendNote` keeps its words in a local interpolated into a template (`` `${who} stays a friend: still shared through ${linked}.` ``). The pin for the sibling branch was an artefact of the overrun, not of a read. - **So the walk now records the names a `${}` hole interpolates.** A template token carries `holes` (the identifiers inside its `${}` code), and `yieldedLiterals` counts a name as read when it appears in a hole *and* the sentence is a substituted template — a hole has no room for a bare word, so a bare word read only through a hole stays data (`|| 'share'` is still not copy). Both branches of that sentence are then reported on their own merit, and the baseline grows by exactly one pin. - `core/test/i18n-missing-entries.test.js` gains three tests (below); `changelog.d/3077-i18n-indexed-read.changed.md` carries the note. The `patches/ds-bot/files/` copies of the two test-side files are the same bytes (`ds-bot-patches check` proves it). ## Red first Base `f2105675` with the three new tests present and `scripts/i18n-usage.mjs` at its base blob, run twice; output identical after normalising durations and the checkout path (both normalised files sha256 `97e7ad2a45aeb92447ea…`). ``` TMPDIR=/tmp/i18nred TZ=UTC node --import ./test/harness-resolve.mjs --test \ --test-name-pattern '#3077' test/i18n-missing-entries.test.js ``` ``` not ok 1 - #3077: the tokenizer closes the bracket it opened location: 'core/test/i18n-missing-entries.test.js:287:1' error: Expected values to be strictly deep-equal: + actual - expected actual: ['=', '['] expected: ['=', '[', ']'] (assertion at :289:10) not ok 2 - #3077: a literal beside an indexed read is still copy location: 'core/test/i18n-missing-entries.test.js:294:1' error: Expected values to be strictly deep-equal: + actual - expected actual: [] expected: ['None yet'] (assertion at :299:10) not ok 3 - #3077: a name a template interpolates is a read, not a dead local location: 'core/test/i18n-missing-entries.test.js:306:1' error: Expected values to be strictly deep-equal: + actual - expected actual: [] expected: ['{} stays a friend: still shared through {}.', 'Your own Bots stay shared. {} can share it again.'] (assertion at :316:10) # tests 3 # pass 0 # fail 3 ``` Test 2 is the defect Product reported (`[]` where the copy is `'None yet'`). Test 3 is the pin the `]` fix flips: on the base it reports nothing because the whole `FriendNote` body was lost to the overrun. ## Verification at the tip Tip `cca7f387d09850e0f3bd45bbe4c6a06978de15c9` (tree `1bd6027fa1a4051097bab6de76bdf6149cd0eb39`, base `f2105675`), **1 commit**, 6 files **+104/−6**: - `scripts/i18n-usage.mjs` +19/−4, `core/test/i18n-missing-entries.test.js` +41/−1, `patches/ds-bot/files/test/i18n-missing-entries.test.js` +41/−1, the two baselines +1 each, `changelog.d/3077-i18n-indexed-read.changed.md` +1 - `i18n` suites (`i18n`, `i18n-plugins`, `i18n-plugins-missing-zh`, `i18n-template-draws`, `i18n-missing-entries`): **39/39** - the three new tests: 3/3, `--test-name-pattern '#3077'` - `node scripts/i18n-usage.mjs --missing` → `167 drawn copy strings with no zh entry in 4 files` (main 166; the one new pin is `plugins/social/client.js:Your own Bots stay shared. {} can share it again.`) - `node scripts/i18n-usage.mjs` → `20 bare draws in 4 files` (unchanged); `--template-draws` → `43 template draws … in 3 files` (unchanged) - `node scripts/i18n-usage.mjs --missing --write core/test/i18n-missing-entries.baseline.json --force` → `wrote 167 entries (+0/-0)` (the pinned file is exactly what the scan now produces; the +1 is the accepted intake, nothing was removed) - `node core/scripts/build-client.mjs --check`: exit 0 - `node scripts/ds-bot-patches.mjs check`: `ok ds-bot/ = Fei2411/DeepSeek-Bot@2b10c70 (tree 5834c00, v0.1.5)`, `ok 10 patches apply cleanly in order`, `ok core/ equals ds-bot/ + patches/ds-bot/`; `canonical`: `ok patches/ds-bot/ is what generate writes (generate is a no-op)` - `node .github/scripts/changelog.mjs check`: `7 fragment(s) in changelog.d/ are well formed` - `ARCH_DOCS_BASE=f2105675ee6fee5fefbef9e73ef32c4003677e90 node core/scripts/check-arch-docs.mjs`: `arch-docs: OK (0 changed file(s))` - `git rev-list --count f210567..HEAD` = 1 ## Checkpoint <!-- flybot:checkpoint --> ```json {"pr":3078,"claim":"Dev-3","issue":3077,"summary":"`tokens()` sent a punct token for `(` and `[` but not `]`, so bracket depth never returned to zero: `? :` and `||` stopped splitting and `return list[0] || 'None yet'` was dropped. `]` now emits its token before the single `calls.pop()`, so each close decrements once. Closing the bracket flipped one existing pin off by accident, so templates record the names their `${}` holes interpolate; a name read only in a hole counts for a substituted template. `--missing` moves 166 to 167 (a string already on main); bare draws stay 20, template draws 43, `--write --force` is +0/-0, no gate file touched.","files":["scripts/i18n-usage.mjs","core/test/i18n-missing-entries.test.js","core/test/i18n-missing-entries.baseline.json","patches/ds-bot/files/test/i18n-missing-entries.test.js","patches/ds-bot/files/test/i18n-missing-entries.baseline.json","changelog.d/3077-i18n-indexed-read.changed.md"],"checks":{"ran":"base f210567 + three new tests: 3 tests, 0 pass, 3 fail; tip: five i18n suites 39/39, --missing 167, bare 20, template 43, --write --force +0/-0, build-client --check exit 0, ds-bot-patches check 3 ok + canonical no-op, changelog 7 fragments, 1 commit","result":"green","at":"cca7f387d09850e0f3bd45bbe4c6a06978de15c9"},"next":"head frozen at cca7f38; the PASS at this tip lets enqueue-guard pass; merge then closes #3077"} ``` <!-- /flybot:checkpoint -->
1 parent f210567 commit ae268ae

6 files changed

Lines changed: 104 additions & 6 deletions

File tree

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
`scripts/i18n-usage.mjs` now closes the `]` it opens: an indexed read like `list[0]` no longer leaves the scan inside a bracket it can never close, so a literal beside it (`return list[0] || 'None yet'`) is reported by `--missing` instead of being swallowed along with the rest of the file. The same walk also records the names a `${}` hole interpolates, so a sentence a helper builds from a local counts as read rather than as a dead local. The missing-entry baseline grows by one pin — `Your own Bots stay shared. {} can share it again.`, the other branch of a sentence already pinned — and the bare-draw and template-draw counts are unchanged. See [#3077](https://github.com/ThinkFlowLab/FlyBot/issues/3077).

‎core/test/i18n-missing-entries.baseline.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -110,6 +110,7 @@
110110
"plugins/social/client.js:You already joined with that link using other Bots. Leave the group first to join with different ones.",
111111
"plugins/social/client.js:You asked them to share a Bot.",
112112
"plugins/social/client.js:Your friend gets this Bot only.",
113+
"plugins/social/client.js:Your own Bots stay shared. {} can share it again.",
113114
"plugins/social/client.js:mine#1",
114115
"plugins/social/client.js:mine#2",
115116
"plugins/social/client.js:pending",

‎core/test/i18n-missing-entries.test.js‎

Lines changed: 41 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ import { spawnSync } from 'node:child_process'
3030
import { readFileSync } from 'node:fs'
3131
import { test } from 'node:test'
3232
import { fileURLToPath } from 'node:url'
33-
import { baselineOf, missingCopy, scanMissing } from '../../scripts/i18n-usage.mjs'
33+
import { baselineOf, missingCopy, scanMissing, tokens } from '../../scripts/i18n-usage.mjs'
3434

3535
const baselineFile = new URL('./i18n-missing-entries.baseline.json', import.meta.url)
3636

@@ -278,3 +278,43 @@ test('a key the plugin table already translates is not a gap', () => {
278278
].join('\n')
279279
assert.deepEqual(missingCopy(source, 'plugins/sample/client.js'), [])
280280
})
281+
282+
// The tokenizer opened a bracket on `[` and never closed it, so every depth in this file counted one
283+
// too many from the first `[…]` onward: `? :` and `||` no longer split at the top level, and a
284+
// `return` that reads an index — `return list[0] || 'None yet'` — handed back one unsplittable part,
285+
// which is not a literal, so the sentence was dropped and both i18n checks stayed green. The token is
286+
// the fix at the source: an opener closes what it opened (Product, round 138).
287+
test('#3077: the tokenizer closes the bracket it opened', () => {
288+
const punct = tokens('const first = list[0]').filter(token => token.type === 'punct').map(token => token.value)
289+
assert.deepEqual(punct, ['=', '[', ']'])
290+
})
291+
292+
// The same walk, from the reader's side: a literal beside an indexed read is copy a zh window still
293+
// shows in English.
294+
test('#3077: a literal beside an indexed read is still copy', () => {
295+
const source = [
296+
"function emptyLabel(list) { return list[0] || 'None yet' }",
297+
"h('span', null, emptyLabel(items))",
298+
].join('\n')
299+
assert.deepEqual(missingCopy(source, 'plugins/sample/client.js').map(entry => entry.key), ['None yet'])
300+
})
301+
302+
// A sentence a helper keeps in a local and interpolates into what it draws is read by the person, so
303+
// the scan has to see the name the hole names. Closing the bracket above turned this pin's old
304+
// accident off — its `const` sits inside a run that used to overrun the file — so both branches of
305+
// the sentence are reported instead of one, and neither by an overrun.
306+
test('#3077: a name a template interpolates is a read, not a dead local', () => {
307+
const source = [
308+
'function FriendNote({ who, linked, shared }) {',
309+
' const line = shared > 0',
310+
' ? `${who} stays a friend: still shared through ${linked}.`',
311+
' : `Your own Bots stay shared. ${who} can share it again.`',
312+
" return h('span', null, `Remove? ${line}`)",
313+
'}',
314+
'h(FriendNote, { who, linked, shared })',
315+
].join('\n')
316+
assert.deepEqual(missingCopy(source, 'plugins/sample/client.js').map(entry => entry.key), [
317+
'{} stays a friend: still shared through {}.',
318+
'Your own Bots stay shared. {} can share it again.',
319+
])
320+
})

‎patches/ds-bot/files/test/i18n-missing-entries.baseline.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -110,6 +110,7 @@
110110
"plugins/social/client.js:You already joined with that link using other Bots. Leave the group first to join with different ones.",
111111
"plugins/social/client.js:You asked them to share a Bot.",
112112
"plugins/social/client.js:Your friend gets this Bot only.",
113+
"plugins/social/client.js:Your own Bots stay shared. {} can share it again.",
113114
"plugins/social/client.js:mine#1",
114115
"plugins/social/client.js:mine#2",
115116
"plugins/social/client.js:pending",

‎patches/ds-bot/files/test/i18n-missing-entries.test.js‎

Lines changed: 41 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ import { spawnSync } from 'node:child_process'
3030
import { readFileSync } from 'node:fs'
3131
import { test } from 'node:test'
3232
import { fileURLToPath } from 'node:url'
33-
import { baselineOf, missingCopy, scanMissing } from '../../scripts/i18n-usage.mjs'
33+
import { baselineOf, missingCopy, scanMissing, tokens } from '../../scripts/i18n-usage.mjs'
3434

3535
const baselineFile = new URL('./i18n-missing-entries.baseline.json', import.meta.url)
3636

@@ -278,3 +278,43 @@ test('a key the plugin table already translates is not a gap', () => {
278278
].join('\n')
279279
assert.deepEqual(missingCopy(source, 'plugins/sample/client.js'), [])
280280
})
281+
282+
// The tokenizer opened a bracket on `[` and never closed it, so every depth in this file counted one
283+
// too many from the first `[…]` onward: `? :` and `||` no longer split at the top level, and a
284+
// `return` that reads an index — `return list[0] || 'None yet'` — handed back one unsplittable part,
285+
// which is not a literal, so the sentence was dropped and both i18n checks stayed green. The token is
286+
// the fix at the source: an opener closes what it opened (Product, round 138).
287+
test('#3077: the tokenizer closes the bracket it opened', () => {
288+
const punct = tokens('const first = list[0]').filter(token => token.type === 'punct').map(token => token.value)
289+
assert.deepEqual(punct, ['=', '[', ']'])
290+
})
291+
292+
// The same walk, from the reader's side: a literal beside an indexed read is copy a zh window still
293+
// shows in English.
294+
test('#3077: a literal beside an indexed read is still copy', () => {
295+
const source = [
296+
"function emptyLabel(list) { return list[0] || 'None yet' }",
297+
"h('span', null, emptyLabel(items))",
298+
].join('\n')
299+
assert.deepEqual(missingCopy(source, 'plugins/sample/client.js').map(entry => entry.key), ['None yet'])
300+
})
301+
302+
// A sentence a helper keeps in a local and interpolates into what it draws is read by the person, so
303+
// the scan has to see the name the hole names. Closing the bracket above turned this pin's old
304+
// accident off — its `const` sits inside a run that used to overrun the file — so both branches of
305+
// the sentence are reported instead of one, and neither by an overrun.
306+
test('#3077: a name a template interpolates is a read, not a dead local', () => {
307+
const source = [
308+
'function FriendNote({ who, linked, shared }) {',
309+
' const line = shared > 0',
310+
' ? `${who} stays a friend: still shared through ${linked}.`',
311+
' : `Your own Bots stay shared. ${who} can share it again.`',
312+
" return h('span', null, `Remove? ${line}`)",
313+
'}',
314+
'h(FriendNote, { who, linked, shared })',
315+
].join('\n')
316+
assert.deepEqual(missingCopy(source, 'plugins/sample/client.js').map(entry => entry.key), [
317+
'{} stays a friend: still shared through {}.',
318+
'Your own Bots stay shared. {} can share it again.',
319+
])
320+
})

‎scripts/i18n-usage.mjs‎

Lines changed: 19 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -139,6 +139,9 @@ export function tokens(source) {
139139
index += 1
140140
let value = ''
141141
let substituted = false
142+
// The names a `${}` hole mentions: the hole's code is not tokenised, but a local interpolated
143+
// here is read by the reader all the same (`${keepLine}` draws the sentence that local holds).
144+
const holes = []
142145
while (index < source.length && source[index] !== '`') {
143146
if (source[index] === '\\') {
144147
const [each, next] = escapeAt(source, index)
@@ -157,6 +160,12 @@ export function tokens(source) {
157160
if (source[index] === '{') depth += 1
158161
else if (source[index] === '}') depth -= 1
159162
else if (source[index] === '\n') line += 1
163+
else if (isIdent(source[index])) {
164+
let name = ''
165+
while (index < source.length && isIdent(source[index])) { name += source[index]; index += 1 }
166+
holes.push(name)
167+
continue
168+
}
160169
index += 1
161170
}
162171
continue
@@ -166,7 +175,7 @@ export function tokens(source) {
166175
index += 1
167176
}
168177
index += 1
169-
place(substituted ? { type: 'string', value, substituted: true } : { type: 'string', value })
178+
place(substituted ? { type: 'string', value, substituted: true, holes } : { type: 'string', value })
170179
previous = 'string'
171180
lastValue = value
172181
continue
@@ -193,7 +202,7 @@ export function tokens(source) {
193202
continue
194203
}
195204
if (ch === ')') { place({ type: 'punct', value: ')' }); calls.pop(); previous = ')'; index += 1; continue }
196-
if (ch === ']') { if (current()?.kind === 'other') calls.pop(); previous = ']'; index += 1; continue }
205+
if (ch === ']') { place({ type: 'punct', value: ']' }); if (current()?.kind === 'other') calls.pop(); previous = ']'; index += 1; continue }
197206
// Braces are emitted too, so a copy table can be delimited by its own body rather than by
198207
// guessing from the props depth.
199208
if (ch === '{') {
@@ -609,15 +618,21 @@ function yieldedLiterals(list, body) {
609618
const rest = list.slice(at + 3, body.end)
610619
// The name must be read again — that is how the helper builds its sentence — and must not be
611620
// compared: `side === 'below'` says the local holds a machine id, not the words on the label.
612-
const used = rest.some(each => each.type === 'ident' && each.value === name.value)
621+
const readPlain = rest.some(each => each.type === 'ident' && each.value === name.value)
622+
// A name interpolated into a template is read just as plainly as one passed as an argument:
623+
// `${keepLine}` puts the sentence that local holds on the screen. A hole has no room for a bare
624+
// word, though — the sentence around it says what the value means — so only a sentence (a
625+
// substituted template) counts when the hole is the only place the name is read.
626+
const readInHole = rest.some(each => each.type === 'string' && Array.isArray(each.holes) && each.holes.includes(name.value))
627+
const used = readPlain || readInHole
613628
const comparedName = rest.some((each, index) => each.type === 'ident' && each.value === name.value
614629
&& rest[index - 1]?.value !== '.'
615630
&& ((rest[index - 1]?.type === 'punct' && COMPARISONS.has(rest[index - 1].value))
616631
|| (rest[index + 1]?.type === 'punct' && COMPARISONS.has(rest[index + 1].value))))
617632
// A local is only half a sentence: `|| 'share'` is the last resort key of a lookup, not words on
618633
// the screen, so a bare word from a local has to be spelled as a sentence (`'the room'`, a
619634
// template with a hole) to count. A helper's own `return 'Closed'` is not filtered this way.
620-
const sentence = values.filter(each => each.substituted || /\s/.test(each.value))
635+
const sentence = values.filter(each => readPlain ? each.substituted || /\s/.test(each.value) : each.substituted)
621636
if (used && !comparedName) literals.push(...sentence)
622637
}
623638
return literals

0 commit comments

Comments
 (0)