Repository navigation
Normalize Chinese locale tags when looking up phonetic letters, and announce Latin letters with their position in zh-CN - #56
Conversation
The phonetic letters JSON only has zh, zh-CN and zh-TW nodes, and the lookup used the raw BCP-47 tag, so many Chinese locales never matched: zh-Hans-CN (reported by many Chinese ROMs such as Honor/Huawei), zh-Hans, zh-Hant-TW, zh-HK, zh-MO and others all fell through to the zh node - simplified users got the inherited table and traditional users on zh-Hant-* locales got simplified-style descriptions. Normalize the tag before lookup: script Hans or region CN/SG resolves to zh-CN, script Hant or region TW/HK/MO resolves to zh-TW (script takes priority), and anything else keeps the existing behavior (bare zh uses the zh node; per-entry fallback to zh still applies). Unit tests cover the full tag matrix.
Prepend the letter's position (e.g. "第1个字母, alpha") to the 26 Latin letter entries in the zh-CN node, matching the format commonly used by screen readers in China. With locale normalization in place, devices reporting zh-Hans-CN or zh-Hans now reach these entries directly.
aaron-gh
left a comment
There was a problem hiding this comment.
No functional defects found in the normalization logic. Checked against getPhoneticLetter and LocaleUtils.parseLocaleString: for zh-Hans-CN the parsed locale is zh/HANS, so a letter missing from the normalized node still reaches the base-language zh fallback because the country is non-empty. The map cache is now keyed by the normalized tag, which is correct. Extension and private-use tags (zh-CN-x-foo, zh-u-ca-chinese) end up at the right node or at zh.
Minor points:
ProcessorPhoneticLetters.java,normalizePhoneticLocale: thesubtags.length == 0check is dead, becauseString.splitwith a non-empty input always returns at least one element. For an empty stringsubtags[0]is""and is handled by theequalsIgnoreCasecheck.- Same method: the
TW/HK/MOcondition line is over 100 columns. Wrap it to match the file's formatting. - Same method:
split("-", 3)plus the length-4 and length-2 checks accept anything of that length, including the tail of an extension such asx-ab. This is harmless today because only exactHans/Hant/CN/SG/TW/HK/MOvalues match. Stopping at the first singleton subtag (x,u) would be stricter. getPhoneticLetterstill buildsLocaleSpanand lowercases with the un-normalizedparsedLocale(zh/HANS). This predates the PR, but the spoken text forzh-Hans-CNis now zh-CN content tagged with an odd locale. Consider using the normalized tag forparsedLocale.phonetic_letters.json: the position prefix applies to every consumer of thezh-CNnode, not only letter navigation. That includesgetPhoneticTextand the typing and keyboard-focus paths that substitute the phonetic letter for the letter (getPhoneticLetterTakesPlaceOfLetter). Those will now speak第N个字母, alphatoo. Confirm this is intended.ProcessorPhoneticLettersLocaleTest.kt: the test only calls a static pure function, soRobolectricTestRunnerand@Configare unnecessary and slow it down. A plain JUnit runner would do. No test covers thegetPhoneticLetterpath, so thezh-CNdata change and the lookup change are untested together.
Not compiled or tested.
- Drop dead subtags.length == 0 check (String.split never returns an empty array for non-empty input) - Wrap the region condition to match the file's 100-column limit - Stop at singleton subtags (x-, u-) so extension or private-use tails are never mistaken for script or region subtags - Derive the parsed locale used for lowercasing and the LocaleSpan from the normalized tag instead of the raw locale - Test the locale normalization with a plain JUnit runner and cover extension/private-use tags - Add an end-to-end lookup test over the real phonetic_letters.json covering the zh-CN letters, the zh fallback for Chinese characters and zh-TW's Latin-letter fallback
|
Thanks for the careful review — all points addressed in 0f4e704:
Unit tests for these changes are running on our fork's test-build branch: https://github.com/XHG20033/SandFly/actions/runs/42854209554 (Note: this reply and the accompanying changes were drafted with AI assistance and reviewed by @XHG20033.) |
The zh-Hans-CN fallback case assumed the zh-CN node has no Chinese characters, which no longer holds on branches with expanded zh-CN data. Use a synthetic loader so the base-language fallback path is tested regardless of the resource contents.
0f4e704 to
6e2e45d
Compare
aaron-gh
left a comment
There was a problem hiding this comment.
No defects found in normalizePhoneticLocale; subtag handling, singleton cut-off and script-over-region priority behave as the tests describe. Points on the data change:
-
talkback/src/main/res/raw/phonetic_letters.json,zh-CNnode: the position prefix is stored in the shared letter data, so it applies to every consumer ofgetPhoneticLetter, not only letter navigation.getPhoneticText(ProcessorPhoneticLetters.java, around lines 337-359) buildsa: 第1个字母, alphaand joins entries with", ", givinga: 第1个字母, alpha, b: 第2个字母, bravo. The separator inside each entry is the same as the one between entries, so entries are harder to tell apart when spelled out. A different inner separator, or applying the prefix only in the traversal path, avoids this. -
Same node: with
getPhoneticLetterTakesPlaceOfLetterenabled (lines 243 and 260), typed-key echo and focus announcements for zh-CN now always include the position. That is intentional per the PR discussion, but it is a behaviour change for all zh-CN users of those paths, including those who already getzh-Hans-CNresolved tozhand a different result before. -
The Chinese text is hard-coded in the JSON rather than in a string resource, so it cannot be adjusted per locale variant.
zh-TWand thezhfallback do not get an equivalent prefix, so Traditional Chinese devices that now resolve tozh-TWremain inconsistent withzh-CN. -
ProcessorPhoneticLettersLookupTest.kt,zhTwLettersFallBackToZhNode: depends on thezhnode keeping"a": "Alfa"; the other tests in that file use synthetic data for exactly that reason, so this one is inconsistent and will break on unrelated data edits.
aaron-gh
left a comment
There was a problem hiding this comment.
Changes requested.
ProcessorPhoneticLetters.java:431, 453: theLocaleSpanis now built from the normalized node key, sozh-HKandzh-MOtext is markedzh_TW, andzh-SGtextzh_CN. Switch language automatically and Switch dialect automatically are on by default, soLanguageSwitchkeeps the mark, and a zh-HK user hears the description in a separate utterance with the zh-TW voice. Use the normalized tag only to choose the JSON node and for the base-language fallback, and mark the result withLocale.forLanguageTag(locale)from the original tag. This replaces the earlier suggestion to use the normalized tag forparsedLocale.ProcessorPhoneticLettersLookupTest.kt: assert theLocaleSpanlocale as well as the text, at least forzh-HKandzh-Hans-CN.phonetic_letters.json:59177-59202: #44 proposedAlfa,第1个字母. Thezh-TWdata and the agreed template ("长短的长,长大的长") use the fullwidth,, and no Chinese entry uses,. Use,, and put the word first as in #44, or state why position first is better.normalizePhoneticLocalereturnsnullwithout@Nullableon the return type. All callers passtoLanguageTag()output, so remove the null branch or annotate it.- PR description: "Spelling ... untouched" is incorrect. Spelling, typing, keyboard keys and Announce phonetic pronunciation all read the zh-CN node and now include the position. Update the description.
- The normalized tag now only chooses the JSON node and drives the base-language fallback; the LocaleSpan and lowercasing use the original tag, so zh-HK/zh-MO/zh-SG users hear the description in their own voice instead of the voice of the node the tag maps to - Assert the LocaleSpan locale for zh-HK and zh-Hans-CN - Use the fullwidth comma in the zh-CN letter entries, consistent with the rest of the Chinese entries - Drop the null branch of normalizePhoneticLocale; callers always pass toLanguageTag() output - Use synthetic data in the zh-TW fallback test so it does not depend on the JSON contents
|
All five points addressed in ae16bcf:
Also from your earlier notes: Unit tests for these changes are running on our fork's test-build branch: https://github.com/XHG20033/SandFly/actions/runs/42941195595 (Note: this reply and the accompanying changes were drafted with AI assistance and reviewed by @XHG20033.) |
ab2f055 to
ae16bcf
Compare
Resolves #44 (first step).
Locale normalization. Devices report Chinese locales in more shapes than the JSON's two keys (
zh-CN/zh-TW) cover. A device reportingzh-Hans-CN, for example, previously fell through to the plainzhnode, so thezh-CNletter entries never played.ProcessorPhoneticLettersnow normalizes tags before lookup:zh-Hans*→zh-CN,zh-Hant*→zh-TWCN/SG→zh-CN,TW/HK/MO→zh-TWzh-Hans-MO→zh-CN)zhLocaleUtils.parseLocaleStringdoesn't parse script subtags yet (see the TODO referencing b/455704867), so the method parses the tag itself; covered byProcessorPhoneticLettersLocaleTest(13 tag shapes, conflict cases, casing, non-Chinese pass-through).Latin letters in zh-CN only. The 26 letter entries in
zh-CNare prepended with their position ("第1个字母,alpha"), matching the format commonly used by screen readers in China. Note that every consumer ofgetPhoneticLetterreads this node, so character navigation, spelling, typed-key echo, keyboard focus and "Announce phonetic pronunciation" all include the position prefix; the other nodes (zh,zh-TW) are untouched.Provenance: no third-party word lists are involved; the entries are the project's own Apache-2.0 data with positions added.
Tested: built via fork CI (phone-full + unit tests green). On-device with a
zh-Hans-CNdevice, letter navigation now announces "第1个字母, alpha". Manually switching the device to Traditional Chinese confirmedzh-TWbehavior is unchanged.AI disclosure: AI assistance was limited to GitHub operations. The changes themselves were designed in discussion in #44 and manually reviewed and tested.