Skip to content

v3: 多言語CC0 pilotとlanguage detector観測を統合 - #140

Merged
PyYoshi merged 2 commits into
devfrom
v3/script-corpus-results
Sep 20, 2026
Merged

PyYoshi merged 2 commits into
devfrom
v3/script-corpus-results

Conversation

@PyYoshi

@PyYoshi PyYoshi commented Sep 20, 2026

Copy link
Copy Markdown
Owner

変更

  • native dev a56fd9584d11e7a1f5cbcc75df9b5c3ec4837ab4へ固定(uchardet misdetecting windows-1251 for mac-cyrillic #24/#25は各11CI成功で統合済み)。
  • 日本語2文・アラビア語2文・ヘブライ語27文のCC0 validation pilot。186 variantsの再生成一致と、代表的評価には不足することを日本語で記録。
  • opt-in traceのlanguage state/counter観測。既存の小fixtureで候補と旧snapshot不変、GCC同一buildの全61 engine object member不変。
  • sequence分母と4分類counter合計の違いを記録。新しいSBCS trainerの適合や精度改善とは混同しない。

検証

uvで拡張を再build。最新devへrebase後232 passed / 14 skipped / 68 subtests passed、mypy、diff check成功。
language観測比較4 testsとnested候補比較はnative側で別途実行済み。

公開API/model/規格変更なし。P01再開・新規fuzz・大入力・独立holdout評価なし。
先行PR #139のCI完了後に一度pushし、2件のnative統合を1回のwheel matrixへ集約。

Refs #120, #124, #114

@PyYoshi
PyYoshi marked this pull request as ready for review September 20, 2026 17:05
@PyYoshi
PyYoshi merged commit 5aafb07 into dev Sep 20, 2026
20 checks passed
@PyYoshi
PyYoshi deleted the v3/script-corpus-results branch September 20, 2026 17:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant