diff --git a/docs/dncl/README.md b/docs/dncl/README.md index c4ea4c64d3..d141832ecb 100644 --- a/docs/dncl/README.md +++ b/docs/dncl/README.md @@ -139,8 +139,9 @@ JaAST では 1 つの形にする。エディタの日本語 → Ruby(`dnclToR DNCL 風の形の読み方(切りかえる前の変換に合わせたもの): - `:` の無い開く行(`もし … ならば`、`… の間`、`… ずつ増やしながら`、`関数 名前(引数)`、`名前(引数) を定義する`)のブロックは、閉じる行 - (`を実行する` / `を繰り返す` / `と定義する`。`end` も)か `⎿` で終わる。中の字下げは見ない。閉じる言葉とブロックの種類は照らし合わせない(前と同じ)。 - ただし数える繰り返しは、どの言葉で閉じても増分を足す(前は `を繰り返す` で閉じたときだけ足したので、`を実行する` で閉じると終わらなかった)。 + (`を実行する` / `を繰り返す` / `と定義する`。`end` も)か `⎿` で終わる。中の字下げは見ない。閉じる言葉はブロックの種類に合わせる(`もし` は `を実行する`、`〜の間` と数える繰り返しは `を繰り返す`、関数は `と定義する`)。 + 合わない言葉で閉じると、初心者に少しでも多く指摘するため誤りにする(下の「閉じ忘れのエラー」)。`end` は Ruby と同じく、どのブロックも閉じる。 + 標準の形(字下げ)では閉じる言葉を書かないので関係しない。 - `:` のある行のブロックは字下げで終わる。同じ字下げの閉じる行はそのブロックを閉じる。`:` があっても次の行が深くないときは、閉じる行で終わるブロックになる。2 つの形は混ぜてよい。 - `←` は `=`、`かつ` / `または` は `and` / `or`、後ろに付く `でない` は、前の `かつ` / `または`・`(`・`,`・`=` までをまとめて否定する(`a > 10 でない` は `not (a > 10)`)。 `でない` の後に演算子が続くときは、否定したものがその左側になる(`a でない == b` は Ruby の `!a == b`)。 @@ -150,21 +151,24 @@ DNCL 風の形の読み方(切りかえる前の変換に合わせたもの) | ファイル | 役割 | |---|---| | `packages/ja-notation/`(ワークスペースのパッケージ `@smalruby/ja-notation`。MIT。`LICENSE` あり) | ライブラリ。依存は無く、`src/` の外を import しない(`packages/ja-notation/test/boundary.test.js`)。使い方・JaAST の形・テストのデータは `packages/ja-notation/README.md`。下の `ja-notation/<ファイル>` は `packages/ja-notation/src/<ファイル>` | -| `ja-notation/lexer.js` | 1 行を字句に分ける。全角の演算子・`≦` など、数の全角数字(`12`)、コメントの `#` は半角のものとして読み、`“…”` は文字列として読む。名前の文字の判定(`isName` / `isNameHead` / `isNameChar`。下の「名前に使える文字」)もここにあり、エディタの判定(`paren-utils.js`)も同じものを使う | +| `ja-notation/lexer.js` | 1 行を字句に分ける。文字列とコメントの外の全角英字・全角数字(`x1` `12`)はまず半角にし(`halfWidthAlphanumerics`)、全角の演算子・`≦` など、コメントの `#` は半角のものとして読み、`“…”` は文字列として読む。名前の文字の判定(`isName` / `isNameHead` / `isNameChar`。下の「名前に使える文字」)もここにあり、エディタの判定(`paren-utils.js`)も同じものを使う | | `ja-notation/parse.js` | 日本語を読む。標準の形(行番号・`|` `⎿`・字下げ・`ならば:`・`and` / `or` / `not`)と DNCL 風の形(閉じる行・`←`・`かつ`・`でない`・「...」)。JaAST の形は `ja-notation/ja-ast.js` | | `ja-notation/build-ruby.js` | JaAST → Ruby の構文木(prism のノード名を使う小さな形)。変数の名前・関数・入力は外から受けとる表で決める | | `ja-notation/write-ruby.js` | Ruby のテキストと、Ruby の行ごとの日本語の行を書く | +| `ja-notation/run.js` | 構文木をたどる処理(ジェネレーター)を自前のスタックで動かす(`run`・`each`)。深い構文木で呼び出しのスタックが尽きないように | 名前に使える文字(`ja-notation/lexer.js` の `NAME_HEAD_RE` / `NAME_TAIL_RE`。Issue #1714): -- 1 文字目: 英字・`_`・どの文字体系の文字(Unicode の `\p{L}`。かな・漢字(CJK 拡張 A〜と BMP の外の `𠮷` も)・`々`・`〆`・`ヵ` `ヶ`・`ー`・全角英字・ギリシャ文字など)・ +- 1 文字目: 英字・`_`・どの文字体系の文字(Unicode の `\p{L}`。かな・漢字(CJK 拡張 A〜と BMP の外の `𠮷` も)・`々`・`〆`・`ヵ` `ヶ`・`ー`・ギリシャ文字など)・ 文字としての数字(`\p{Nl}`。`〇` `Ⅱ`)と、ひらがな・カタカナの区画の記号(`゛` `・` など。前から読めていたもの) -- 2 文字目から: 上に加えて、数字(`\p{Nd}`。全角の `1` も)と結合文字(`\p{M}`。`か` + `゙`) +- 2 文字目から: 上に加えて、数字(`\p{Nd}`)と結合文字(`\p{M}`。`か` + `゙`) - Ruby はこれらをどれも名前に使えて、Ruby → 日本語は名前をそのまま書く(`@佐々木` → `佐々木`)。日本語 → Ruby が同じ範囲を名前として読むので、書いた名前は読み戻せる。 -- 全角英字は全角のままの別の名前(`x` は `x` ではない)。Ruby でも別の変数なので、見た目の揺れとして 1 つにそろえない(`policy.md`)。 +- 全角英字・全角数字は、文字列(`"…"` `'…'` `「…」`)とコメントの外では、読む前にすべて半角にする(`x1` は `x1`、`and` は `and`)。見た目だけの揺れで(`policy.md`)、Ruby モードの自動置換も半角にするのと同じ考え。 + 文字列とコメントの中は書いたとおりに残す(`表示する("ABC")` は `"ABC"` のまま)。全角と半角は 1 字ずつ同じ長さなので、誤りの列は変わらない。 + Ruby → 日本語は Ruby の名前をそのまま書く(`@x` → `x`)ので、日本語から Ruby に戻すと `@x` になる。 - `☆` `①` `😀` のような記号は名前に使えず、「「☆」が読めません」の誤りになる(BMP の外の文字も 1 文字として示す)。 -誤りは `{code, message, line, column}` に、日本語モードの文言を作るための `name`(字句)・`opener` / `direction` / `colon` / `word`(ブロック)・`block`(`stray-closer` / `closer-indent` / `stray-branch`)を添えて返す。 +誤りは `{code, message, line, column}` に、日本語モードの文言を作るための `name`(字句)・`opener` / `direction` / `colon` / `word`(ブロック)・`block`(`stray-closer` / `closer-indent` / `closer-kind` / `stray-branch`)・`expected`(`closer-kind` のブロックの閉じのことば)を添えて返す。 テスト: ライブラリだけのもの(表なし・ほかの処理系の表)は `packages/ja-notation/test/`(データは `test/data/*.json`。`node --test`)。スモウルビーの表を通したものは scratch-gui の `test/unit/lib/ja-notation/corpus.test.js`(コーパスのゴールデンとの比較)と `build-and-write-ruby.test.js`。 @@ -178,21 +182,26 @@ prism の解析の関数は外から受けとり(`rubyToJa(source, { parse, .. 本体のほかの文が変数も増分も変えないときだけ `〜ずつ増やしながら繰り返す` にする。それ以外は `i = 1` と `i <= 10 の間繰り返す:` のまま。 `(a / b).to_i` は `a ÷ b`、`-((a / b).to_i)` は `-a ÷ b`(日本語では `-(a ÷ b)` と読むので、符号を `÷` の左に置く)。`((-a) / b).to_i` は `÷` にしない(`整数(-a / b)`)。 `!(...)` のかっこは残す(`not (a > 10)`)。`until 条件` は `not 条件 の間繰り返す:`、`wait until 条件` は中身の無い繰り返し。 -- 日本語に書きかえるものが無い Ruby(`with_screen_refresh do` / `loop do` / `list.each do |x|` などの `xxx do`、`unless`、後置の `if`、`begin … end while`、`$g`、`nil` など)は、 +- 日本語に書きかえるものが無い Ruby(`with_screen_refresh do` / `loop do` / `list.each do |x|` などの `xxx do`、`unless`、後置の `if`、`begin … end while`、`nil` など)は、 **書いてあるとおりの Ruby のまま**(`end` も)残す(JaAST の `RubyCode`)。中の命令と数の誤りは、そのまま Ruby の行に出す(日本語モードはこうした Ruby を切りかえで受け付けない)。 - それ自体が誤りでない Ruby(`case` / `when`、ハッシュ `{}`、シンボル `:a`、後置の `unless` / `if`、三項演算子、`nil`、`$g` など)も、黙って残さずに `no-ja-form` の誤りにする + それ自体が誤りでない Ruby(`case` / `when`、ハッシュ `{}`、シンボル `:a`、後置の `unless` / `if`、三項演算子、`nil` など)も、黙って残さずに `no-ja-form` の誤りにする (黙って残すと、日本語に戻したときに「「@」は使えません」「「{」が読めません」のような見当違いの誤りになるため)。誤りは残した Ruby のいちばん外のものに 1 つだけ出し、 キーワードで始まるものはキーワードの位置にキーワードの名前で(`unless`)、それ以外は先頭の位置に Ruby の字(1 行目の 20 字まで。長いものは `…`)で出す。 その Ruby に命令や `break` の誤りがすでに出ているときは足さない。prism が文法の誤りを出したときは足さない(立て直しのノードの誤りを並べないため)。 + ステージの変数・リスト(`$g`)は、日本語モードが想定しない([`design.md`](design.md) の「ステージの変数は書かない」)ので、 + 書いてあるとおりに残したうえで、`$g` の名前の位置(読むところ・代入・`+=`・多重代入の左辺・`$g.push(1)` の受け手)に `stage-variable` の誤りを出す + (`no-ja-form` や `push` の命令の誤りは足さない)。 表を渡さないとき(ほかの処理系)は、大文字の名前(Ruby の定数 `A`)も名前として読む(`buildRuby` がそう書くため)。 `N.times do` と `(a..b).step(s) do |i|` / `a.step(b, -s) do |i|` は `〜ずつ増やしながら繰り返す` にする。 - 書き方(`write-ja.js`): 下の「書き出しの形」。演算子は日本語の読み方(`parse.js`)の強さで、要るところだけかっこを付ける(Ruby に書いてあったかっこは残さない。`not` の後ろは上のとおり)。 空行は Ruby の空行の位置に残す(読み手が JaAST に `blankLines` を付ける)。行末のコメントは、その Ruby の行から作った日本語の行の末尾に付け直す。 - 誤り: 日本語モードに無い命令(`unknown-command`。`(` の付いた名前、括弧の無いメソッド、受け手のあるメソッド、`xxx do` の `xxx`、`break` / `next` / `redo` / `retry`)、 - 数の書き方(`bad-number`)、後判定の繰り返し(`post-test-loop`。`begin` と `end while` の行)、日本語に無い書き方(`no-ja-form`。上の `RubyCode`)、Ruby の文法の誤り(`ruby-syntax`)。行と列は Ruby の行。 + 数の書き方(`bad-number`)、後判定の繰り返し(`post-test-loop`。`begin` と `end while` の行)、日本語に無い書き方(`no-ja-form`。上の `RubyCode`)、ステージの変数(`stage-variable`)、Ruby の文法の誤り(`ruby-syntax`)。行と列は Ruby の行。 命令と取らないのは、表にある呼び出し、プログラムで `def` した名前、`dncl-keywords.js` の `RUBY_LITERALS`(`answer` / `ask` など)。ローカル変数は prism が変数と分かるので取らない。 -- 構文木を歩く処理は、長い `and` / `+` の並び(左に深い木)をくり返しで歩くので、項が数千あってもスタックが尽きない。prism 自身はおよそ 5000 項の式を解析できないので、それより長い式は誤り - (「プログラムが長すぎて日本語にできません」)にする。 +- 構文木をたどる処理(日本語の読み方・Ruby の組み立てと書き出し・prism の構文木の読み方・日本語の書き出し)は、再帰ではなく自前のスタックでたどる + (`ja-notation/run.js`。処理はジェネレーターで書き、子をたどるところで `yield` する)。項が数千ある式、数千重ねたかっこ・符号・呼び出し、深く入れ子にしたブロックでも、 + JavaScript の呼び出しのスタックが尽きない。prism 自身は構文木を JavaScript に読み出すところが再帰なので、数千項の `+` などは解析できず、 + Ruby → 日本語ではその式を誤り(「プログラムが長すぎて日本語にできません」)にする(`ruby-to-dncl.js`)。 | ファイル | 役割 | |---|---| @@ -214,6 +223,7 @@ ja-notation の誤りは `ja-notation-errors.js` が、生徒に見えている | `post-test-loop` | 後判定の繰り返しの文(`postTestLoop`) | | `unknown-command` | 「「move」は日本語モードでは書けない命令です。Ruby モードでは書けます。」、日本語の名前は「「差分」は定義されていない関数です。「関数 差分(…):」で定義してください。」(`unsupportedCommand`) | | `no-ja-form` | 「「case」は日本語モードでは書けません。Ruby モードでは書けます。」(`unsupportedNotation`。Ruby → 日本語だけ) | +| `stage-variable` | 「ステージの変数($a)は日本語モードでは書けません。Ruby モードでは書けます。」(`stageVariable`。Ruby → 日本語だけ) | | `unclosed` / `empty-block` / `block`(閉じのことば・`そうでなければ`) | 下の「閉じ忘れのエラー」の文 | | そのほか(読めない字句・式が要る など) | ja-notation の文のまま(「「//」が読めません」など) | @@ -250,11 +260,12 @@ DNCL モードでは、DNCL を Ruby に変換してからブロックに変換 | DNCL 風の形の `もし〜ならば` / `〜の間` / `〜ずつ増やしながら` / `〜ずつ減らしながら` / `関数` / `〜 do` を閉じていない | 開いた行に「「もし〜ならば」を「を実行する」で閉じてください」など | | 閉じるブロックが無いのに `を実行する` などがある | 「「を実行する」で閉じるものがありません」 | | `もし` の外の `そうでなければ` / `そうでなくもし` | 「「そうでなければ」の前に「もし〜ならば」がありません」。標準の形(行の終わりが `:`)では、`もし` より浅く字下げしたときにこうなるので「「そうでなければ:」は、「もし〜ならば:」と同じ字下げで書いてください」 | -| 種類の違う閉じのことば(ほかに合うブロックが無い) | 閉じのことばの行に「「もし〜ならば」は「を繰り返す」ではなく「を実行する」で閉じてください」 | +| 種類の違う閉じのことば(`もし … を繰り返す`・数える繰り返しを `を実行する` で閉じるなど) | 閉じのことばの行に「「もし〜ならば」は「を繰り返す」ではなく「を実行する」で閉じてください」。`:` で開いたブロックを同じ字下げの閉じの行で閉じたときは「「〜の間繰り返す:」は…」(ja-notation の `closer-kind`) | - 標準の形か DNCL 風の形かは、開いた行の終わりに `:` / `:` があるか(行末のコメントを除く)で決める(ja-notation の誤りの `colon`)。標準の形は字下げでブロックを示すので、閉じのことばを書くようには言わない。 -- ja-notation は閉じのことばの種類を見ずに、いちばん内側のブロックを閉じる(Ruby の `end` と同じ)。そのため、種類の違う閉じのことばでも数が合えばエラーにしない(`もし … を繰り返す` など)。 -- 数が合わずエラーになったときで、ブロックがすべて DNCL 風の形のときは、閉じのことばの種類を見て言う(`ja-notation-errors.js` の `closerFormBlockErrors`)。内側のブロックを閉じ忘れて外側の閉じのことばが来たとき(`〜の間` の中の `もし` を閉じ忘れて `を繰り返す`)は、外側ではなく**閉じ忘れた内側**の行に付ける。 +- ja-notation は閉じのことばとブロックの種類を照らし合わせ、合わなければ誤りにする(`closer-kind`。数が合っていて Ruby が作れるときも)。`end` はどのブロックも閉じる(Ruby と同じ)。 +- 内側のブロックを閉じ忘れて外側の閉じのことばが来たとき(`〜の間` の中の `もし` を閉じ忘れて `を繰り返す`)は、外側ではなく**閉じ忘れた内側**の行に付ける(ja-notation の `unclosed`。間のブロックがすべて DNCL 風の形のとき)。 + ブロックがすべて DNCL 風の形のときは、エディタも閉じのことばの種類を見て同じことを言う(`ja-notation-errors.js` の `closerFormBlockErrors`)。 - `with_screen_refresh do` / `loop do` / `when_flag_clicked do` / `list.each do |x|` など、日本語に書きかえるものが無い `xxx do`(`|x|` 付きも)は Ruby → 日本語で書いてあるとおりの Ruby のまま(`end` も)残す(`N.times do` と `(a..b).step(s) do |i|` は `〜ずつ増やしながら` に、`while x do` / `until x do` は `〜の間` になる)。構文木で読むので、外側のブロックの閉じ方は変わらない。日本語の中にそうした行があるときの判定は `dncl-validator.js` の `isDoBlockLine`(行末のコメントを除いてから行末の `do` だけを見るので、文字列やコメントの中の `do` は数えない)。日本語モードにはこうした命令が無いので、ほかの対応していない命令と同じくその行に「「with_screen_refresh」は日本語モードでは書けない命令です。…」を出し、Ruby には変換しない(下の「日本語モードに無い命令」)。 - `dnclToRuby` はこのエラーを `errors`(日本語の行)として返し、Ruby は作らない。エディタは Ruby を送らず、エラーの印(`ruby-tab.jsx` の日本語の誤りのマーカー。owner は `DNCL_UNSUPPORTED_OWNER`)をその行に出し、前の変換の印(`smalruby`)は消す。生成AI に相談の確かめ(`genai-dncl-answer.js`)も、この誤りを問題として返す。 @@ -421,7 +432,7 @@ Smalruby は **共通テスト用プログラム表記の形のプログラム | 戻り値(標準の形) | `x * 2 を返す` | `return x * 2`(`返す x * 2` も読む) | | 要素の追加 | `Data に 3 を追加する` | `@_array_Data_.push(3)`(ブロックは「(3) を (Data) に追加する」。Ruby の `<<` も `を追加する` に戻す) | | 余り(全角) | `a % 3` | `a % 3` | -| 全角数字 | `a = 12` | `a = 12`(見た目だけの揺れとして半角で読む。名前の中の `x1` は名前のまま) | +| 全角英字・全角数字 | `a = 12` | `@a = 12`(見た目だけの揺れとして半角で読む。名前の中も `x1` → `x1`。文字列とコメントの中は変えない) | | インデントマーカー (`|`) | ` | a = 1` | ` a = 1`(2 スペース/marker)。`|` `│` も同じ | | 暗黙の `end` (`⎿`) | ` ⎿ a = 1` (1 marker) | ` a = 1` + 自動で `を実行する` 追加。`└` も同じ(英字の `L` は貼り付けで `⎿` に直す) | | 同一行複数 `⎿` | ` ⎿ ⎿ a = 1` | 2 個の `end` を一度に追加 | @@ -549,6 +560,8 @@ Smalruby は **共通テスト用プログラム表記の形のプログラム - `dncl-standard-form.test.js` — 書き出しの標準の形(字下げと `:`・`and` / `or` / `not`・`%`・`を返す`・終端の行を残す場合)と、DNCL 風の形も読むこと - `dncl-loop-jump.test.js` — 繰り返しの中の `break` / `next`(後置の `if` / `unless` 付き・1 行だけ・`redo` / `retry`)がその行のエラーで止まり、`@break` のような Ruby にならないこと。同じ名前の変数は往復できること - `dncl-no-ja-form.test.js` — 日本語に無い書き方(`case` / `when`・ハッシュ・シンボル・後置の `unless` / `if`・三項演算子)が Ruby → 日本語でその場所の誤り(`unsupportedNotation`)になり、日本語モードにある Ruby は誤りにならないこと + - `dncl-stage-variable.test.js` — ステージの変数・リスト(`$a`)が Ruby → 日本語でその名前の位置の誤り(`stageVariable`)になり、スプライトの変数(`@a`)は日本語になること + - `dncl-long-program.test.js` — 項が数千ある式・数千重ねたかっこ・深いブロックが日本語 → Ruby で変換でき、1000 項の Ruby が日本語になること(ライブラリの `packages/ja-notation/test/deep.test.js` も) - `dncl-unsupported-command.test.js` — 日本語モードに無い命令(引数あり・なし・変数と同じ名前・式や条件の中・`xxx do` と `|x|`)がその行のエラーで止まり、定義した関数の呼び出しや組み込み関数は往復できること(DNCL → Ruby と Ruby → DNCL の両方) - `dncl-do-block.test.js` — 日本語にしない `xxx do ... end` が `を実行する` で閉じ、外側の `〜の間` / `もし` / `関数` の閉じのことばがずれないこと(入れ子・`until x do`・後判定の中・文字列やコメントの中の `do`) - `dncl-roundtrip.test.js` — DNCL ↔ Ruby round-trip 安定性 diff --git a/docs/dncl/design.md b/docs/dncl/design.md index 12c68e4ac2..f35a054e05 100644 --- a/docs/dncl/design.md +++ b/docs/dncl/design.md @@ -148,6 +148,15 @@ Ruby や日本語のプログラムの `表示する` は、キャラクター Ruby タブは Scratch の部分集合(Scratch でできることを Ruby で書く)なので、コンソールのブロックを Ruby で書く形は足さない。 - 将来の案: Ruby タブに「コンソールモード」のような切りかえを置き、そのときだけ `puts` / `gets` がコンソールを使う。いまは作らない。 +### ステージの変数は書かない + +- 日本語モードは、**ターミナルで動かすような、1 つで動くプログラム**を想定する。上のとおりステージはコンソールになり、 + ステージという考え方はできるだけ薄くする。ステージの変数・リストや、ほかのスプライトの変数は使わない。 +- そのため、Ruby の `$a`(ステージの変数・リスト)は日本語にしない。Ruby → 日本語では、`$a` の書いてある位置に + 「ステージの変数($a)は日本語モードでは書けません。Ruby モードでは書けます。」の誤りを出す(ほかの日本語にできない Ruby と同じく、 + 日本語に切りかえず Ruby タブにとどまる)。日本語の変数は、いつもそのスプライトの変数(Ruby の `@a`)になる。 +- ステージの変数を使うプログラムは Ruby タブで書く。 + ### 要素の追加 - 日本語の `A に 値 を追加する` で書けるようにする。 diff --git a/docs/dncl/policy.md b/docs/dncl/policy.md index ed471125d1..cd66bd625e 100644 --- a/docs/dncl/policy.md +++ b/docs/dncl/policy.md @@ -110,7 +110,7 @@ | 種類 | 扱い | 例 | |---|---|---| | **意味が変わるもの、または問題文で約束されるもの** | **設定にする**。選んだ設定はプログラムの冒頭に説明文として書く | 添字の始まり(0 から / 1 から)、コメントの記号(`#` か `//` か。`//` をコメントにしたときは、`//` を整数の割り算に使えない)、論理演算子の書き方 | -| **意味が変わらない、見た目だけの揺れ** | **設定にしない**。読むとき(貼り付けなど)はどの書き方も受け付ける。書くとき(ブロックや Ruby から日本語にするとき)は 1 つにそろえる | 行番号の形(`(01)` / `(1)`)、罫線の文字(`|` `⎿` / `|` `└`)、`:` と `:`、全角と半角の演算子(`+` `%` `<`)、引き算の記号(`-` / `−` / `–`)、`,` の後の空白 | +| **意味が変わらない、見た目だけの揺れ** | **設定にしない**。読むとき(貼り付けなど)はどの書き方も受け付ける。書くとき(ブロックや Ruby から日本語にするとき)は 1 つにそろえる | 行番号の形(`(01)` / `(1)`)、罫線の文字(`|` `⎿` / `|` `└`)、`:` と `:`、全角と半角の演算子(`+` `%` `<`)、文字列とコメントの外の全角英字・全角数字(`x1` `12`)、引き算の記号(`-` / `−` / `–`)、`,` の後の空白 | - 書くときにそろえる 1 つは、上の「文法の決め方」で決める(新しい年度の問題に合わせる)。 - 専門的でない受験者には細かい設定が難しいので、**よくある設定のパターンを用意**する。 diff --git a/docs/smalruby-dncl-spec.ja.md b/docs/smalruby-dncl-spec.ja.md index 4c746183cf..91b323968b 100644 --- a/docs/smalruby-dncl-spec.ja.md +++ b/docs/smalruby-dncl-spec.ja.md @@ -151,8 +151,8 @@ c = "「く」がある" # → "「く」がある" のまま ### 識別子に使える文字 - ASCII 英字(`a-z`, `A-Z`)、アンダースコア(`_`)、2 文字目からは数字(`0-9`) -- ひらがな・カタカナ・漢字(CJK 拡張の漢字も)と、`々` `〆` `ヶ` `ー` など名前に使う文字。全角英字・ギリシャ文字など、ほかの文字体系の文字も使えます(`佐々木`、`α`) -- 全角英字は半角とは別の名前です(`x` と `x` は別の変数) +- ひらがな・カタカナ・漢字(CJK 拡張の漢字も)と、`々` `〆` `ヶ` `ー` など名前に使う文字。ギリシャ文字など、ほかの文字体系の文字も使えます(`佐々木`、`α`) +- 全角英字・全角数字は半角として読みます(`x1` は `x1` と同じ変数)。文字列とコメントの中は書いたとおりです - `☆` `①` のような記号は使えません 範囲の詳細は `docs/dncl/README.md` の「名前に使える文字」を参照してください。 @@ -382,6 +382,8 @@ DNCL 風の形の終端キーワードも読みます(読み込みだけ)。 | `を繰り返す` | `の間`(while ループ)、`ずつ増やしながら/減らしながら`(for ループ) | | `と定義する` | `関数〜`(関数定義) | +ブロックの種類に合わない終端キーワードで閉じると誤りです(`もし〜ならば` を `を繰り返す` で閉じるなど。「「もし〜ならば」は「を繰り返す」ではなく「を実行する」で閉じてください」)。`end` はどのブロックも閉じます。 + ## 7. 関数 ### 関数定義 @@ -652,6 +654,7 @@ Monaco エディタで以下の要素がハイライトされます: | 等価比較 | `==`(条件式の中の `=` も比較として読むが、`==` を推奨) | | 割り算 | `/`(小数になる割り算)、`÷`(商の整数値)。`//` も商の整数値として使える(コメントの記号を `//` にしたときを除く)。負の数の商は小さいほうへ丸める(`-7 ÷ 2` は `-4`) | | 全角の記号 | `%` `≦` `≧` `≠` `+` などの全角の演算子、`:` `,` も読む。書き出しは半角(余りだけ `%`) | +| 全角英字・全角数字 | 文字列とコメントの外では半角として読む(`x1` は `x1`、`12` は `12`) | | 入力 | `【外部からの入力】`(答えをそのまま使う)と `【整数を入力】`(整数にする) | | 要素の追加 | `A に 値 を追加する`(最後に足す。例題に無いので、方針の決め方の ⑤ で決めた) | | 戻り値 | `値 を返す`(例題に無いので、方針の決め方の ⑤ で決めた) | diff --git a/packages/ja-notation/README.md b/packages/ja-notation/README.md index 0fb6afb1f8..77866bffad 100644 --- a/packages/ja-notation/README.md +++ b/packages/ja-notation/README.md @@ -87,4 +87,5 @@ npm run test:unit --workspace=packages/ja-notation - データは **自作のプログラムだけ** にします。**大学入試センターの問題は使いません**(`docs/dncl/policy.md`)。 - AGPL のコードを写しません(MIT で出すため)。ファイルの先頭に `// SPDX-License-Identifier: MIT` を書きます(`test/boundary.test.js` が確かめます)。 - `src/` は `src/` の中だけを import します(拡張子 `.js` 付き)。 +- 構文木をたどる処理は再帰で書かず、ジェネレーターにして `src/run.js` の `run` で動かします(子をたどるところで `yield`)。数千項の式や深いブロックでも呼び出しのスタックが尽きないように(`test/deep.test.js`)。 - スモウルビーの表を通した記録(コーパスとゴールデン)は scratch-gui の `test/fixtures/ja-notation/` にあります。 diff --git a/packages/ja-notation/src/build-ruby.js b/packages/ja-notation/src/build-ruby.js index ffab3433bb..ca345b9037 100644 --- a/packages/ja-notation/src/build-ruby.js +++ b/packages/ja-notation/src/build-ruby.js @@ -11,6 +11,7 @@ // // Every Ruby statement has `jaLine` (the Japanese line it comes from) and may // have `comment`; a block has `closeLine` / `closeAfter` (see ./ja-ast.js). +import { each, run } from './run.js'; import { indexToRuby, settingsOf } from './settings.js'; /** @@ -71,42 +72,52 @@ const RUBY = { const SAME_OPERATORS = new Set(['+', '-', '*', '/', '%', '==', '!=', '<', '<=', '>', '>=']); /** - * Walk the statements of a JaAST (into blocks), calling `visit` on each. + * Walk the statements of a JaAST (into blocks), calling `visit` on each. The + * blocks are followed with a stack of their own, so blocks in blocks of any + * depth do not run out of the call stack (./run.js). * @param {Array} body - The statements. * @param {function(object): void} visit - Called with each statement. */ const walkStatements = (body, visit) => { - body.forEach((statement) => { - visit(statement); - if (statement.type === 'If') { - statement.branches.forEach((branch) => walkStatements(branch.body, visit)); - if (statement.elseBody) walkStatements(statement.elseBody, visit); - } else if (statement.body) { - walkStatements(statement.body, visit); - } - }); + const bodies = [body]; + while (bodies.length > 0) { + bodies.pop().forEach((statement) => { + visit(statement); + if (statement.type === 'If') { + statement.branches.forEach((branch) => bodies.push(branch.body)); + if (statement.elseBody) bodies.push(statement.elseBody); + } else if (statement.body) { + bodies.push(statement.body); + } + }); + } }; /** - * Walk the expressions under a node (a statement or an expression). - * @param {object} node - The node. + * Walk the expressions under a node (a statement or an expression), with a + * stack of its own: an expression of thousands of terms is as deep. + * @param {object} root - The node. * @param {function(object): void} visit - Called with each expression. */ -const walkExpressions = (node, visit) => { - if (!node || typeof node !== 'object') return; - if (Array.isArray(node)) { - node.forEach((child) => walkExpressions(child, visit)); - return; - } - if (node.type && !node.body && !node.branches) visit(node); - Object.entries(node).forEach(([key, value]) => { - if (key === 'body' || key === 'elseBody') return; - if (key === 'branches') { - value.forEach((branch) => walkExpressions(branch.condition, visit)); - return; +const walkExpressions = (root, visit) => { + const nodes = [root]; + while (nodes.length > 0) { + const node = nodes.pop(); + if (!node || typeof node !== 'object') continue; + if (Array.isArray(node)) { + node.forEach((child) => nodes.push(child)); + continue; } - if (value && typeof value === 'object') walkExpressions(value, visit); - }); + if (node.type && !node.body && !node.branches) visit(node); + Object.entries(node).forEach(([key, value]) => { + if (key === 'body' || key === 'elseBody') return; + if (key === 'branches') { + value.forEach((branch) => nodes.push(branch.condition)); + return; + } + if (value && typeof value === 'object') nodes.push(value); + }); + } }; /** @@ -210,18 +221,20 @@ class RubyBuilder { /** * The Ruby index of an element of an array (the arrays of Ruby start at 0). * @param {object} index - The JaAST index. + * @yields {object} The walks of its parts (./run.js). * @returns {object} The RubyAST index. */ - index(index) { - return this.expression(indexToRuby(index, this.settings)); + *index(index) { + return yield this.expression(indexToRuby(index, this.settings)); } /** * An expression. * @param {object} e - The JaAST expression. + * @yields {object} The walks of its parts (./run.js). * @returns {object} The RubyAST expression. */ - expression(e) { + *expression(e) { switch (e.type) { case 'Number': return { type: e.value.includes('.') ? 'FloatNode' : 'IntegerNode', text: e.value }; @@ -232,15 +245,15 @@ class RubyBuilder { case 'Name': return this.name(e); case 'Index': - return call(this.expression(e.target), '[]', [this.index(e.index)]); + return call(yield this.expression(e.target), '[]', [yield this.index(e.index)]); case 'ArrayLiteral': - return { type: 'ArrayNode', elements: e.elements.map((x) => this.expression(x)) }; + return { type: 'ArrayNode', elements: yield each(e.elements, (x) => this.expression(x)) }; case 'Call': - return this.call(e); + return yield this.call(e); case 'Unary': - return this.unary(e); + return yield this.unary(e); default: - return this.binary(e); + return yield this.binary(e); } } @@ -260,10 +273,11 @@ class RubyBuilder { /** * A call of a function. * @param {object} e - The Call. + * @yields {object} The walks of its arguments (./run.js). * @returns {object} The node. */ - call(e) { - const args = e.args.map((a) => this.expression(a)); + *call(e) { + const args = yield each(e.args, (a) => this.expression(a)); const entry = this.functions[e.name]; if (entry) { const built = entry.build(args, RUBY); @@ -283,14 +297,15 @@ class RubyBuilder { /** * A sign or `not`. A minus before a number is a negative number. * @param {object} e - The Unary. + * @yields {object} The walk of its operand (./run.js). * @returns {object} The node. */ - unary(e) { + *unary(e) { if (e.operator === '-' && e.operand.type === 'Number') { - const number = this.expression(e.operand); + const number = yield this.expression(e.operand); return { ...number, text: `-${number.text}` }; } - const operand = this.expression(e.operand); + const operand = yield this.expression(e.operand); if (e.operator === 'not') return call(operand, '!'); return call(operand, e.operator === '-' ? '-@' : '+@'); } @@ -298,16 +313,17 @@ class RubyBuilder { /** * A binary operator. * @param {object} e - The Binary. + * @yields {object} The walks of its sides (./run.js). * @returns {object} The node. */ - binary(e) { + *binary(e) { // A minus sign before a quotient is the sign of the quotient: // `-a ÷ 2` is `-(a ÷ 2)`, not `(-a) ÷ 2` (they differ for an odd `a`). if (e.operator === '÷' && e.left.type === 'Unary' && e.left.operator === '-') { - return call(this.binary({ ...e, left: e.left.operand }), '-@'); + return call(yield this.binary({ ...e, left: e.left.operand }), '-@'); } - const left = this.expression(e.left); - const right = this.expression(e.right); + const left = yield this.expression(e.left); + const right = yield this.expression(e.right); if (e.operator === 'and') return { type: 'AndNode', left, right }; if (e.operator === 'or') return { type: 'OrNode', left, right }; if (e.operator === '..' || e.operator === '...') { @@ -324,11 +340,12 @@ class RubyBuilder { * The node that gives a value to a target. * @param {object} target - The JaAST Name or Index. * @param {object} value - The RubyAST value. + * @yields {object} The walks of the target (./run.js). * @returns {object} The node. */ - write(target, value) { + *write(target, value) { if (target.type === 'Index') { - return call(this.expression(target.target), '[]=', [this.index(target.index), value]); + return call(yield this.expression(target.target), '[]=', [yield this.index(target.index), value]); } const name = this.variableName(target.name); return { type: name.startsWith('@') ? 'InstanceVariableWriteNode' : 'LocalVariableWriteNode', name, value }; @@ -337,12 +354,13 @@ class RubyBuilder { /** * Statements. * @param {Array} body - The JaAST statements. + * @yields {object} The walks of the statements (./run.js). * @returns {Array} The RubyAST statements. */ - statements(body) { + *statements(body) { const out = []; - body.forEach((s) => { - const built = this.statement(s); + for (const s of body) { + const built = yield this.statement(s); const list = Array.isArray(built) ? built : [built]; list.forEach((node) => { if (typeof node.jaLine === 'undefined') node.jaLine = s.line; @@ -352,45 +370,46 @@ class RubyBuilder { // The comment of the line goes after the last Ruby statement made from it. if (typeof s.comment !== 'undefined' && list.length > 0) list[list.length - 1].comment = s.comment; out.push(...list); - }); + } return out; } /** * One statement (some become more than one Ruby statement). * @param {object} s - The JaAST statement. + * @yields {object} The walks of its parts (./run.js). * @returns {object|Array} The RubyAST statement(s). */ - statement(s) { + *statement(s) { switch (s.type) { case 'Comment': return { type: 'CommentLine', text: s.text }; case 'Assign': - return this.write(s.target, this.expression(s.value)); + return yield this.write(s.target, yield this.expression(s.value)); case 'OpAssign': - return this.opWrite(s.target, s.operator, this.expression(s.value)); + return yield this.opWrite(s.target, s.operator, yield this.expression(s.value)); case 'MultiAssign': return { type: 'MultiWriteNode', - targets: s.targets.map((t) => this.write(t, null)), - values: s.values.map((v) => this.expression(v)), + targets: yield each(s.targets, (t) => this.write(t, null)), + values: yield each(s.values, (v) => this.expression(v)), }; case 'Input': - return this.input(s); + return yield this.input(s); case 'ExpressionStatement': - return this.expressionStatement(s); + return yield this.expressionStatement(s); case 'If': - return this.ifStatement(s); + return yield this.ifStatement(s); case 'While': - return this.whileStatement(s); + return yield this.whileStatement(s); case 'For': - return this.forStatement(s); + return yield this.forStatement(s); case 'FunctionDef': - return this.functionDef(s); + return yield this.functionDef(s); case 'Return': - return { type: 'ReturnNode', value: s.value ? this.expression(s.value) : null }; + return { type: 'ReturnNode', value: s.value ? yield this.expression(s.value) : null }; case 'Append': - return call(this.expression(s.target), 'push', [this.expression(s.value)]); + return call(yield this.expression(s.target), 'push', [yield this.expression(s.value)]); default: this.fail('syntax', `「${s.type}」は書けません`, s); return []; @@ -402,14 +421,15 @@ class RubyBuilder { * @param {object} target - The JaAST target. * @param {string} operator - The operator (`+`, ...). * @param {object} value - The RubyAST value. + * @yields {object} The walks of the target (./run.js). * @returns {object} The node. */ - opWrite(target, operator, value) { + *opWrite(target, operator, value) { if (target.type === 'Index') { return { type: 'IndexOperatorWriteNode', - receiver: this.expression(target.target), - index: this.index(target.index), + receiver: yield this.expression(target.target), + index: yield this.index(target.index), operator, value, }; @@ -422,16 +442,17 @@ class RubyBuilder { /** * An input (`x = 【整数を入力】`), as the table makes it. * @param {object} s - The Input. + * @yields {object} The walk of the target (./run.js). * @returns {Array} The statements. */ - input(s) { + *input(s) { const make = this.options.input; if (!make) { this.fail('syntax', '入力は使えません', s); return []; } const { before, value } = make(s.kind, RUBY); - return [...before, this.write(s.target, value)]; + return [...before, yield this.write(s.target, value)]; } /** @@ -439,9 +460,10 @@ class RubyBuilder { * program gives it a value or it starts with an uppercase letter, a call * when it is a function; else it is a command the Japanese does not have. * @param {object} s - The ExpressionStatement. + * @yields {object} The walk of the expression (./run.js). * @returns {object} The node. */ - expressionStatement(s) { + *expressionStatement(s) { const e = s.expression; if ( e.type === 'Name' && @@ -452,26 +474,27 @@ class RubyBuilder { ) { this.fail('unknown-command', `「${e.name}」は日本語モードでは書けない命令です`, e, { name: e.name }); } - return this.expression(e); + return yield this.expression(e); } /** * `もし`. * @param {object} s - The If. + * @yields {object} The walks of its parts (./run.js). * @returns {object} The IfNode. */ - ifStatement(s) { + *ifStatement(s) { let subsequent = null; if (s.elseBody) { - subsequent = { type: 'ElseNode', statements: this.statements(s.elseBody), jaLine: s.elseLine }; + subsequent = { type: 'ElseNode', statements: yield this.statements(s.elseBody), jaLine: s.elseLine }; if (typeof s.elseComment !== 'undefined') subsequent.comment = s.elseComment; } for (let i = s.branches.length - 1; i >= 0; i--) { const branch = s.branches[i]; const node = { type: 'IfNode', - predicate: this.expression(branch.condition), - statements: this.statements(branch.body), + predicate: yield this.expression(branch.condition), + statements: yield this.statements(branch.body), subsequent, jaLine: branch.line, }; @@ -484,14 +507,15 @@ class RubyBuilder { /** * `の間繰り返す`. A loop whose condition is a `not` is an `until` loop. * @param {object} s - The While. + * @yields {object} The walks of its parts (./run.js). * @returns {object} The node. */ - whileStatement(s) { + *whileStatement(s) { const negated = s.condition.type === 'Unary' && s.condition.operator === 'not'; return { type: negated ? 'UntilNode' : 'WhileNode', - predicate: this.expression(negated ? s.condition.operand : s.condition), - statements: this.statements(s.body), + predicate: yield this.expression(negated ? s.condition.operand : s.condition), + statements: yield this.statements(s.body), ...closing(s), }; } @@ -501,26 +525,27 @@ class RubyBuilder { * the start, and the loop runs while it has not passed the end, adding the * step at the end of each time. * @param {object} s - The For. + * @yields {object} The walks of its parts (./run.js). * @returns {Array} The assignment and the loop. */ - forStatement(s) { + *forStatement(s) { const variable = s.variable; const up = s.direction === 'up'; - let step = this.expression(s.step); + let step = yield this.expression(s.step); if (!up) { step = (step.type === 'IntegerNode' || step.type === 'FloatNode') && !step.text.startsWith('-') ? { ...step, text: `-${step.text}` } : call(step, '-@'); } - const init = this.write(variable, this.expression(s.from)); - const increment = this.opWrite(variable, '+', step); + const init = yield this.write(variable, yield this.expression(s.from)); + const increment = yield this.opWrite(variable, '+', step); increment.jaLine = s.closeLine; - const body = this.statements(s.body); + const body = yield this.statements(s.body); body.push(increment); const loop = { type: 'WhileNode', - predicate: call(this.read(variable.name), up ? '<=' : '>=', [this.expression(s.to)]), + predicate: call(this.read(variable.name), up ? '<=' : '>=', [yield this.expression(s.to)]), statements: body, ...closing(s), jaLine: s.line, @@ -533,12 +558,13 @@ class RubyBuilder { /** * `関数 名前(引数):`. Inside, the parameters are local variables. * @param {object} s - The FunctionDef. + * @yields {object} The walks of its statements (./run.js). * @returns {object} The DefNode. */ - functionDef(s) { + *functionDef(s) { const outer = this.params; this.params = new Set(s.params); - const statements = this.statements(s.body); + const statements = yield this.statements(s.body); this.params = outer; return { type: 'DefNode', @@ -566,7 +592,7 @@ class RubyBuilder { */ const buildRuby = (program, options = {}) => { const builder = new RubyBuilder(program, options); - const statements = builder.statements(program.body); + const statements = run(builder.statements(program.body)); if (builder.errors.length > 0) return { ruby: null, errors: builder.errors }; return { ruby: { type: 'ProgramNode', statements }, errors: [] }; }; diff --git a/packages/ja-notation/src/lexer.js b/packages/ja-notation/src/lexer.js index 7693908de9..4967ac97fb 100644 --- a/packages/ja-notation/src/lexer.js +++ b/packages/ja-notation/src/lexer.js @@ -4,18 +4,19 @@ /** * The first character of a name: an ASCII letter, `_`, or a letter of any - * script (`\p{L}`: kana, kanji with the CJK extensions, `々`, `〆`, `ヶ`, full-width - * letters, Greek, ...), a letter number (`\p{Nl}`: `〇`, `Ⅱ`), and the marks of - * the kana blocks (`゛`, `・`). Ruby takes any of them in a name, and Ruby → + * script (`\p{L}`: kana, kanji with the CJK extensions, `々`, `〆`, `ヶ`, + * Greek, ...), a letter number (`\p{Nl}`: `〇`, `Ⅱ`), and the marks of the + * kana blocks (`゛`, `・`). Ruby takes any of them in a name, and Ruby → * Japanese writes the name as it is (`@佐々木` → `佐々木`), so the Japanese reads - * back every name it writes (Issue #1714). A full-width letter is a name of its - * own (`x` is not `x`): Ruby has them as two variables. + * back every name it writes (Issue #1714). A full-width letter or digit out of + * a string and a comment is read as the half-width one before (`x1` is + * `x1`; see `halfWidthAlphanumerics`), as the auto-correct of the Ruby mode does. */ const NAME_HEAD_RE = /[A-Za-z_\p{L}\p{Nl}\u3040-\u309F\u30A0-\u30FF]/u; /** - * A character after the first one of a name: the same, the digits (`\p{Nd}`, - * full-width ones too) and the combining marks (`\p{M}`: `か` and `゙`). + * A character after the first one of a name: the same, the digits (`\p{Nd}`) + * and the combining marks (`\p{M}`: `か` and `゙`). */ const NAME_TAIL_RE = /[A-Za-z0-9_\p{L}\p{Nl}\p{Nd}\p{M}\u3040-\u309F\u30A0-\u30FF]/u; @@ -129,20 +130,33 @@ const stringEnd = (text, start) => { return -1; }; +/** A digit (the full-width ones are made half-width before, see `halfWidthAlphanumerics`). */ +const DIGIT_RE = /[0-9]/; + +/** The full-width letters and digits (`A`–`Z`, `a`–`z`, `0`–`9`). */ +const FULL_WIDTH_ALPHANUMERIC_RE = /[A-Za-z0-9]/g; + /** - * A digit, half-width or full-width (`12` is `12`): only how it looks differs, so a number - * written with full-width digits is read as the number (docs/dncl/policy.md: 見た目だけの揺れ). - * In a name (`x1`) a full-width digit stays a part of the name, as Ruby has it. + * The text with its full-width letters and digits made half-width (`x1 = 12` → + * `x1 = 12`). Only how they look differs (docs/dncl/policy.md: 見た目だけの揺れ), and + * the Ruby mode makes them half-width too (its auto-correct), so the Japanese reads + * them as the half-width ones: `x` is the name `x`, `if` would be `if`. Each is one + * UTF-16 unit, as the half-width one, so the columns stay the same. + * @param {string} text - The code (no string and no comment in it). + * @returns {string} The text. */ -const DIGIT_RE = /[0-90-9]/; +const halfWidthAlphanumerics = (text) => + text.replace(FULL_WIDTH_ALPHANUMERIC_RE, (ch) => String.fromCharCode(ch.charCodeAt(0) - 0xfee0)); /** - * The number as the Ruby writes it: the full-width digits and `.` as half-width ones. - * @param {string} text - The number as written. + * The number as the Ruby writes it: `.` as `.`. + * @param {string} text - The number as written (its digits are half-width). * @returns {string} The number. */ -const halfWidthNumber = (text) => - text.replace(/[0-9.]/g, (ch) => (ch === '.' ? '.' : String.fromCharCode(ch.charCodeAt(0) - 0xfee0))); +const halfWidthNumber = (text) => text.replace(/./g, '.'); + +/** The characters that start a string or a comment, which are not made half-width. */ +const STRING_OR_COMMENT_RE = /["'「“##]|\/\//g; /** * The end of a number that starts at `text[start]`, and whether it is written @@ -185,7 +199,10 @@ const longOperatorAt = (text, i) => LONG_OPERATORS.find((op) => text.startsWith( /** * Read one line of code into tokens. A token is `{type, text, column}` and, - * for some types, `value`: + * for some types, `value`. The full-width letters and digits out of the strings + * and the comments are read as the half-width ones (`halfWidthAlphanumerics`): + * the `text` of a name or a number has the half-width ones. The strings and the + * comments are kept as they are written. * * - `number` (`value`: the number as written, with half-width digits), `badNumber`, `string` (`text` * is the literal with its quotes), `name`, `op` (`value`: the operator, a @@ -198,15 +215,27 @@ const longOperatorAt = (text, i) => LONG_OPERATORS.find((op) => text.startsWith( * - `forbidden`: `@` or `$`, which Japanese names do not have. * - `unknown`: any other character, and a string that is not closed. * - * `column` is the 0-based index in `text` plus `offset`. - * @param {string} text - The code of the line (without its line number and indent). + * `column` is the 0-based index in `source` plus `offset`. + * @param {string} source - The code of the line (without its line number and indent). * @param {number} [offset] - What to add to each column. * @param {string} [commentMarker] - The comment mark of the settings (`#` or * `//`). With `//`, `//` starts a comment (and `#` still does) instead of * being the integer quotient. * @returns {Array} The tokens. */ -const tokenize = (text, offset = 0, commentMarker = '#') => { +const tokenize = (source, offset = 0, commentMarker = '#') => { + let text = source; + // The code is made half-width up to the next string or comment, each time the reading gets there. + let halfWidthTo = 0; + const makeHalfWidthFrom = (start) => { + STRING_OR_COMMENT_RE.lastIndex = start + 1; + const next = STRING_OR_COMMENT_RE.exec(text); + halfWidthTo = next ? next.index : text.length; + text = + text.substring(0, start) + + halfWidthAlphanumerics(text.substring(start, halfWidthTo)) + + text.substring(halfWidthTo); + }; const tokens = []; const push = (type, start, end, extra = {}) => tokens.push({ type, text: text.substring(start, end), column: start + offset, ...extra }); @@ -217,6 +246,10 @@ const tokenize = (text, offset = 0, commentMarker = '#') => { i++; continue; } + if (i >= halfWidthTo && !/["'「“##]/.test(ch) && !text.startsWith('//', i)) { + makeHalfWidthFrom(i); + continue; + } if (ch === '#' || ch === '#') { // A full-width `#` is read as `#` (only how it looks differs). tokens.push({ type: 'comment', text: `#${text.substring(i + 1)}`, column: i + offset }); diff --git a/packages/ja-notation/src/parse.js b/packages/ja-notation/src/parse.js index 5c80d80fc2..5164e2815c 100644 --- a/packages/ja-notation/src/parse.js +++ b/packages/ja-notation/src/parse.js @@ -10,6 +10,7 @@ // The two may be mixed: a block opened with `:` ends where the indentation // ends, one opened without `:` ends at its closing line (or a `⎿`). import { tokenize } from './lexer.js'; +import { run } from './run.js'; import { readSettings } from './settings.js'; /** The label of a numbered line: `(1)`, `(01)` and `(00-1)` (a line put before `(01)`). */ @@ -97,7 +98,8 @@ const STOP_NAMES = new Set([ * @param {object} [detail] - What a caller may need to say the error in its own * words: `name` (the token), `opener` / `direction` / `colon` / `word` (the * block of a block error), `block` (`stray-closer`, `closer-indent`, - * `stray-branch`) and `branch`. + * `closer-kind`, `stray-branch`), `expected` (the closing word of the block + * of a `closer-kind`) and `branch`. * @returns {object} The error. */ const makeError = (code, message, line, column = 0, detail = {}) => ({ code, message, line, column, ...detail }); @@ -257,7 +259,11 @@ const headerComments = (source) => { return comments; }; -/** Reads an expression from tokens (recursive descent, lowest precedence first). */ +/** + * Reads an expression from tokens (recursive descent, lowest precedence first). + * The readings are generators run by `run` (./run.js), so an expression nested + * thousands deep (parentheses, signs, calls) does not run out of the call stack. + */ class ExpressionReader { /** * @param {Array} tokens - The tokens of the expression. @@ -332,23 +338,25 @@ class ExpressionReader { /** * Read the whole tokens as one expression. + * @yields {object} The readings of its parts (./run.js). * @returns {object} The expression. */ - readAll() { - const expression = this.readExpression(); + *readAll() { + const expression = yield this.readExpression(); if (this.pos < this.tokens.length) this.fail(this.peek(), `「${this.peek().text}」が読めません`); return expression; } /** * An expression: a range (`1..10`) or anything with a weaker operator. + * @yields {object} The readings of its parts (./run.js). * @returns {object} The expression. */ - readExpression() { - const left = this.readOr(); + *readExpression() { + const left = yield this.readOr(); if (this.isOp('..') || this.isOp('...')) { const op = this.tokens[this.pos++]; - const right = this.readOr(); + const right = yield this.readOr(); return this.binary(op, left, right); } return left; @@ -384,26 +392,28 @@ class ExpressionReader { /** * `or` (`または`). + * @yields {object} The readings of its parts (./run.js). * @returns {object} The expression. */ - readOr() { - let left = this.readAnd(); + *readOr() { + let left = yield this.readAnd(); while (this.isNameIn(OR_WORDS)) { const op = this.tokens[this.pos++]; - left = { ...this.binary(op, left, this.readAnd()), operator: 'or' }; + left = { ...this.binary(op, left, yield this.readAnd()), operator: 'or' }; } return left; } /** * `and` (`かつ`). + * @yields {object} The readings of its parts (./run.js). * @returns {object} The expression. */ - readAnd() { - let left = this.readNegated(); + *readAnd() { + let left = yield this.readNegated(); while (this.isNameIn(AND_WORDS)) { const op = this.tokens[this.pos++]; - left = { ...this.binary(op, left, this.readNegated()), operator: 'and' }; + left = { ...this.binary(op, left, yield this.readNegated()), operator: 'and' }; } return left; } @@ -412,17 +422,18 @@ class ExpressionReader { * A `でない` after what it negates: it takes all before it up to the * nearest `かつ` / `または` (`a > 10 でない` is `not (a > 10)`), as `not` * does after it. + * @yields {object} The readings of its parts (./run.js). * @returns {object} The expression. */ - readNegated() { - let operand = this.readNot(); + *readNegated() { + let operand = yield this.readNot(); while (this.isName(POSTFIX_NOT)) { this.pos++; const negated = { type: 'Unary', operator: 'not', operand, line: operand.line, column: operand.column }; const next = this.peek(); if (next && next.type === 'op' && AFTER_NEGATION.has(next.value)) { this.pending = negated; - operand = this.readNot(); + operand = yield this.readNot(); } else { operand = negated; } @@ -432,79 +443,85 @@ class ExpressionReader { /** * `not`, weaker than a comparison and stronger than `and` / `or`. + * @yields {object} The readings of its parts (./run.js). * @returns {object} The expression. */ - readNot() { + *readNot() { if (!this.pending && this.isName('not')) { const op = this.tokens[this.pos++]; - const operand = this.readNot(); + const operand = yield this.readNot(); return { type: 'Unary', operator: 'not', operand, line: op.line, column: op.column }; } - return this.readComparison(); + return yield this.readComparison(); } /** * A comparison. + * @yields {object} The readings of its parts (./run.js). * @returns {object} The expression. */ - readComparison() { - const left = this.readAdditive(); + *readComparison() { + const left = yield this.readAdditive(); const t = this.peek(); if (t && t.type === 'op' && COMPARISONS.has(t.value)) { this.pos++; - return this.binary(t, left, this.readAdditive()); + return this.binary(t, left, yield this.readAdditive()); } return left; } /** * `+` / `-`. + * @yields {object} The readings of its parts (./run.js). * @returns {object} The expression. */ - readAdditive() { - let left = this.readMultiplicative(); + *readAdditive() { + let left = yield this.readMultiplicative(); while (this.isOp('+') || this.isOp('-')) { const op = this.tokens[this.pos++]; - left = this.binary(op, left, this.readMultiplicative()); + left = this.binary(op, left, yield this.readMultiplicative()); } return left; } /** * `*` / `/` / `÷` (and `//`) / `%`. + * @yields {object} The readings of its parts (./run.js). * @returns {object} The expression. */ - readMultiplicative() { - let left = this.readUnary(); + *readMultiplicative() { + let left = yield this.readUnary(); while (this.isOp('*') || this.isOp('/') || this.isOp('÷') || this.isOp('%')) { const op = this.tokens[this.pos++]; - left = this.binary(op, left, this.readUnary()); + left = this.binary(op, left, yield this.readUnary()); } return left; } /** * A sign before a value. + * @yields {object} The readings of its parts (./run.js). * @returns {object} The expression. */ - readUnary() { + *readUnary() { if (!this.pending && (this.isOp('-') || this.isOp('+'))) { const op = this.tokens[this.pos++]; - const operand = this.readUnary(); + const operand = yield this.readUnary(); return { type: 'Unary', operator: op.value, operand, line: op.line, column: op.column }; } - return this.readPostfix(); + return yield this.readPostfix(); } /** * A value and the indexes after it (`A[i]`, `A[i][j]`). + * @yields {object} The readings of its parts (./run.js). * @returns {object} The expression. */ - readPostfix() { - let value = this.readPrimary(); + *readPostfix() { + let value = yield this.readPrimary(); while (this.isOp('[')) { this.pos++; - const index = this.readExpression(); + const index = yield this.readExpression(); this.expectOp(']'); value = { type: 'Index', target: value, index, line: value.line, column: value.column }; } @@ -513,16 +530,17 @@ class ExpressionReader { /** * The arguments of a call, after its `(`, up to its `)`. + * @yields {object} The readings of its parts (./run.js). * @returns {Array} The arguments. */ - readArguments() { + *readArguments() { const args = []; if (this.isOp(')')) { this.pos++; return args; } for (;;) { - args.push(this.readExpression()); + args.push(yield this.readExpression()); if (this.isOp(',')) { this.pos++; continue; @@ -534,9 +552,10 @@ class ExpressionReader { /** * A value: a number, string, name, call, `( ... )` or `[ ... ]`. + * @yields {object} The readings of its parts (./run.js). * @returns {object} The expression. */ - readPrimary() { + *readPrimary() { if (this.pending) { const value = this.pending; this.pending = null; @@ -558,13 +577,13 @@ class ExpressionReader { if (t.text === '真' || t.text === '偽') return { type: 'Boolean', value: t.text === '真', ...at }; if (this.isOp('(')) { this.pos++; - return { type: 'Call', name: t.text, args: this.readArguments(), ...at }; + return { type: 'Call', name: t.text, args: yield this.readArguments(), ...at }; } return { type: 'Name', name: t.text, ...at }; } if (t.type === 'op' && t.value === '(') { this.pos++; - const inner = this.readExpression(); + const inner = yield this.readExpression(); this.expectOp(')'); return inner; } @@ -575,7 +594,7 @@ class ExpressionReader { this.pos++; } else { for (;;) { - elements.push(this.readExpression()); + elements.push(yield this.readExpression()); if (this.isOp(',')) { this.pos++; continue; @@ -596,7 +615,7 @@ class ExpressionReader { * @param {number} line - The line, for an error at the end. * @returns {object} The expression. */ -const readExpression = (tokens, line) => new ExpressionReader(tokens, line).readAll(); +const readExpression = (tokens, line) => run(new ExpressionReader(tokens, line).readAll()); /** * Split tokens at the commas outside brackets. @@ -914,6 +933,33 @@ const tokenError = (tokens) => { return null; }; +/** + * The closing word of a block in the DNCL-like form. + * @param {object} node - The block (`If`, `While`, `For`, `FunctionDef`). + * @returns {string} `を実行する`, `を繰り返す` or `と定義する`. + */ +const closerOf = (node) => { + if (node.type === 'If') return 'を実行する'; + if (node.type === 'FunctionDef') return 'と定義する'; + return 'を繰り返す'; +}; + +/** + * Where in the open blocks is the nearest one a closing word closes, when only + * blocks closed by a line of their own are inside it (so the word closes it and + * the blocks inside it were forgotten), or -1. + * @param {Array} stack - The open blocks (the innermost last). + * @param {string} word - The closing word. + * @returns {number} The index in the stack, or -1. + */ +const matchingCloser = (stack, word) => { + for (let i = stack.length - 1; i >= 0; i--) { + if (stack[i].mode !== 'closer') return -1; + if (closerOf(stack[i].node) === word) return i; + } + return -1; +}; + /** * Read the Japanese (the standard form, the DNCL-like form or both mixed) * into a JaAST. @@ -947,6 +993,16 @@ const parse = (source) => { ...(entry.node.type === 'For' ? { direction: entry.node.direction } : {}), colon: entry.colon, }); + // A block closed by a line of its own that was not closed. + const unclosed = (entry) => { + const word = closerOf(entry.node); + errors.push( + makeError('unclosed', `「${word}」で閉じていません`, entry.node.line, entry.node.column, { + ...openerOf(entry), + word, + }), + ); + }; const emptyBlock = (entry) => errors.push( makeError( @@ -1021,6 +1077,33 @@ const parse = (source) => { ); return; } + // The closing word must be the one of the block (Issue #1731, W3): a + // `もし` closed by `を繰り返す` is told. `end` closes any block, as in Ruby. + if (block.word !== END_WORD && block.word !== closerOf(top.node)) { + // A block forgotten to be closed inside the one the word closes (a + // `もし` in a loop closed by `を繰り返す`) is told as not closed. + const index = matchingCloser(stack, block.word); + if (index < 0) { + stack.pop(); + const expected = closerOf(top.node); + errors.push( + makeError( + 'syntax', + `「${block.word}」ではなく「${expected}」で閉じてください`, + entry.line, + column, + { ...openerOf(top), block: 'closer-kind', word: block.word, expected }, + ), + ); + return; + } + while (stack.length > index + 1) unclosed(stack.pop()); + const found = stack.pop(); + close(found, entry.line, entry.line); + found.node.closeOwnLine = true; + lastCodeLine = entry.line; + return; + } stack.pop(); close(top, entry.line, entry.line); top.node.closeOwnLine = true; @@ -1081,14 +1164,7 @@ const parse = (source) => { } else if (top.mode === 'indent') { emptyBlock(top); } else { - const word = - top.node.type === 'If' ? 'を実行する' : top.node.type === 'FunctionDef' ? 'と定義する' : 'を繰り返す'; - errors.push( - makeError('unclosed', `「${word}」で閉じていません`, top.node.line, top.node.column, { - ...openerOf(top), - word, - }), - ); + unclosed(top); } } return errors.length > 0 ? { program: null, errors } : { program, errors }; diff --git a/packages/ja-notation/src/read-ruby.js b/packages/ja-notation/src/read-ruby.js index 38140bf47b..a55ad68f8c 100644 --- a/packages/ja-notation/src/read-ruby.js +++ b/packages/ja-notation/src/read-ruby.js @@ -16,6 +16,7 @@ // the comments at the top (./settings.js), and what they change is changed // back on the tree: with indexes from 1, `@A[i - 1]` is read as `A[i]`. import { callOf, childrenOf, field, hasFlag, SourceIndex, statementsOf, typeOf, unwrap } from './prism-reader.js'; +import { each, run } from './run.js'; import { indexToJa, readSettings } from './settings.js'; /** The binary operators of Ruby that are the same in the Japanese. */ @@ -43,6 +44,20 @@ const WRITE_TYPES = new Set([ /** The statements that hold a body (they take the comments of their own lines). */ const BLOCK_TYPES = new Set(['If', 'While', 'For', 'FunctionDef']); +/** + * The nodes of a variable of the stage (`$a`, a variable or a list of the + * stage in Smalruby). The Japanese mode is written for a program that runs + * alone, as on a terminal, and has no stage: it does not write them (Issue #1706). + */ +const STAGE_VARIABLE_TYPES = new Set([ + 'GlobalVariableReadNode', + 'GlobalVariableWriteNode', + 'GlobalVariableOperatorWriteNode', + 'GlobalVariableOrWriteNode', + 'GlobalVariableAndWriteNode', + 'GlobalVariableTargetNode', +]); + /** The keywords that jump in a loop: the Japanese has no such statement. */ const JUMPS = { BreakNode: 'break', NextNode: 'next', RedoNode: 'redo', RetryNode: 'retry' }; @@ -61,6 +76,8 @@ class RubyReader { this.errors = []; // Where each error is (character index), to tell whether a RubyCode has one in it. this.errorStarts = []; + // The errors told already (code and place), so a node read twice is told once. + this.told = new Set(); // Ruby prism could not read has nodes of its recovery: only its first error is told. this.hasSyntaxErrors = (result.errors || []).length > 0; this.rawDepth = 0; @@ -187,7 +204,8 @@ class RubyReader { /** * Note an error at a node or a location. - * @param {string} code - `unknown-command`, `bad-number`, `post-test-loop`, `no-ja-form` or `ruby-syntax`. + * @param {string} code - `unknown-command`, `bad-number`, `post-test-loop`, `no-ja-form`, + * `stage-variable` or `ruby-syntax`. * @param {string} message - The message. * @param {object} at - The node or location. * @param {object} [detail] - More (`name`). @@ -195,11 +213,24 @@ class RubyReader { fail(code, message, at, detail = {}) { const loc = typeof at.startOffset === 'number' ? at : field(at, 'location'); const start = this.index.charAt(loc.startOffset); + const key = `${code}:${start}`; + if (this.told.has(key)) return; + this.told.add(key); const { line, column } = this.index.place(start); this.errors.push({ code, message, line, column, ...detail }); this.errorStarts.push(start); } + /** + * Note a variable of the stage (`$a`; Issue #1706), at its name. + * @param {object} node - A node of `STAGE_VARIABLE_TYPES`. + */ + failStageVariable(node) { + const name = field(node, 'name'); + const at = field(node, 'nameLoc') || node; + this.fail('stage-variable', `ステージの変数(${name})は日本語では書けません`, at, { name }); + } + /** * Note Ruby with no Japanese form that no other error tells (`case`, a * hash, a symbol, a modifier `unless`; Issue #1740): kept as it is @@ -234,9 +265,10 @@ class RubyReader { * read, so the calls and numbers in it are told, and when none of them * is, the Ruby itself is (`no-ja-form`). * @param {object} node - The node. + * @yields {object} The readings of its parts (./run.js). * @returns {object} The RubyCode. */ - raw(node) { + *raw(node) { const { start, end } = this.span(node); this.useCommentsIn(start, end); // An error already told at this Ruby (a call the Japanese does not @@ -244,7 +276,7 @@ class RubyReader { const told = this.errorStarts.some((at) => at >= start && at < end); // The Ruby inside it is a part of it: only the outermost is told. this.rawDepth += 1; - this.scan(node); + yield this.scan(node); this.rawDepth -= 1; if (!told && this.rawDepth === 0 && !this.hasSyntaxErrors) this.failNoForm(node); const { line, column } = this.index.place(start); @@ -254,14 +286,15 @@ class RubyReader { /** * Read the parts of a node only for their errors. * @param {object} node - The node. + * @yields {object} The readings of its parts (./run.js). */ - scan(node) { - childrenOf(node).forEach((child) => { + *scan(node) { + for (const child of childrenOf(node)) { const type = typeOf(child); - if (type === 'StatementsNode') this.statements(statementsOf(child), null); - else if (type === 'BlockNode') this.scan(child); - else if (type) this.expression(child); - }); + if (type === 'StatementsNode') yield this.statements(statementsOf(child), null); + else if (type === 'BlockNode') yield this.scan(child); + else if (type) yield this.expression(child); + } } /** @@ -281,23 +314,25 @@ class RubyReader { * @param {Array} nodes - The prism statements. * @param {?{from: number, to: number}} range - The lines the body has (null * when the statements are only read for their errors). + * @yields {object} The readings of its parts (./run.js). * @returns {Array} The JaAST statements. */ - statements(nodes, range) { + *statements(nodes, range) { const out = []; for (let i = 0; i < nodes.length; i++) { - const pair = this.countingLoop(nodes[i], nodes[i + 1]) || this.input(nodes[i], nodes[i + 1]); + const pair = + (yield this.countingLoop(nodes[i], nodes[i + 1])) || (yield this.input(nodes[i], nodes[i + 1])); if (pair) { out.push(pair); i++; continue; } - const alone = this.inputAlone(nodes[i]); + const alone = yield this.inputAlone(nodes[i]); if (alone) { out.push(alone); continue; } - const statement = this.statement(nodes[i]); + const statement = yield this.statement(nodes[i]); if (!BLOCK_TYPES.has(statement.type)) { const comment = this.takeComments(statement.line, statement.endLine); if (comment) statement.comment = comment; @@ -316,54 +351,61 @@ class RubyReader { /** * One statement. * @param {object} node - The prism statement. + * @yields {object} The readings of its parts (./run.js). * @returns {object} The JaAST statement. */ - statement(node) { + *statement(node) { const type = typeOf(node); switch (type) { case 'IfNode': - return this.ifStatement(node); + return yield this.ifStatement(node); case 'WhileNode': case 'UntilNode': - return this.loopStatement(node, type === 'UntilNode'); + return yield this.loopStatement(node, type === 'UntilNode'); case 'DefNode': - return this.functionDef(node); + return yield this.functionDef(node); case 'ReturnNode': { const args = field(node, 'arguments'); const values = args ? field(args, 'arguments') : []; - if (values.length > 1) return this.rawStatement(node); - return this.at(node, { type: 'Return', value: values[0] ? this.expression(values[0]) : null }); + if (values.length > 1) return yield this.rawStatement(node); + return this.at(node, { type: 'Return', value: values[0] ? yield this.expression(values[0]) : null }); } case 'MultiWriteNode': - return this.multiAssign(node); + return yield this.multiAssign(node); + case 'GlobalVariableWriteNode': + case 'GlobalVariableOperatorWriteNode': + case 'GlobalVariableOrWriteNode': + case 'GlobalVariableAndWriteNode': + this.failStageVariable(node); + return yield this.rawStatement(node); case 'InstanceVariableOperatorWriteNode': case 'LocalVariableOperatorWriteNode': case 'ConstantOperatorWriteNode': case 'IndexOperatorWriteNode': - return this.opAssign(node); + return yield this.opAssign(node); default: break; } if (VARIABLE_WRITES[type]) { const target = this.target(node); - if (!target) return this.rawStatement(node); - return this.at(node, { type: 'Assign', target, value: this.expression(field(node, 'value')) }); + if (!target) return yield this.rawStatement(node); + return this.at(node, { type: 'Assign', target, value: yield this.expression(field(node, 'value')) }); } if (JUMPS[type]) { this.fail('unknown-command', `「${JUMPS[type]}」は日本語では書けない命令です`, node, { name: JUMPS[type], }); - return this.rawStatement(node); + return yield this.rawStatement(node); } const call = callOf(node); if (call && call.name === '[]=' && call.receiver && call.args && call.args.length === 2 && !call.block) { - const target = this.indexOf(call.receiver, call.args[0]); - return this.at(node, { type: 'Assign', target, value: this.expression(call.args[1]) }); + const target = yield this.indexOf(call.receiver, call.args[0]); + return this.at(node, { type: 'Assign', target, value: yield this.expression(call.args[1]) }); } - if (call && call.block) return this.blockCall(call); - const append = this.append(node, call); + if (call && call.block) return yield this.blockCall(call); + const append = yield this.append(node, call); if (append) return append; - const expression = this.expression(node); + const expression = yield this.expression(node); if (expression.type === 'RubyCode') return this.at(node, { ...expression, type: 'RubyCode' }); return this.at(node, { type: 'ExpressionStatement', expression }); } @@ -372,23 +414,30 @@ class RubyReader { * `A.push(値)` / `A << 値` of an array variable: `A に 値 を追加する` (Issue #1759). * @param {object} node - The statement. * @param {?object} call - The call of the statement. + * @yields {object} The readings of its parts (./run.js). * @returns {?object} The Append statement, or null when it is not one. */ - append(node, call) { + *append(node, call) { if (!call || (call.name !== 'push' && call.name !== '<<') || !call.receiver || call.block) return null; if (!call.args || call.args.length !== 1) return null; - const target = this.expression(call.receiver); + // `$a.push(1)`: the list of the stage is told, not the call (Issue #1706). + if (STAGE_VARIABLE_TYPES.has(typeOf(call.receiver))) { + this.failStageVariable(call.receiver); + return yield this.rawStatement(node); + } + const target = yield this.expression(call.receiver); if (target.type !== 'Name') return null; - return this.at(node, { type: 'Append', target, value: this.expression(call.args[0]) }); + return this.at(node, { type: 'Append', target, value: yield this.expression(call.args[0]) }); } /** * A statement with no Japanese form. * @param {object} node - The node. + * @yields {object} The readings of its parts (./run.js). * @returns {object} The RubyCode statement. */ - rawStatement(node) { - return this.at(node, this.raw(node)); + *rawStatement(node) { + return this.at(node, yield this.raw(node)); } /** @@ -413,74 +462,96 @@ class RubyReader { * An element of an array. * @param {object} receiver - The array. * @param {object} index - The index. + * @yields {object} The readings of its parts (./run.js). * @returns {object} The Index. */ - indexOf(receiver, index) { - const target = this.expression(receiver); - const ja = indexToJa(this.expression(index), this.settings); + *indexOf(receiver, index) { + const target = yield this.expression(receiver); + const ja = indexToJa(yield this.expression(index), this.settings); return { type: 'Index', target, index: ja, line: target.line, column: target.column }; } + /** + * A target of a multiple assignment: a variable or an element of an array, + * or null when it has no Japanese form. + * @param {object} left - The prism target. + * @yields {object} The reading of an index (./run.js). + * @returns {?object} The Name or the Index. + */ + *multiTarget(left) { + const type = typeOf(left); + if (/^(?:InstanceVariable|LocalVariable|Constant)TargetNode$/.test(type)) return this.target(left); + if (STAGE_VARIABLE_TYPES.has(type)) { + this.failStageVariable(left); + return null; + } + // `@a[0], @a[1] = @a[1], @a[0]`: an element of an array (Issue #2237). + if (type === 'IndexTargetNode') { + const args = field(left, 'arguments'); + const indexes = args ? field(args, 'arguments') : []; + if (indexes.length !== 1 || field(left, 'block')) return null; + return yield this.indexOf(field(left, 'receiver'), indexes[0]); + } + return null; + } + /** * `a, b = 1, 2`. * @param {object} node - The MultiWriteNode. + * @yields {object} The readings of its parts (./run.js). * @returns {object} The statement. */ - multiAssign(node) { + *multiAssign(node) { const lefts = field(node, 'lefts'); const value = field(node, 'value'); - const targets = lefts.map((left) => { - const type = typeOf(left); - if (/^(?:InstanceVariable|LocalVariable|Constant)TargetNode$/.test(type)) return this.target(left); - // `@a[0], @a[1] = @a[1], @a[0]`: an element of an array (Issue #2237). - if (type === 'IndexTargetNode') { - const args = field(left, 'arguments'); - const indexes = args ? field(args, 'arguments') : []; - if (indexes.length !== 1 || field(left, 'block')) return null; - return this.indexOf(field(left, 'receiver'), indexes[0]); - } - return null; - }); + const targets = yield each(lefts, (left) => this.multiTarget(left)); const plain = field(node, 'rest') === null && field(node, 'rights').length === 0 && targets.every(Boolean) && typeOf(value) === 'ArrayNode' && field(value, 'openingLoc') === null; - if (!plain) return this.rawStatement(node); - const values = field(value, 'elements').map((element) => this.expression(element)); + if (!plain) return yield this.rawStatement(node); + const values = yield each(field(value, 'elements'), (element) => this.expression(element)); return this.at(node, { type: 'MultiAssign', targets, values }); } /** * `i += 1`, `A[i] += 1`. * @param {object} node - The operator write. + * @yields {object} The readings of its parts (./run.js). * @returns {object} The statement. */ - opAssign(node) { + *opAssign(node) { const operator = field(node, 'binaryOperator'); - if (!OP_ASSIGN_OPERATORS.has(operator)) return this.rawStatement(node); + if (!OP_ASSIGN_OPERATORS.has(operator)) return yield this.rawStatement(node); let target; if (typeOf(node) === 'IndexOperatorWriteNode') { const args = field(node, 'arguments'); const indexes = args ? field(args, 'arguments') : []; - if (indexes.length !== 1 || field(node, 'block')) return this.rawStatement(node); - target = this.indexOf(field(node, 'receiver'), indexes[0]); + if (indexes.length !== 1 || field(node, 'block')) return yield this.rawStatement(node); + target = yield this.indexOf(field(node, 'receiver'), indexes[0]); } else { target = this.target(node); } - if (!target) return this.rawStatement(node); - return this.at(node, { type: 'OpAssign', target, operator, value: this.expression(field(node, 'value')) }); + if (!target) return yield this.rawStatement(node); + return this.at(node, { + type: 'OpAssign', + target, + operator, + value: yield this.expression(field(node, 'value')), + }); } /** * `if`, `elsif`, `else`. A modifier (`x if a`) has no Japanese form. * @param {object} node - The IfNode. + * @yields {object} The readings of its parts (./run.js). * @returns {object} The If. */ - ifStatement(node) { + *ifStatement(node) { const end = field(node, 'endKeywordLoc'); - if (!end) return this.rawStatement(node); + if (!end) return yield this.rawStatement(node); const closeLine = this.lineOf(end); const line = this.lineOf(node); const parts = []; @@ -493,22 +564,26 @@ class RubyReader { const elseLine = elseNode ? this.lineOf(field(elseNode, 'elseKeywordLoc')) : null; const statement = { type: 'If', branches: [], elseBody: null, elseLine, line, column: 0, endLine: closeLine }; const ends = [...parts.slice(1).map((part) => part.line), elseLine || closeLine, closeLine]; - parts.forEach((part, i) => { + for (let i = 0; i < parts.length; i++) { + const part = parts[i]; const predicate = field(part.node, 'predicate'); - const condition = this.expression(predicate); + const condition = yield this.expression(predicate); const comment = this.takeComments(part.line, this.endLineOf(predicate)); const range = { from: part.line + 1, to: ends[i] - 1 }; - const branch = { condition, body: this.statements(statementsOf(field(part.node, 'statements')), range) }; + const branch = { + condition, + body: yield this.statements(statementsOf(field(part.node, 'statements')), range), + }; branch.line = part.line; if (i === 0 && comment) statement.comment = comment; if (i > 0 && comment) branch.comment = comment; statement.branches.push(branch); - }); + } if (elseNode) { const elseComment = this.takeComments(elseLine); if (elseComment) statement.elseComment = elseComment; const range = { from: elseLine + 1, to: closeLine - 1 }; - statement.elseBody = this.statements(statementsOf(field(elseNode, 'statements')), range); + statement.elseBody = yield this.statements(statementsOf(field(elseNode, 'statements')), range); } statement.closeLine = closeLine; const closeComment = this.takeComments(closeLine); @@ -522,16 +597,17 @@ class RubyReader { * `begin ... end while c` (the post-test loop) has no Japanese form. * @param {object} node - The WhileNode or UntilNode. * @param {boolean} until - Whether it is an `until`. + * @yields {object} The readings of its parts (./run.js). * @returns {object} The While (or the RubyCode). */ - loopStatement(node, until) { + *loopStatement(node, until) { const predicate = field(node, 'predicate'); if (hasFlag(node, 'isBeginModifier')) { const message = '後判定の繰り返しは日本語では書けません'; const body = statementsOf(field(node, 'statements')); this.fail('post-test-loop', message, body[0] || node); this.fail('post-test-loop', message, field(node, 'keywordLoc')); - return this.rawStatement(node); + return yield this.rawStatement(node); } const negate = (condition) => ({ type: 'Unary', @@ -545,11 +621,11 @@ class RubyReader { const body = statementsOf(field(node, 'statements')); const call = body.length === 1 ? callOf(body[0]) : null; const isWait = call && call.name === 'wait' && !call.receiver && call.isVariableCall; - if (!until || !isWait) return this.rawStatement(node); + if (!until || !isWait) return yield this.rawStatement(node); const line = this.lineOf(node); return { type: 'While', - condition: negate(this.expression(predicate)), + condition: negate(yield this.expression(predicate)), body: [], line, column: 0, @@ -558,13 +634,13 @@ class RubyReader { } const line = this.lineOf(field(node, 'keywordLoc')); const closeLine = this.lineOf(closing); - const condition = this.expression(predicate); + const condition = yield this.expression(predicate); const comment = this.takeComments(line, this.endLineOf(predicate)); const range = { from: line + 1, to: closeLine - 1 }; const statement = { type: 'While', condition: until ? negate(condition) : condition, - body: this.statements(statementsOf(field(node, 'statements')), range), + body: yield this.statements(statementsOf(field(node, 'statements')), range), line, column: 0, endLine: closeLine, @@ -579,9 +655,10 @@ class RubyReader { /** * `def name(a, b)`. Only plain parameters have a Japanese form. * @param {object} node - The DefNode. + * @yields {object} The readings of its parts (./run.js). * @returns {object} The FunctionDef. */ - functionDef(node) { + *functionDef(node) { const parameters = field(node, 'parameters'); const body = field(node, 'body'); const end = field(node, 'endKeywordLoc'); @@ -592,9 +669,9 @@ class RubyReader { (parameters === null || ['optionals', 'posts', 'keywords'].every((key) => field(parameters, key).length === 0)) && (parameters === null || ['rest', 'keywordRest', 'block'].every((key) => field(parameters, key) === null)); - if (!plain) return this.rawStatement(node); + if (!plain) return yield this.rawStatement(node); const params = parameters ? field(parameters, 'requireds').map((p) => field(p, 'name')) : []; - if (params.some((p) => typeof p !== 'string')) return this.rawStatement(node); + if (params.some((p) => typeof p !== 'string')) return yield this.rawStatement(node); const line = this.lineOf(node); const closeLine = this.lineOf(end); const comment = this.takeComments(line); @@ -602,7 +679,7 @@ class RubyReader { type: 'FunctionDef', name: field(node, 'name'), params, - body: this.statements(statementsOf(body), { from: line + 1, to: closeLine - 1 }), + body: yield this.statements(statementsOf(body), { from: line + 1, to: closeLine - 1 }), line, column: 0, endLine: closeLine, @@ -618,21 +695,23 @@ class RubyReader { * A call with a block. `n.times do`, `(a..b).step(s) do |i|` and * `a.step(b, -s) do |i|` are counting loops; any other has no Japanese form. * @param {object} call - The call. + * @yields {object} The readings of its parts (./run.js). * @returns {object} The statement. */ - blockCall(call) { - const loop = this.iteratorLoop(call); + *blockCall(call) { + const loop = yield this.iteratorLoop(call); if (loop) return loop; if (!this.isKnownCall(call)) this.failCall(call); - return this.rawStatement(call.node); + return yield this.rawStatement(call.node); } /** * The counting loop of a Ruby iterator, or null. * @param {object} call - The call. + * @yields {object} The readings of its parts (./run.js). * @returns {?object} The For. */ - iteratorLoop(call) { + *iteratorLoop(call) { const block = call.block; if (typeOf(block) !== 'BlockNode' || !call.receiver || !call.args) return null; const body = field(block, 'body'); @@ -649,20 +728,20 @@ class RubyReader { if (call.name === 'times' && call.args.length === 0 && !blockParams) { variable = '_'; from = one; - to = this.expression(call.receiver); + to = yield this.expression(call.receiver); step = one; } else if (call.name === 'step' && names.length === 1 && !params.rest) { const range = unwrap(call.receiver); if (call.args.length === 1 && typeOf(range) === 'RangeNode' && !hasFlag(range, 'isExcludeEnd')) { if (!field(range, 'left') || !field(range, 'right')) return null; - from = this.expression(field(range, 'left')); - to = this.expression(field(range, 'right')); - step = this.expression(call.args[0]); + from = yield this.expression(field(range, 'left')); + to = yield this.expression(field(range, 'right')); + step = yield this.expression(call.args[0]); } else if (call.args.length === 2) { - const negated = this.negatedOperand(call.args[1]); + const negated = yield this.negatedOperand(call.args[1]); if (!negated) return null; - from = this.expression(call.receiver); - to = this.expression(call.args[0]); + from = yield this.expression(call.receiver); + to = yield this.expression(call.args[0]); step = negated; direction = 'down'; } else { @@ -682,7 +761,7 @@ class RubyReader { to, step, direction, - body: this.statements(statementsOf(body), { from: line + 1, to: closeLine - 1 }), + body: yield this.statements(statementsOf(body), { from: line + 1, to: closeLine - 1 }), line, column: 0, endLine: closeLine, @@ -697,16 +776,17 @@ class RubyReader { /** * The JaAST of `X` when the node is `-X` (or a negative number), else null. * @param {object} node - The node. + * @yields {object} The readings of its parts (./run.js). * @returns {?object} The expression of `X`. */ - negatedOperand(node) { + *negatedOperand(node) { const type = typeOf(node); if ((type === 'IntegerNode' || type === 'FloatNode') && this.text(node).startsWith('-')) { - const number = this.expression(node); + const number = yield this.expression(node); return number.operand; } const call = callOf(node); - if (call && call.name === '-@' && call.receiver) return this.expression(call.receiver); + if (call && call.name === '-@' && call.receiver) return yield this.expression(call.receiver); return null; } @@ -732,13 +812,14 @@ class RubyReader { * @param {function(object, object): ?string} kindOf - The kind of input the value is. * @param {number} line - The first line of the Input. * @param {number} endLine - The last line of the Input. + * @yields {object} The readings of its parts (./run.js). * @returns {?object} The Input, or null. */ - inputOf(assignment, kindOf, line, endLine) { + *inputOf(assignment, kindOf, line, endLine) { let target = assignment && assignment.target; const kind = target && kindOf(assignment.value, this.helpers()); if (!kind) return null; - if (target.write) target = this.indexOf(target.write.receiver, target.write.args[0]); + if (target.write) target = yield this.indexOf(target.write.receiver, target.write.args[0]); const statement = { type: 'Input', target, kind, line, column: 0, endLine }; const comment = this.takeComments(line, endLine); if (comment) statement.comment = comment; @@ -750,14 +831,15 @@ class RubyReader { * its answer on the next line, as the caller tells them (`options.input`). * @param {object} node - The statement. * @param {object} [next] - The statement after it. + * @yields {object} The readings of its parts (./run.js). * @returns {?object} The Input, or null. */ - input(node, next) { + *input(node, next) { const input = this.options.input; const call = callOf(node); if (!input || !input.isQuestion || !next || !call || !input.isQuestion(call, this.helpers())) return null; if (this.lineOf(next) !== this.endLineOf(node) + 1) return null; - return this.inputOf( + return yield this.inputOf( this.assignmentOf(next), (value, helpers) => input.kindOf(value, helpers), this.lineOf(node), @@ -769,12 +851,13 @@ class RubyReader { * `x = 【外部からの入力】` on one line: an assignment of a value that is the * input by itself, as the caller tells (`options.input.kindOfAlone`). * @param {object} node - The statement. + * @yields {object} The readings of its parts (./run.js). * @returns {?object} The Input, or null. */ - inputAlone(node) { + *inputAlone(node) { const input = this.options.input; if (!input || !input.kindOfAlone) return null; - return this.inputOf( + return yield this.inputOf( this.assignmentOf(node), (value, helpers) => input.kindOfAlone(value, helpers), this.lineOf(node), @@ -790,9 +873,10 @@ class RubyReader { * or the step. Any other `while` stays a `の間` loop. * @param {object} init - The statement. * @param {object} [loop] - The statement after it. + * @yields {object} The readings of its parts (./run.js). * @returns {?object} The For, or null. */ - countingLoop(init, loop) { + *countingLoop(init, loop) { if (!loop || typeOf(init) !== 'InstanceVariableWriteNode' || typeOf(loop) !== 'WhileNode') return null; if (hasFlag(loop, 'isBeginModifier') || !field(loop, 'closingLoc')) return null; if (this.lineOf(loop) !== this.endLineOf(init) + 1) return null; @@ -821,11 +905,11 @@ class RubyReader { const statement = { type: 'For', variable: target, - from: this.expression(field(init, 'value')), - to: this.expression(condition.args[0]), + from: yield this.expression(field(init, 'value')), + to: yield this.expression(condition.args[0]), step: increment.negativeNumber - ? this.expression(increment.step).operand - : this.expression(increment.step), + ? (yield this.expression(increment.step)).operand + : yield this.expression(increment.step), direction: increment.up ? 'up' : 'down', body: null, line, @@ -835,7 +919,7 @@ class RubyReader { }; const comment = this.takeComments(line, this.endLineOf(field(loop, 'predicate'))); if (comment) statement.comment = comment; - statement.body = this.statements(body.slice(0, -1), { from: whileLine + 1, to: closeLine - 1 }); + statement.body = yield this.statements(body.slice(0, -1), { from: whileLine + 1, to: closeLine - 1 }); const closeComment = this.takeComments(this.lineOf(last), closeLine); if (closeComment) statement.closeComment = closeComment; return statement; @@ -966,7 +1050,7 @@ class RubyReader { helpers() { if (!this.helperSet) { this.helperSet = { - expression: (node) => this.expression(node), + expression: (node) => run(this.expression(node)), call: (name, args) => ({ type: 'Call', name, args, line: 0, column: 0 }), callOf, typeOf, @@ -991,9 +1075,10 @@ class RubyReader { /** * An expression. * @param {object} node - The prism node. + * @yields {object} The readings of its parts (./run.js). * @returns {object} The JaAST expression. */ - expression(node) { + *expression(node) { const where = { line: this.lineOf(node), column: this.index.place(this.span(node).start).column }; const type = typeOf(node); switch (type) { @@ -1008,7 +1093,7 @@ class RubyReader { } case 'StringNode': { const text = this.text(node); - if (!/^["']/.test(text)) return this.raw(node); + if (!/^["']/.test(text)) return yield this.raw(node); return { type: 'String', raw: text, ...where }; } case 'TrueNode': @@ -1016,41 +1101,46 @@ class RubyReader { return { type: 'Boolean', value: type === 'TrueNode', ...where }; case 'InstanceVariableReadNode': { const name = this.options.variable ? this.options.variable(field(node, 'name')) : field(node, 'name'); - return name ? { type: 'Name', name, ...where } : this.raw(node); + return name ? { type: 'Name', name, ...where } : yield this.raw(node); } case 'LocalVariableReadNode': return { type: 'Name', name: field(node, 'name'), ...where }; + case 'GlobalVariableReadNode': + this.failStageVariable(node); + return yield this.raw(node); case 'ConstantReadNode': // With no table, a name in capitals is a constant (see `target`). - return this.options.variable ? this.raw(node) : { type: 'Name', name: field(node, 'name'), ...where }; + return this.options.variable + ? yield this.raw(node) + : { type: 'Name', name: field(node, 'name'), ...where }; case 'ArrayNode': - if (field(node, 'openingLoc') === null || !/^\[/.test(this.text(node))) return this.raw(node); + if (field(node, 'openingLoc') === null || !/^\[/.test(this.text(node))) return yield this.raw(node); return { type: 'ArrayLiteral', - elements: field(node, 'elements').map((element) => this.expression(element)), + elements: yield each(field(node, 'elements'), (element) => this.expression(element)), ...where, }; case 'ParenthesesNode': { const body = statementsOf(field(node, 'body')); - if (body.length !== 1) return this.raw(node); - return this.expression(body[0]); + if (body.length !== 1) return yield this.raw(node); + return yield this.expression(body[0]); } case 'RangeNode': - if (!field(node, 'left') || !field(node, 'right')) return this.raw(node); - return this.binary( + if (!field(node, 'left') || !field(node, 'right')) return yield this.raw(node); + return yield this.binary( hasFlag(node, 'isExcludeEnd') ? '...' : '..', field(node, 'left'), field(node, 'right'), where, ); case 'AndNode': - return this.binary('and', field(node, 'left'), field(node, 'right'), where); + return yield this.binary('and', field(node, 'left'), field(node, 'right'), where); case 'OrNode': - return this.binary('or', field(node, 'left'), field(node, 'right'), where); + return yield this.binary('or', field(node, 'left'), field(node, 'right'), where); case 'CallNode': - return this.callExpression(node, where); + return yield this.callExpression(node, where); default: - return this.raw(node); + return yield this.raw(node); } } @@ -1081,9 +1171,10 @@ class RubyReader { * @param {object} left - The prism left side. * @param {object} right - The prism right side. * @param {object} where - The place. + * @yields {object} The readings of its parts (./run.js). * @returns {object} The Binary. */ - binary(operator, left, right, where) { + *binary(operator, left, right, where) { // A long chain (`a && b && ... && z`, thousands of terms) is as deep as // its terms on the left: the left side is followed in a loop, not by // recursion, so it does not run out of the stack. @@ -1095,14 +1186,14 @@ class RubyReader { chain.push(step); base = step.left; } - let result = this.expression(base); + let result = yield this.expression(base); for (let i = chain.length - 1; i >= 0; i--) { const step = chain[i]; result = { type: 'Binary', operator: step.operator, left: result, - right: this.expression(step.right), + right: yield this.expression(step.right), ...step.where, }; } @@ -1155,33 +1246,41 @@ class RubyReader { * table, a function of the program, or one the Japanese does not have. * @param {object} node - The CallNode. * @param {object} where - The place. + * @yields {object} The readings of its parts (./run.js). * @returns {object} The expression. */ - callExpression(node, where) { + *callExpression(node, where) { const call = callOf(node); const { name, receiver, args } = call; if (call.block || !args) { if (!this.isKnownCall(call)) this.failCall(call); - return this.raw(node); + return yield this.raw(node); } if (receiver && args.length === 0) { if (name === '!') { - const negated = this.unary('not', this.expression(receiver), where); + const negated = this.unary('not', yield this.expression(receiver), where); if (typeOf(receiver) === 'ParenthesesNode') negated.parens = true; return negated; } - if (name === '-@' || name === '+@') return this.unary(name[0], this.expression(receiver), where); + if (name === '-@' || name === '+@') return this.unary(name[0], yield this.expression(receiver), where); } if (receiver && args.length === 1 && BINARY_OPERATORS.has(name)) { - return this.binary(name, receiver, args[0], where); + return yield this.binary(name, receiver, args[0], where); } - if (receiver && args.length === 1 && name === '[]') return { ...this.indexOf(receiver, args[0]), ...where }; + if (receiver && args.length === 1 && name === '[]') + return { ...(yield this.indexOf(receiver, args[0])), ...where }; const quotient = name === 'to_i' && receiver && args.length === 0 ? callOf(unwrap(receiver)) : null; if (quotient && quotient.name === '/' && quotient.receiver && quotient.args && quotient.args.length === 1) { - const left = this.expression(quotient.receiver); + const left = yield this.expression(quotient.receiver); // `(-a / 2).to_i` is not `-a ÷ 2`, which the Japanese reads as `-(a ÷ 2)`. if (!(left.type === 'Unary' && left.operator === '-')) { - return { type: 'Binary', operator: '÷', left, right: this.expression(quotient.args[0]), ...where }; + return { + type: 'Binary', + operator: '÷', + left, + right: yield this.expression(quotient.args[0]), + ...where, + }; } } const table = this.calls[name]; @@ -1189,16 +1288,16 @@ class RubyReader { if (read) return { ...read, ...where }; if (!receiver && this.functions.has(name)) { if (call.isVariableCall) return { type: 'Name', name, ...where }; - return { type: 'Call', name, args: args.map((arg) => this.expression(arg)), ...where }; + return { type: 'Call', name, args: yield each(args, (arg) => this.expression(arg)), ...where }; } if (!receiver && this.passthrough.has(name)) { if (!call.hasParens && args.length === 0) return { type: 'Name', name, ...where }; - return { type: 'Call', name, args: args.map((arg) => this.expression(arg)), ...where }; + return { type: 'Call', name, args: yield each(args, (arg) => this.expression(arg)), ...where }; } this.failCall(call); - if (receiver) return this.raw(node); + if (receiver) return yield this.raw(node); if (call.isVariableCall) return { type: 'Name', name, ...where }; - return { type: 'Call', name, args: args.map((arg) => this.expression(arg)), ...where }; + return { type: 'Call', name, args: yield each(args, (arg) => this.expression(arg)), ...where }; } /** @@ -1211,7 +1310,9 @@ class RubyReader { this.fail('ruby-syntax', error.message, error.location); }); const lineCount = this.index.lineCount; - const body = this.statements(statementsOf(field(result.value, 'statements')), { from: 1, to: lineCount }); + const body = run( + this.statements(statementsOf(field(result.value, 'statements')), { from: 1, to: lineCount }), + ); // A comment no statement took (one at the end of a line of an expression // over lines) is kept on its own line rather than lost. this.lineComments.forEach((entry) => { diff --git a/packages/ja-notation/src/run.js b/packages/ja-notation/src/run.js new file mode 100644 index 0000000000..64ba0232aa --- /dev/null +++ b/packages/ja-notation/src/run.js @@ -0,0 +1,70 @@ +// SPDX-License-Identifier: MIT +// ja-notation: run a walk of a syntax tree written as generators, with a +// stack of its own instead of the call stack. See ../LICENSE. +// +// A syntax tree is as deep as the program: an expression of thousands of +// terms (`1 + 1 + ... + 1`), a long chain of signs or parentheses, or blocks +// in blocks. Walked by recursion, each level takes a frame of the call stack +// of JavaScript, which runs out at some thousands (RangeError). The walks of +// this library are written as generator functions instead: where they would +// call themselves on a child, they `yield` the walk of the child and are +// given back its result, and `run` keeps the walks that wait for a child on an +// array. The depth of the tree then only takes memory. +// +// *expression(e) { +// const left = yield this.expression(e.left); // not this.expression(e.left) +// ... +// } +// const ruby = run(builder.expression(e)); + +/** + * Run a walk (a generator) to its end and give its result. A walk yields the + * walks it needs the results of (generators); each runs before the walk goes on + * with its result. An error thrown in a walk is thrown into the walk that + * waits for it, and out of `run` when no walk catches it. + * @param {object} walk - The walk (a generator). + * @returns {unknown} What the walk returns. + */ +const run = (walk) => { + const stack = [walk]; + let value = null; + let failed = false; + let error; + while (stack.length > 0) { + const top = stack[stack.length - 1]; + let step; + try { + step = failed ? top.throw(error) : top.next(value); + failed = false; + } catch (e) { + stack.pop(); + failed = true; + error = e; + continue; + } + if (step.done) { + stack.pop(); + value = step.value; + } else { + stack.push(step.value); + value = null; + } + } + if (failed) throw error; + return value; +}; + +/** + * The walk of each item of a list, one after another, and the results. + * @param {Array} items - The items. + * @param {function(unknown, number): object} walkOf - The walk of an item. + * @yields {object} The walk of each item. + * @returns {Array} The results, in the order of the items. + */ +const each = function* (items, walkOf) { + const results = []; + for (let i = 0; i < items.length; i++) results.push(yield walkOf(items[i], i)); + return results; +}; + +export { each, run }; diff --git a/packages/ja-notation/src/settings.js b/packages/ja-notation/src/settings.js index d656cce8a3..16a7a112d4 100644 --- a/packages/ja-notation/src/settings.js +++ b/packages/ja-notation/src/settings.js @@ -200,24 +200,43 @@ const leftOfOne = (e, operator) => // `dropsToRuby(e)` / `dropsToJa(e)` say when the middle rule is taken. /** - * Whether the index shift to Ruby takes `e` (`x + 1`) to `x`. + * Whether the index shift by `operator` takes `e` to its left side: to Ruby + * (`+`) `x + 1` → `x` unless `x` would come back as `x + 1` (it is `y - 1` + * that is dropped to the Japanese), and so on down the left sides. Each step + * is the opposite of the one under it, so the steps are counted in a loop + * (a chain of thousands does not run out of the stack): it drops when they + * are odd. * @param {object} e - The JaAST expression. + * @param {string} operator - `+` (to Ruby) or `-` (to the Japanese). * @returns {boolean} True if it does. */ -const dropsToRuby = (e) => { - const x = leftOfOne(e, '+'); - return x !== null && integerOf(x) === null && !dropsToJa(x); +const drops = (e, operator) => { + let count = 0; + let current = e; + let op = operator; + for (;;) { + const x = leftOfOne(current, op); + if (x === null || integerOf(x) !== null) break; + count++; + current = x; + op = op === '+' ? '-' : '+'; + } + return count % 2 === 1; }; +/** + * Whether the index shift to Ruby takes `e` (`x + 1`) to `x`. + * @param {object} e - The JaAST expression. + * @returns {boolean} True if it does. + */ +const dropsToRuby = (e) => drops(e, '+'); + /** * Whether the index shift to the Japanese takes `e` (`x - 1`) to `x`. * @param {object} e - The JaAST expression. * @returns {boolean} True if it does. */ -const dropsToJa = (e) => { - const x = leftOfOne(e, '-'); - return x !== null && integerOf(x) === null && !dropsToRuby(x); -}; +const dropsToJa = (e) => drops(e, '-'); /** * Whether an index is a key that is not counted (a string: `X["a"]`), so it is not shifted. diff --git a/packages/ja-notation/src/write-ja.js b/packages/ja-notation/src/write-ja.js index 618e8963ad..96212ae1a2 100644 --- a/packages/ja-notation/src/write-ja.js +++ b/packages/ja-notation/src/write-ja.js @@ -17,6 +17,7 @@ // as a line of the block. // // The comments are written with the comment mark of the settings (`//` or `#`). +import { each, run } from './run.js'; import { settingsOf, writeComment } from './settings.js'; /** The spaces of one level of indent. */ @@ -73,9 +74,10 @@ const precedenceOf = (e) => { /** * Write an expression. * @param {object} e - The JaAST expression. + * @yields {object} The writings of its parts (./run.js). * @returns {string} The Japanese. */ -const expression = (e) => { +const expression = function* (e) { switch (e.type) { case 'Number': return e.value; @@ -88,15 +90,15 @@ const expression = (e) => { case 'RubyCode': return e.text; case 'Index': - return `${operand(e.target, PRIMARY)}[${expression(e.index)}]`; + return `${yield operand(e.target, PRIMARY)}[${yield expression(e.index)}]`; case 'ArrayLiteral': - return `[${e.elements.map(expression).join(', ')}]`; + return `[${(yield each(e.elements, expression)).join(', ')}]`; case 'Call': - return `${e.name}(${e.args.map(expression).join(', ')})`; + return `${e.name}(${(yield each(e.args, expression)).join(', ')})`; case 'Unary': - return unary(e); + return yield unary(e); case 'Binary': - return binary(e); + return yield binary(e); default: throw new Error(`ja-notation: cannot write ${e.type}`); } @@ -106,24 +108,27 @@ const expression = (e) => { * Write an operand, in parentheses when it binds less than its place needs. * @param {object} e - The JaAST expression. * @param {number} needed - The precedence the place needs. + * @yields {object} The writing of the expression (./run.js). * @returns {string} The Japanese. */ -const operand = (e, needed) => { - const text = expression(e); +const operand = function* (e, needed) { + const text = yield expression(e); return precedenceOf(e) < needed ? `(${text})` : text; }; /** * A sign or `not`. * @param {object} e - The Unary. + * @yields {object} The writing of its operand (./run.js). * @returns {string} The Japanese. */ -const unary = (e) => { +const unary = function* (e) { // `parens`: the operand was written in parentheses (`!(a > 10)`), which are // kept so a student does not read `not a > 10` as `(not a) > 10`. - if (e.operator === 'not') return `not ${operand(e.operand, e.parens ? PRIMARY : PRECEDENCE.not)}`; + if (e.operator === 'not') return `not ${yield operand(e.operand, e.parens ? PRIMARY : PRECEDENCE.not)}`; // `- -5` and `-(-a)` are written with parentheses. - const inner = e.operand.type === 'Unary' ? `(${expression(e.operand)})` : operand(e.operand, PRECEDENCE.sign); + const inner = + e.operand.type === 'Unary' ? `(${yield expression(e.operand)})` : yield operand(e.operand, PRECEDENCE.sign); return `${e.operator}${inner}`; }; @@ -131,11 +136,12 @@ const unary = (e) => { * A binary operator: the left side may be of the same precedence (the * operators are read from the left), the right side binds tighter. * @param {object} e - The Binary. + * @yields {object} The writings of its sides (./run.js). * @returns {string} The Japanese. */ -const binary = (e) => { +const binary = function* (e) { // The left sides of a long chain (`a and b and ... and z`) are followed in - // a loop, not by recursion, so thousands of terms do not run out of the stack. + // a loop, so the text is not made again and again through the chain. const needsOf = (b) => { const own = PRECEDENCE[b.operator]; return { own, left: NON_ASSOCIATIVE.has(b.operator) ? own + 1 : own }; @@ -146,10 +152,10 @@ const binary = (e) => { current = current.left; chain.push(current); } - let text = operand(current.left, needsOf(current).left); + let text = yield operand(current.left, needsOf(current).left); for (let i = chain.length - 1; i >= 0; i--) { const b = chain[i]; - const right = operand(b.right, needsOf(b).own + 1); + const right = yield operand(b.right, needsOf(b).own + 1); text = b.operator === '..' || b.operator === '...' ? `${text}${b.operator}${right}` @@ -202,16 +208,17 @@ class JaWriter { * @param {Array} body - The statements. * @param {number} depth - The indent level. * @param {?number} after - The line before the first statement (for the blank lines). + * @yields {object} The writings of the statements (./run.js). * @returns {?number} The last line the statements take. */ - body(body, depth, after) { + *body(body, depth, after) { let last = after; - body.forEach((s) => { + for (const s of body) { if (typeof s.line === 'number' && typeof last === 'number') this.gap(last + 1, s.line - 1); - this.statement(s, depth); + yield this.statement(s, depth); const end = s.closeLine || s.endLine || s.line; if (typeof end === 'number') last = end; - }); + } return last; } @@ -219,25 +226,26 @@ class JaWriter { * Write one statement. * @param {object} s - The statement. * @param {number} depth - The indent level. + * @yields {object} The writings of its parts (./run.js). */ - statement(s, depth) { + *statement(s, depth) { switch (s.type) { case 'If': - this.ifStatement(s, depth); + yield this.ifStatement(s, depth); return; case 'While': - this.block(s, depth, `${expression(s.condition)} の間繰り返す:`); + yield this.block(s, depth, `${yield expression(s.condition)} の間繰り返す:`); return; case 'For': { const way = s.direction === 'down' ? '減らしながら' : '増やしながら'; const head = - `${s.variable.name} を ${expression(s.from)} から ${expression(s.to)} まで ` + - `${expression(s.step)} ずつ${way}繰り返す:`; - this.block(s, depth, head); + `${s.variable.name} を ${yield expression(s.from)} から ${yield expression(s.to)} まで ` + + `${yield expression(s.step)} ずつ${way}繰り返す:`; + yield this.block(s, depth, head); return; } case 'FunctionDef': - this.block(s, depth, `関数 ${s.name}(${s.params.join(', ')}):`); + yield this.block(s, depth, `関数 ${s.name}(${s.params.join(', ')}):`); return; case 'Comment': this.push(depth, writeComment(s.text, this.settings), null, { isComment: true }); @@ -246,31 +254,32 @@ class JaWriter { this.rubyCode(s, depth); return; default: - this.push(depth, this.simple(s), s.comment); + this.push(depth, yield this.simple(s), s.comment); } } /** * The text of a statement of one line. * @param {object} s - The statement. + * @yields {object} The writings of its expressions (./run.js). * @returns {string} The Japanese. */ - simple(s) { + *simple(s) { switch (s.type) { case 'Assign': - return `${expression(s.target)} = ${expression(s.value)}`; + return `${yield expression(s.target)} = ${yield expression(s.value)}`; case 'OpAssign': - return `${expression(s.target)} ${s.operator}= ${expression(s.value)}`; + return `${yield expression(s.target)} ${s.operator}= ${yield expression(s.value)}`; case 'MultiAssign': - return `${s.targets.map(expression).join(', ')} = ${s.values.map(expression).join(', ')}`; + return `${(yield each(s.targets, expression)).join(', ')} = ${(yield each(s.values, expression)).join(', ')}`; case 'Input': - return `${expression(s.target)} = ${INPUTS[s.kind]}`; + return `${yield expression(s.target)} = ${INPUTS[s.kind]}`; case 'ExpressionStatement': - return expression(s.expression); + return yield expression(s.expression); case 'Return': - return s.value ? `${expression(s.value)} を返す` : '返す'; + return s.value ? `${yield expression(s.value)} を返す` : '返す'; case 'Append': - return `${expression(s.target)} に ${expression(s.value)} を追加する`; + return `${yield expression(s.target)} に ${yield expression(s.value)} を追加する`; default: throw new Error(`ja-notation: cannot write ${s.type}`); } @@ -307,10 +316,11 @@ class JaWriter { * @param {object} s - The statement. * @param {number} depth - The indent level. * @param {string} head - The opening line. + * @yields {object} The writings of its body (./run.js). */ - block(s, depth, head) { + *block(s, depth, head) { this.push(depth, head, s.comment); - const last = this.body(s.body, depth + 1, s.line); + const last = yield this.body(s.body, depth + 1, s.line); this.close(s, depth, last, !JaWriter.hasLine(s.body)); } @@ -318,21 +328,24 @@ class JaWriter { * `もし`, with its branches. * @param {object} s - The If. * @param {number} depth - The indent level. + * @yields {object} The writings of its parts (./run.js). */ - ifStatement(s, depth) { + *ifStatement(s, depth) { let empty = false; let last = null; - s.branches.forEach((branch, i) => { + for (let i = 0; i < s.branches.length; i++) { + const branch = s.branches[i]; if (i > 0) this.gap(last + 1, branch.line - 1); const word = i === 0 ? 'もし' : 'そうでなくもし'; - this.push(depth, `${word} ${expression(branch.condition)} ならば:`, i === 0 ? s.comment : branch.comment); - last = this.body(branch.body, depth + 1, branch.line); + const condition = yield expression(branch.condition); + this.push(depth, `${word} ${condition} ならば:`, i === 0 ? s.comment : branch.comment); + last = yield this.body(branch.body, depth + 1, branch.line); if (!JaWriter.hasLine(branch.body)) empty = true; - }); + } if (s.elseBody) { if (typeof last === 'number' && typeof s.elseLine === 'number') this.gap(last + 1, s.elseLine - 1); this.push(depth, 'そうでなければ:', s.elseComment); - last = this.body(s.elseBody, depth + 1, s.elseLine); + last = yield this.body(s.elseBody, depth + 1, s.elseLine); if (!JaWriter.hasLine(s.elseBody)) empty = true; } this.close(s, depth, last, empty); @@ -377,7 +390,7 @@ class JaWriter { */ const writeJa = (program) => { const writer = new JaWriter(program); - const last = writer.body(program.body, 0, 0); + const last = run(writer.body(program.body, 0, 0)); if (typeof program.lineCount === 'number') writer.gap((last || 0) + 1, program.lineCount); return writer.lines().join('\n'); }; diff --git a/packages/ja-notation/src/write-ruby.js b/packages/ja-notation/src/write-ruby.js index d32febed1e..0162e31a6a 100644 --- a/packages/ja-notation/src/write-ruby.js +++ b/packages/ja-notation/src/write-ruby.js @@ -1,6 +1,7 @@ // SPDX-License-Identifier: MIT // ja-notation: write the Ruby text of a RubyAST (./build-ruby.js), and which // Japanese line each Ruby line comes from. See ../LICENSE. +import { each, run } from './run.js'; /** How strongly each operator binds (higher binds tighter). */ const PRECEDENCE = { @@ -41,14 +42,19 @@ const precedenceOf = (node) => { return PRIMARY; }; -/** Writes the Ruby of expressions and of statements of one line. */ +/** + * Writes the Ruby of expressions and of statements of one line. The writings + * are generators run by `run` (./run.js), so a deep expression does not run + * out of the call stack. + */ const TEXT = { /** * Write an expression. * @param {object} node - The RubyAST expression. + * @yields {object} The writings of its parts (./run.js). * @returns {string} The Ruby. */ - expression(node) { + *expression(node) { switch (node.type) { case 'IntegerNode': case 'FloatNode': @@ -66,16 +72,16 @@ const TEXT = { case 'ConstantReadNode': return node.name; case 'ArrayNode': - return `[${node.elements.map((x) => this.expression(x)).join(', ')}]`; + return `[${(yield each(node.elements, (x) => this.expression(x))).join(', ')}]`; case 'AndNode': - return this.binary(node, '&&', node.left, node.right, 'AndNode'); + return yield this.binary(node, '&&', node.left, node.right, 'AndNode'); case 'OrNode': - return this.binary(node, '||', node.left, node.right, 'OrNode'); + return yield this.binary(node, '||', node.left, node.right, 'OrNode'); case 'RangeNode': // Written without spaces (`1..10`), as Ruby programs usually write it. - return this.binary(node, node.exclusive ? '...' : '..', node.left, node.right, 'RangeNode', ''); + return yield this.binary(node, node.exclusive ? '...' : '..', node.left, node.right, 'RangeNode', ''); case 'CallNode': - return this.callNode(node); + return yield this.callNode(node); default: throw new Error(`ja-notation: cannot write ${node.type}`); } @@ -86,10 +92,11 @@ const TEXT = { * @param {object} child - The operand. * @param {number} parent - The precedence of the operator. * @param {boolean} strict - Whether the same precedence needs parentheses too. + * @yields {object} The writings of its parts (./run.js). * @returns {string} The Ruby. */ - operand(child, parent, strict) { - const text = this.expression(child); + *operand(child, parent, strict) { + const text = yield this.expression(child); const own = precedenceOf(child); return own < parent || (strict && own === parent) ? `(${text})` : text; }, @@ -102,80 +109,89 @@ const TEXT = { * @param {object} right - The right side. * @param {string} key - The key in `PRECEDENCE`. * @param {string} [space] - What is put on each side of the operator. + * @yields {object} The writings of its parts (./run.js). * @returns {string} The Ruby. */ - binary(node, symbol, left, right, key, space = ' ') { + *binary(node, symbol, left, right, key, space = ' ') { const own = PRECEDENCE[key]; const strict = NON_ASSOCIATIVE.has(key); - return `${this.operand(left, own, strict)}${space}${symbol}${space}${this.operand(right, own, true)}`; + return `${yield this.operand(left, own, strict)}${space}${symbol}${space}${yield this.operand(right, own, true)}`; }, /** * A call: an operator, an index, a method or a function. * @param {object} node - The CallNode. + * @yields {object} The writings of its parts (./run.js). * @returns {string} The Ruby. */ - callNode(node) { + *callNode(node) { const { receiver, name } = node; const args = node.arguments; if (receiver === null) - return args === null ? name : `${name}(${args.map((x) => this.expression(x)).join(', ')})`; + return args === null ? name : `${name}(${(yield each(args, (x) => this.expression(x))).join(', ')})`; if (name === '-@' || name === '+@') { const sign = name[0]; // `-5` is a number to Ruby, and `-5.abs` is `(-5).abs`: only a variable // is written right after the sign. const plain = receiver.type === 'InstanceVariableReadNode' || receiver.type === 'LocalVariableReadNode'; - return plain ? `${sign}${this.expression(receiver)}` : `${sign}(${this.expression(receiver)})`; + return plain + ? `${sign}${yield this.expression(receiver)}` + : `${sign}(${yield this.expression(receiver)})`; } - if (name === '!') return `!${this.operand(receiver, PRECEDENCE['!'], false)}`; + if (name === '!') return `!${yield this.operand(receiver, PRECEDENCE['!'], false)}`; if (PRECEDENCE[name]) { - return this.binary(node, name, receiver, args[0], name); + return yield this.binary(node, name, receiver, args[0], name); } - const target = this.operand(receiver, PRIMARY, false); - if (name === '[]') return `${target}[${this.expression(args[0])}]`; - if (name === '[]=') return `${target}[${this.expression(args[0])}] = ${this.expression(args[1])}`; + const target = yield this.operand(receiver, PRIMARY, false); + if (name === '[]') return `${target}[${yield this.expression(args[0])}]`; + if (name === '[]=') return `${target}[${yield this.expression(args[0])}] = ${yield this.expression(args[1])}`; return args === null || args.length === 0 ? `${target}.${name}` - : `${target}.${name}(${args.map((x) => this.expression(x)).join(', ')})`; + : `${target}.${name}(${(yield each(args, (x) => this.expression(x))).join(', ')})`; }, /** * The Ruby of a statement that is one line. * @param {object} node - The RubyAST statement. + * @yields {object} The writings of its parts (./run.js). * @returns {string} The Ruby. */ - simpleStatement(node) { + *simpleStatement(node) { switch (node.type) { case 'InstanceVariableWriteNode': case 'LocalVariableWriteNode': - return `${node.name} = ${this.expression(node.value)}`; + return `${node.name} = ${yield this.expression(node.value)}`; case 'InstanceVariableOperatorWriteNode': case 'LocalVariableOperatorWriteNode': - return `${node.name} ${node.operator}= ${this.expression(node.value)}`; + return `${node.name} ${node.operator}= ${yield this.expression(node.value)}`; case 'IndexOperatorWriteNode': - return `${this.operand(node.receiver, PRIMARY, false)}[${this.expression(node.index)}] ${node.operator}= ${this.expression( + return `${yield this.operand(node.receiver, PRIMARY, false)}[${yield this.expression(node.index)}] ${node.operator}= ${yield this.expression( node.value, )}`; - case 'MultiWriteNode': - return `${node.targets.map((t) => (t.type === 'CallNode' ? this.simpleTarget(t) : t.name)).join(', ')} = ${node.values - .map((x) => this.expression(x)) - .join(', ')}`; + case 'MultiWriteNode': { + const targets = yield each(node.targets, function* (target) { + return target.type === 'CallNode' ? yield TEXT.simpleTarget(target) : target.name; + }); + const values = yield each(node.values, (x) => this.expression(x)); + return `${targets.join(', ')} = ${values.join(', ')}`; + } case 'ReturnNode': - return node.value ? `return ${this.expression(node.value)}` : 'return'; + return node.value ? `return ${yield this.expression(node.value)}` : 'return'; case 'CommentLine': return node.text; default: - return this.expression(node); + return yield this.expression(node); } }, /** * An index as the target of a multiple assignment (`A[i], b = 1, 2`). * @param {object} node - The `[]=` call without a value. + * @yields {object} The writings of its parts (./run.js). * @returns {string} The Ruby. */ - simpleTarget(node) { - return `${this.operand(node.receiver, PRIMARY, false)}[${this.expression(node.arguments[0])}]`; + *simpleTarget(node) { + return `${yield this.operand(node.receiver, PRIMARY, false)}[${yield this.expression(node.arguments[0])}]`; }, }; @@ -240,7 +256,7 @@ class LineWriter { // after its `end`, as the current reader (`dnclToRuby`) puts them. this.fill(node.closeLine); } - before.forEach((s) => this.line(depth + 1, TEXT.simpleStatement(s), node.closeLine, s.comment)); + before.forEach((s) => this.line(depth + 1, run(TEXT.simpleStatement(s)), node.closeLine, s.comment)); this.line(depth, 'end', node.closeLine, node.closeComment); } } @@ -250,21 +266,22 @@ class LineWriter { * @param {LineWriter} out - Where to write. * @param {Array} statements - The statements. * @param {number} depth - The block depth. + * @yields {object} The writings of the statements and their parts (./run.js). */ -const writeStatements = (out, statements, depth) => { - statements.forEach((node) => { +const writeStatements = function* (out, statements, depth) { + for (const node of statements) { switch (node.type) { case 'IfNode': { - out.statement(depth, `if ${TEXT.expression(node.predicate)}`, node); - writeStatements(out, node.statements, depth + 1); + out.statement(depth, `if ${yield TEXT.expression(node.predicate)}`, node); + yield writeStatements(out, node.statements, depth + 1); let next = node.subsequent; while (next) { if (next.type === 'IfNode') { - out.statement(depth, `elsif ${TEXT.expression(next.predicate)}`, next); + out.statement(depth, `elsif ${yield TEXT.expression(next.predicate)}`, next); } else { out.statement(depth, 'else', next); } - writeStatements(out, next.statements, depth + 1); + yield writeStatements(out, next.statements, depth + 1); next = next.type === 'IfNode' ? next.subsequent : null; } out.end(depth, node); @@ -273,21 +290,21 @@ const writeStatements = (out, statements, depth) => { case 'WhileNode': case 'UntilNode': { const keyword = node.type === 'WhileNode' ? 'while' : 'until'; - out.statement(depth, `${keyword} ${TEXT.expression(node.predicate)}`, node); + out.statement(depth, `${keyword} ${yield TEXT.expression(node.predicate)}`, node); const body = node.increment ? node.statements.slice(0, -1) : node.statements; - writeStatements(out, body, depth + 1); + yield writeStatements(out, body, depth + 1); out.end(depth, node, node.increment ? [node.increment] : []); break; } case 'DefNode': out.statement(depth, `def ${node.name}(${node.parameters.join(', ')})`, node); - writeStatements(out, node.statements, depth + 1); + yield writeStatements(out, node.statements, depth + 1); out.end(depth, node); break; default: - out.statement(depth, TEXT.simpleStatement(node), node); + out.statement(depth, yield TEXT.simpleStatement(node), node); } - }); + } }; /** @@ -300,7 +317,7 @@ const writeStatements = (out, statements, depth) => { */ const writeRuby = (program, jaLineCount = 0) => { const out = new LineWriter(); - writeStatements(out, program.statements, 0); + run(writeStatements(out, program.statements, 0)); out.fill(jaLineCount); return { ruby: out.lines.join('\n'), jaLines: out.jaLines }; }; diff --git a/packages/ja-notation/test/data/ja-to-ruby.json b/packages/ja-notation/test/data/ja-to-ruby.json index 09645ecbea..55e2b71a85 100644 --- a/packages/ja-notation/test/data/ja-to-ruby.json +++ b/packages/ja-notation/test/data/ja-to-ruby.json @@ -79,11 +79,23 @@ "jaLines": [1, 1, 2, 3, 3] }, { - "title": "a counting loop closed by another closing word adds the step too", + "title": "a counting loop closed by another closing word is an error that says its closing word (W3 of Issue #1731)", "table": "plain", "ja": "i を 1 から 3 まで 1 ずつ増やしながら\n 表示する(i)\nを実行する", - "ruby": "i = 1\nwhile i <= 3\n puts(i)\n i += 1\nend", - "jaLines": [1, 1, 2, 3, 3] + "errors": [ + { + "code": "syntax", + "message": "「を実行する」ではなく「を繰り返す」で閉じてください", + "line": 3, + "column": 0, + "opener": "For", + "direction": "up", + "colon": false, + "block": "closer-kind", + "word": "を実行する", + "expected": "を繰り返す" + } + ] }, { "title": "the names and functions are what the table says", @@ -148,10 +160,10 @@ "jaLines": [1, 2, 3, 4, 5] }, { - "title": "a Greek or full-width letter is a name, of its own (x is not x), and full-width digits are in a name after its first character", + "title": "a Greek letter is a name, and a full-width letter or digit is read as the half-width one (x1 is x1, as the Ruby mode makes it)", "table": "none", "ja": "α = 1\nx1 = α ÷ 2\nx = x1", - "ruby": "α = 1\nx1 = (α / 2).to_i\nx = x1", + "ruby": "α = 1\nx1 = (α / 2).to_i\nx = x1", "jaLines": [1, 2, 3] }, { @@ -188,10 +200,38 @@ "jaLines": [1, 2, 3] }, { - "title": "full-width digits are read as the digits (only how they look differs), and the Ruby has the half-width ones", + "title": "full-width digits are read as the digits (only how they look differs), in a number and in a name, and the Ruby has the half-width ones", "table": "none", "ja": "x = 12\ny = 3.5 + x1", - "ruby": "x = 12\ny = 3.5 + x1", + "ruby": "x = 12\ny = 3.5 + x1", + "jaLines": [1, 2] + }, + { + "title": "the full-width letters and digits in a string are kept as they are written", + "table": "plain", + "ja": "表示する(\"ABC123\")\n表示する(「x=1」)\ns = 'a1'", + "ruby": "puts(\"ABC123\")\nputs(\"x=1\")\ns = 'a1'", + "jaLines": [1, 2, 3] + }, + { + "title": "the full-width letters and digits in a comment are kept as they are written, and the code before it is made half-width", + "table": "none", + "ja": "# x は 1 から\nx = 1 # x は 2 まで", + "ruby": "# x は 1 から\nx = 1 # x は 2 まで", + "jaLines": [1, 2] + }, + { + "title": "a keyword in full-width letters is the keyword (and is and)", + "table": "none", + "ja": "flag = x > 0 and not y", + "ruby": "flag = x > 0 && !y", + "jaLines": [1] + }, + { + "title": "with // as the comment mark, the full-width letters of the comment are kept", + "table": "none", + "ja": "// 「//」から行の終わりまでは,処理の対象とならない。\nx = 1 // x を 1 に", + "ruby": "# 「//」から行の終わりまでは,処理の対象とならない。\nx = 1 # x を 1 に", "jaLines": [1, 2] }, { @@ -214,7 +254,7 @@ "errors": [ { "code": "syntax", - "message": "「10回」は読めません。数と名前の間には、「10 * 回」のように演算子を書いてください", + "message": "「10回」は読めません。数と名前の間には、「10 * 回」のように演算子を書いてください", "line": 1, "column": 6 } diff --git a/packages/ja-notation/test/data/parse.json b/packages/ja-notation/test/data/parse.json index 7b7d595a4e..1f0f17ae56 100644 --- a/packages/ja-notation/test/data/parse.json +++ b/packages/ja-notation/test/data/parse.json @@ -1292,6 +1292,11 @@ "ja": "もし a > 0 ならば\n b = 1\nend", "as": "もし a > 0 ならば:\n b = 1" }, + { + "title": "`end` closes any kind of block opened without `:` (as in Ruby; the closing words must be of their block)", + "ja": "x > 0 の間\n x = x - 1\nend", + "as": "x > 0 の間繰り返す:\n x = x - 1" + }, { "title": "`を追加する` with no spaces around the value in brackets", "ja": "A に (3) を追加する", @@ -1341,6 +1346,107 @@ } ] }, + { + "title": "a closing word of another kind of block is an error on the closing line (W3 of Issue #1731)", + "ja": "もし a > 0 ならば\n b = 1\nを繰り返す", + "errors": [ + { + "code": "syntax", + "message": "「を繰り返す」ではなく「を実行する」で閉じてください", + "line": 3, + "column": 0, + "opener": "If", + "colon": false, + "block": "closer-kind", + "word": "を繰り返す", + "expected": "を実行する" + } + ] + }, + { + "title": "a loop closed by を実行する is an error", + "ja": "a > 0 の間\n a = a - 1\nを実行する", + "errors": [ + { + "code": "syntax", + "message": "「を実行する」ではなく「を繰り返す」で閉じてください", + "line": 3, + "column": 0, + "opener": "While", + "colon": false, + "block": "closer-kind", + "word": "を実行する", + "expected": "を繰り返す" + } + ] + }, + { + "title": "a counting loop down closed by と定義する is an error", + "ja": "i を 3 から 1 まで 1 ずつ減らしながら\n b = i\nと定義する", + "errors": [ + { + "code": "syntax", + "message": "「と定義する」ではなく「を繰り返す」で閉じてください", + "line": 3, + "column": 0, + "opener": "For", + "direction": "down", + "colon": false, + "block": "closer-kind", + "word": "と定義する", + "expected": "を繰り返す" + } + ] + }, + { + "title": "a function closed by を繰り返す is an error", + "ja": "関数 f(a)\n 返す a\nを繰り返す", + "errors": [ + { + "code": "syntax", + "message": "「を繰り返す」ではなく「と定義する」で閉じてください", + "line": 3, + "column": 0, + "opener": "FunctionDef", + "colon": false, + "block": "closer-kind", + "word": "を繰り返す", + "expected": "と定義する" + } + ] + }, + { + "title": "a block opened with : closed by a line of another kind at its indent is an error", + "ja": "もし a > 0 ならば:\n b = 1\nを繰り返す", + "errors": [ + { + "code": "syntax", + "message": "「を繰り返す」ではなく「を実行する」で閉じてください", + "line": 3, + "column": 0, + "opener": "If", + "colon": true, + "block": "closer-kind", + "word": "を繰り返す", + "expected": "を実行する" + } + ] + }, + { + "title": "a block forgotten to be closed inside the one the closing word closes is told as not closed", + "ja": "i を 1 から 3 まで 1 ずつ増やしながら\n もし i > 1 ならば\n x = 1\nを繰り返す", + "errors": [ + { + "code": "unclosed", + "message": "「を実行する」で閉じていません", + "line": 2, + "column": 2, + "opener": "If", + "colon": false, + "word": "を実行する" + } + ] + }, { "title": "もし not closed", "ja": "もし a > 0 ならば\n b = 1", diff --git a/packages/ja-notation/test/data/ruby-to-ja.json b/packages/ja-notation/test/data/ruby-to-ja.json index 173d0ae301..93d40fd547 100644 --- a/packages/ja-notation/test/data/ruby-to-ja.json +++ b/packages/ja-notation/test/data/ruby-to-ja.json @@ -181,6 +181,102 @@ "ruby": "x = gets\nA[0] = gets.to_i", "ja": "x = 【外部からの入力】\nA[0] = 【整数を入力】", "errors": [] + }, + { + "title": "a variable of the stage ($) is told where it is, as the Japanese has no stage (Issue #1706)", + "table": "none", + "ruby": "$a = 1\nx = $a + 1", + "ja": "$a = 1\nx = $a + 1", + "errors": [ + { + "code": "stage-variable", + "message": "ステージの変数($a)は日本語では書けません", + "line": 1, + "column": 0, + "name": "$a" + }, + { + "code": "stage-variable", + "message": "ステージの変数($a)は日本語では書けません", + "line": 2, + "column": 4, + "name": "$a" + } + ] + }, + { + "title": "a list of the stage is told once, not as a command it calls (Issue #1706)", + "table": "none", + "ruby": "$a = []\n$a.push(1)", + "ja": "$a = []\n$a.push(1)", + "errors": [ + { + "code": "stage-variable", + "message": "ステージの変数($a)は日本語では書けません", + "line": 1, + "column": 0, + "name": "$a" + }, + { + "code": "stage-variable", + "message": "ステージの変数($a)は日本語では書けません", + "line": 2, + "column": 0, + "name": "$a" + } + ] + }, + { + "title": "a variable of the stage changed by an operator is told at its name (Issue #1706)", + "table": "none", + "ruby": "$a += 1", + "ja": "$a += 1", + "errors": [ + { + "code": "stage-variable", + "message": "ステージの変数($a)は日本語では書けません", + "line": 1, + "column": 0, + "name": "$a" + } + ] + }, + { + "title": "a variable of the stage among the targets of a multiple assignment is told (Issue #1706)", + "table": "none", + "ruby": "x, $b = 1, 2", + "ja": "x, $b = 1, 2", + "errors": [ + { + "code": "stage-variable", + "message": "ステージの変数($b)は日本語では書けません", + "line": 1, + "column": 3, + "name": "$b" + } + ] + }, + { + "title": "an element of a list of the stage is told at the list (Issue #1706)", + "table": "none", + "ruby": "$a[0] = 1\nx = $a[1]", + "ja": "$a[0] = 1\nx = $a[1]", + "errors": [ + { + "code": "stage-variable", + "message": "ステージの変数($a)は日本語では書けません", + "line": 1, + "column": 0, + "name": "$a" + }, + { + "code": "stage-variable", + "message": "ステージの変数($a)は日本語では書けません", + "line": 2, + "column": 4, + "name": "$a" + } + ] } ] } diff --git a/packages/ja-notation/test/deep.test.js b/packages/ja-notation/test/deep.test.js new file mode 100644 index 0000000000..8c461a896c --- /dev/null +++ b/packages/ja-notation/test/deep.test.js @@ -0,0 +1,133 @@ +// SPDX-License-Identifier: MIT +// Deep and long programs: the walks of the syntax trees (parse, buildRuby, +// writeRuby, readRuby, writeJa) keep a stack of their own (../src/run.js), so +// an expression of thousands of terms, a chain of thousands of signs or +// parentheses and blocks thousands deep do not run out of the call stack. +// The programs are made by repeating, so they are written here, not in ./data/. +import assert from 'node:assert/strict'; +import { before, describe, test } from 'node:test'; +import { jaToRuby, parse, rubyToJa, writeJa } from '../src/index.js'; +import { prism } from './helpers.js'; + +const N = 5000; + +const repeat = (n, f) => Array.from({ length: n }, (_, i) => f(i)); + +/** + * Compare two long texts, saying only where they first differ (not the + * thousands of characters). + * @param {string} actual - The text made. + * @param {string} expected - The text expected. + */ +const sameText = (actual, expected) => { + if (actual === expected) return; + const at = [...expected].findIndex((ch, i) => actual[i] !== ch); + assert.fail( + `differs at ${at}: ${JSON.stringify(String(actual).slice(at, at + 40))} for ${JSON.stringify(expected.slice(at, at + 40))}`, + ); +}; + +describe('Japanese → Ruby of deep programs', () => { + const cases = [ + { + title: `an expression of ${N} terms`, + ja: `x = ${repeat(N, () => '1').join(' + ')}`, + ruby: `x = ${repeat(N, () => '1').join(' + ')}`, + }, + { + title: `${N} conditions joined by and`, + ja: `x = ${repeat(N, () => 'a > 1').join(' and ')}`, + ruby: `x = ${repeat(N, () => 'a > 1').join(' && ')}`, + }, + { + title: `${N} parentheses in parentheses`, + ja: `x = ${'('.repeat(N)}1${')'.repeat(N)}`, + ruby: 'x = 1', + }, + { + title: `${N} terms nested on the right`, + ja: `x = ${'1 + ('.repeat(N)}1${')'.repeat(N)}`, + ruby: `x = ${'1 + ('.repeat(N - 1)}1 + 1${')'.repeat(N - 1)}`, + }, + { + title: `${N} signs`, + ja: `x = ${'-'.repeat(N)}a`, + ruby: `x = ${'-('.repeat(N - 1)}-a${')'.repeat(N - 1)}`, + }, + { + title: `${N} nots`, + ja: `x = ${'not '.repeat(N)}a`, + ruby: `x = ${'!'.repeat(N)}a`, + }, + { + title: `${N} calls in calls`, + ja: `関数 f(a):\n a を返す\nx = ${'f('.repeat(N)}1${')'.repeat(N)}`, + ruby: `def f(a)\n return a\nend\nx = ${'f('.repeat(N)}1${')'.repeat(N)}`, + }, + { + title: `${N} indexes in indexes`, + ja: `a = [0]\nx = ${'a['.repeat(N)}0${']'.repeat(N)}`, + ruby: `a = [0]\nx = ${'a['.repeat(N)}0${']'.repeat(N)}`, + }, + ]; + cases.forEach(({ title, ja, ruby }) => { + test(title, () => { + const result = jaToRuby(ja); + assert.deepEqual(result.errors, []); + sameText(result.ruby, ruby); + }); + }); + + test('blocks 1000 deep, in the standard form and in the DNCL-like form', () => { + const depth = 1000; + const ruby = [ + ...repeat(depth, (i) => `${' '.repeat(i)}if a > 0`), + `${' '.repeat(depth)}x = 1`, + ...repeat(depth, (i) => `${' '.repeat(depth - 1 - i)}end`), + ].join('\n'); + const standard = [ + ...repeat(depth, (i) => `${' '.repeat(i)}もし a > 0 ならば:`), + `${' '.repeat(depth)}x = 1`, + ].join('\n'); + const dnclLike = [ + ...repeat(depth, () => 'もし a > 0 ならば'), + 'x = 1', + ...repeat(depth, () => 'を実行する'), + ].join('\n'); + sameText(jaToRuby(standard).ruby, ruby); + sameText(jaToRuby(dnclLike).ruby, ruby); + }); + + test(`the JaAST of an expression of ${N} terms is written back as the same Japanese`, () => { + const ja = `x = ${repeat(N, () => '1').join(' + ')} - ${'('.repeat(N)}a${')'.repeat(N)}`; + sameText(writeJa(parse(ja).program), `x = ${repeat(N, () => '1').join(' + ')} - a`); + }); +}); + +describe('Ruby → Japanese of long programs (as long as prism can parse them)', () => { + let parseRuby; + before(async () => { + parseRuby = await prism(); + }); + + test('an expression of 1000 terms', () => { + const ruby = `x = ${repeat(1000, () => '1').join(' + ')}`; + sameText(rubyToJa(ruby, { parse: parseRuby }).ja, ruby); + }); + + test('3000 conditions joined by &&', () => { + const ruby = `x = ${repeat(3000, () => 'a > 1').join(' && ')}`; + sameText(rubyToJa(ruby, { parse: parseRuby }).ja, `x = ${repeat(3000, () => 'a > 1').join(' and ')}`); + }); + + test('1000 elsif', () => { + const ruby = ['a = 1', 'if a > 0', ' x = 0', ...repeat(1000, (i) => `elsif a > ${i}\n x = ${i}`), 'end']; + const ja = [ + 'a = 1', + 'もし a > 0 ならば:', + ' x = 0', + ...repeat(1000, (i) => `そうでなくもし a > ${i} ならば:\n x = ${i}`), + ]; + sameText(rubyToJa(ruby.join('\n'), { parse: parseRuby }).ja, ja.join('\n')); + }); +}); diff --git a/packages/scratch-gui/src/containers/ruby-tab.jsx b/packages/scratch-gui/src/containers/ruby-tab.jsx index 484ba6c015..7ff9339b1f 100644 --- a/packages/scratch-gui/src/containers/ruby-tab.jsx +++ b/packages/scratch-gui/src/containers/ruby-tab.jsx @@ -23,6 +23,7 @@ import { FORBIDDEN_PREFIX_ERROR_ID, isRubyCommandName, POST_TEST_LOOP_ERROR_ID, + STAGE_VARIABLE_ERROR_ID, UNSUPPORTED_COMMAND_ERROR_ID, UNSUPPORTED_NOTATION_ERROR_ID, UNSUPPORTED_NUMBER_ERROR_ID, @@ -812,6 +813,16 @@ const RubyTab = (props) => { { name: err.name }, ); } + if (err.id === STAGE_VARIABLE_ERROR_ID) { + return intl.formatMessage( + { + id: 'gui.rubyTab.dnclStageVariableError', + defaultMessage: + 'A variable of the stage ({name}) cannot be written in Japanese mode. You can write it in Ruby mode.', + }, + { name: err.name }, + ); + } if (err.id === UNSUPPORTED_NOTATION_ERROR_ID) { return intl.formatMessage( { diff --git a/packages/scratch-gui/src/lib/dncl/dncl-to-ruby.js b/packages/scratch-gui/src/lib/dncl/dncl-to-ruby.js index 3110db4a6e..2afe4cb139 100644 --- a/packages/scratch-gui/src/lib/dncl/dncl-to-ruby.js +++ b/packages/scratch-gui/src/lib/dncl/dncl-to-ruby.js @@ -25,8 +25,9 @@ const dnclToRuby = (source) => { try { result = jaToRuby(source, SMALRUBY_JA_NOTATION); } catch (e) { - // The syntax tree is as deep as the longest expression: one of thousands - // of terms is too deep to read, and must not stop the editor. + // ja-notation walks its syntax trees with a stack of its own, so a deep + // program is read; one too large to hold (a RangeError of the memory of + // JavaScript) must still not stop the editor. if (!(e instanceof RangeError)) throw e; return { ruby: null, diff --git a/packages/scratch-gui/src/lib/dncl/dncl-validator.js b/packages/scratch-gui/src/lib/dncl/dncl-validator.js index f1a03aa765..f6b60a99d6 100644 --- a/packages/scratch-gui/src/lib/dncl/dncl-validator.js +++ b/packages/scratch-gui/src/lib/dncl/dncl-validator.js @@ -172,6 +172,22 @@ const UNSUPPORTED_NOTATION_ERROR_ID = 'unsupportedNotation'; */ const unsupportedNotationMessage = (name) => `「${name}」は日本語モードでは書けません。Ruby モードでは書けます。`; +/** + * The id of the error for a variable or a list of the stage (`$a`) in the Ruby + * (issue #1706). The Japanese mode is written for a program that runs alone, + * as on a terminal, and has no stage: Ruby → Japanese does not write them and + * tells them where they are. The `name` is the variable (`$a`). + */ +const STAGE_VARIABLE_ERROR_ID = 'stageVariable'; + +/** + * The message of the stage variable error. + * @param {string} name - The variable (`$a`). + * @returns {string} The message. + */ +const stageVariableMessage = (name) => + `ステージの変数(${name})は日本語モードでは書けません。Ruby モードでは書けます。`; + /** * The command of a `do` block line: the name just before `do` and the * arguments of the command (`with_screen_refresh do` → `with_screen_refresh`, @@ -320,6 +336,8 @@ export { isRubyCommandName, POST_TEST_LOOP_ERROR_ID, POST_TEST_LOOP_MESSAGE, + STAGE_VARIABLE_ERROR_ID, + stageVariableMessage, UNSUPPORTED_COMMAND_ERROR_ID, unsupportedCommandMessage, UNSUPPORTED_NOTATION_ERROR_ID, diff --git a/packages/scratch-gui/src/lib/dncl/ja-notation-errors.js b/packages/scratch-gui/src/lib/dncl/ja-notation-errors.js index 82e8cf5c32..847c229b5e 100644 --- a/packages/scratch-gui/src/lib/dncl/ja-notation-errors.js +++ b/packages/scratch-gui/src/lib/dncl/ja-notation-errors.js @@ -12,6 +12,8 @@ import { FORBIDDEN_PREFIX_ERROR_ID, POST_TEST_LOOP_ERROR_ID, POST_TEST_LOOP_MESSAGE, + STAGE_VARIABLE_ERROR_ID, + stageVariableMessage, UNSUPPORTED_NOTATION_ERROR_ID, UNSUPPORTED_NUMBER_ERROR_ID, UNSUPPORTED_COMMAND_ERROR_ID, @@ -49,6 +51,12 @@ const blockMessage = (error) => { ? `${names.standard}の中の行を字下げしてください` : `${names.name}を「${names.closer}」で閉じてください`; } + if (error.block === 'closer-kind') { + // A closer of another kind of block (W3 of #1731). + const names = BLOCK_NAMES[error.opener === 'For' ? error.direction : error.opener]; + if (!names) return null; + return `${error.colon ? names.standard : names.name}は「${error.word}」ではなく「${names.closer}」で閉じてください`; + } if (error.block === 'stray-closer') return `「${error.word}」で閉じるものがありません`; if (error.block === 'stray-branch') { const name = BRANCH_NAMES[error.branch]; @@ -65,7 +73,7 @@ const blockMessage = (error) => { * @param {object} error - An error of ja-notation. * @returns {?object} The error, or null when it is not one of the words of * the Japanese mode (a Ruby command, a number, Ruby with no Japanese form, - * `@`, the post-test loop). + * a variable of the stage, `@`, the post-test loop). */ const wordError = (error) => { const at = { line: error.line, column: error.column + 1 }; @@ -88,6 +96,13 @@ const wordError = (error) => { name: error.name, message: unsupportedNotationMessage(error.name), }; + case 'stage-variable': + return { + ...at, + id: STAGE_VARIABLE_ERROR_ID, + name: error.name, + message: stageVariableMessage(error.name), + }; case 'unknown-command': return { ...at, @@ -158,10 +173,9 @@ const openerOf = (code) => { * block closed by a line of its own), found by the kind of each closing line * (Issue #1419): a closing line closes the nearest open block it is the * closer of, and the blocks left open inside it are told, so a student who - * forgot one `を実行する` is told about that `もし`. ja-notation closes the - * innermost block with any closing line, as Ruby closes it with any `end`, - * so a program that reads that way has no error, and this is used only to - * say the errors of one that does not. + * forgot one `を実行する` is told about that `もし`. ja-notation tells the same + * (a closing line of another kind of block, W3 of #1731), and this is used + * only to say the errors of a program it tells, in the words of the blocks. * @param {Array} lines - The lines (line numbers and marks blanked out). * @returns {?Array} The errors, or null when a block of the program * is opened with `:` (the standard form, whose errors ja-notation says). diff --git a/packages/scratch-gui/src/lib/dncl/ruby-to-dncl.js b/packages/scratch-gui/src/lib/dncl/ruby-to-dncl.js index 3ce677c339..708038f0e4 100644 --- a/packages/scratch-gui/src/lib/dncl/ruby-to-dncl.js +++ b/packages/scratch-gui/src/lib/dncl/ruby-to-dncl.js @@ -27,8 +27,9 @@ const rubyToDncl = (source) => { const result = rubyToJa(source, { ...SMALRUBY_RUBY_TO_JA, parse: (ruby) => prism.parse(ruby) }); return { dncl: result.ja, errors: rubyErrorWords(result.errors) }; } catch (e) { - // The syntax tree is as deep as the longest expression: one of thousands - // of terms is too deep to read, and must not stop the editor. + // prism reads its syntax tree into JavaScript by recursion, so an expression + // of some thousands of terms is too deep for it (ja-notation itself walks + // the tree with a stack of its own), and must not stop the editor. if (!(e instanceof RangeError)) throw e; return { dncl: source, errors: [{ line: 1, column: 1, message: 'プログラムが長すぎて日本語にできません' }] }; } diff --git a/packages/scratch-gui/src/locales/en.js b/packages/scratch-gui/src/locales/en.js index 6ee6c625ca..4cc0b8c4b3 100644 --- a/packages/scratch-gui/src/locales/en.js +++ b/packages/scratch-gui/src/locales/en.js @@ -1570,6 +1570,8 @@ export default { 'gui.rubyTab.dnclForbiddenPrefixError': '"{name}" cannot be used in a variable in Japanese mode.', 'gui.rubyTab.dnclPostTestLoopError': 'A post-test loop (a loop that runs once before it checks the condition) cannot be written in Japanese. Write the first run before the loop, then make it a "〜の間繰り返す" loop. You can write it in Ruby mode.', + 'gui.rubyTab.dnclStageVariableError': + 'A variable of the stage ({name}) cannot be written in Japanese mode. You can write it in Ruby mode.', 'gui.rubyTab.dnclUndefinedFunctionError': '"{name}" is a function that is not defined. Define it with "関数 {name}(…):".', 'gui.rubyTab.dnclUnsupportedCommandError': diff --git a/packages/scratch-gui/src/locales/ja-Hira.js b/packages/scratch-gui/src/locales/ja-Hira.js index 31fe8a035f..1fce034b5d 100644 --- a/packages/scratch-gui/src/locales/ja-Hira.js +++ b/packages/scratch-gui/src/locales/ja-Hira.js @@ -1948,6 +1948,8 @@ export default { 'gui.rubyTab.dnclForbiddenPrefixError': 'にほんごモードでは、へんすうに「{name}」はつかえません', 'gui.rubyTab.dnclPostTestLoopError': 'あとではんていするくりかえし(さきに 1 かいうごかしてからじょうけんをたしかめるくりかえし)は、にほんごではかけません。さいしょの 1 かいをループのまえにかいてから、「〜のあいだくりかえす」にしてください。Ruby モードではかけます。', + 'gui.rubyTab.dnclStageVariableError': + 'ステージのへんすう({name})は、にほんごモードではかけません。Ruby モードではかけます。', 'gui.rubyTab.dnclUndefinedFunctionError': '「{name}」は、ていぎされていないかんすうです。「関数 {name}(…):」でていぎしてください。', 'gui.rubyTab.dnclUnsupportedCommandError': diff --git a/packages/scratch-gui/src/locales/ja.js b/packages/scratch-gui/src/locales/ja.js index f215ffacce..66a1c0bd88 100644 --- a/packages/scratch-gui/src/locales/ja.js +++ b/packages/scratch-gui/src/locales/ja.js @@ -1907,6 +1907,8 @@ export default { 'gui.rubyTab.dnclForbiddenPrefixError': '日本語モードでは変数に「{name}」は使えません', 'gui.rubyTab.dnclPostTestLoopError': '後判定の繰り返し(先に 1 回動かしてから条件を確かめる繰り返し)は日本語では書けません。最初の 1 回をループの前に書いてから、「〜の間繰り返す」にしてください。Ruby モードでは書けます。', + 'gui.rubyTab.dnclStageVariableError': + 'ステージの変数({name})は日本語モードでは書けません。Ruby モードでは書けます。', 'gui.rubyTab.dnclUndefinedFunctionError': '「{name}」は定義されていない関数です。「関数 {name}(…):」で定義してください。', 'gui.rubyTab.dnclUnsupportedCommandError': diff --git a/packages/scratch-gui/test/fixtures/ja-notation/golden/ja.json b/packages/scratch-gui/test/fixtures/ja-notation/golden/ja.json index edb20b4f77..d23a017d24 100644 --- a/packages/scratch-gui/test/fixtures/ja-notation/golden/ja.json +++ b/packages/scratch-gui/test/fixtures/ja-notation/golden/ja.json @@ -332,7 +332,7 @@ } }, "ja-906fd1a488": { - "ruby": "@x〆 = 1\n@𠮷田 = @x〆 + 2\n@x1 = (@𠮷田 / 2).to_i\nconsole.puts(@x1)", + "ruby": "@x〆 = 1\n@𠮷田 = @x〆 + 2\n@x1 = (@𠮷田 / 2).to_i\nconsole.puts(@x1)", "errors": [], "rubyLines": [ 1, @@ -341,10 +341,10 @@ 4 ], "converterErrors": [], - "blocks": "data_setvariableto(VARIABLE = @x〆, VALUE: \"1\")\ndata_setvariableto(VARIABLE = @𠮷田, VALUE: operator_add(NUM1: @x〆, NUM2: 2))\ndata_setvariableto(VARIABLE = @x1, VALUE: operator_mathop(OPERATOR = \"floor\", NUM: operator_divide(NUM1: @𠮷田, NUM2: 2)) #\"@ruby:method:to_i\")\nsmalrubyConsole_print(TEXT: @x1)", + "blocks": "data_setvariableto(VARIABLE = @x〆, VALUE: \"1\")\ndata_setvariableto(VARIABLE = @𠮷田, VALUE: operator_add(NUM1: @x〆, NUM2: 2))\ndata_setvariableto(VARIABLE = @x1, VALUE: operator_mathop(OPERATOR = \"floor\", NUM: operator_divide(NUM1: @𠮷田, NUM2: 2)) #\"@ruby:method:to_i\")\nsmalrubyConsole_print(TEXT: @x1)", "shown": { - "ruby": "@x〆 = 1\n@𠮷田 = @x〆 + 2\n@x1 = (@𠮷田 / 2).to_i\nconsole.puts(@x1)", - "ja": "x〆 = 1\n𠮷田 = x〆 + 2\nx1 = 𠮷田 ÷ 2\n表示する(x1)", + "ruby": "@x〆 = 1\n@𠮷田 = @x〆 + 2\n@x1 = (@𠮷田 / 2).to_i\nconsole.puts(@x1)", + "ja": "x〆 = 1\n𠮷田 = x〆 + 2\nx1 = 𠮷田 ÷ 2\n表示する(x1)", "errors": [] } }, diff --git a/packages/scratch-gui/test/fixtures/ja-notation/golden/ruby.json b/packages/scratch-gui/test/fixtures/ja-notation/golden/ruby.json index dc1ae666fd..dd2ba056c5 100644 --- a/packages/scratch-gui/test/fixtures/ja-notation/golden/ruby.json +++ b/packages/scratch-gui/test/fixtures/ja-notation/golden/ruby.json @@ -171,9 +171,9 @@ { "line": 1, "column": 6, - "id": "unsupportedNotation", + "id": "stageVariable", "name": "$b", - "message": "「$b」は日本語モードでは書けません。Ruby モードでは書けます。" + "message": "ステージの変数($b)は日本語モードでは書けません。Ruby モードでは書けます。" }, { "line": 3, diff --git a/packages/scratch-gui/test/unit/lib/dncl/dncl-long-program.test.js b/packages/scratch-gui/test/unit/lib/dncl/dncl-long-program.test.js new file mode 100644 index 0000000000..1a2ff0141d --- /dev/null +++ b/packages/scratch-gui/test/unit/lib/dncl/dncl-long-program.test.js @@ -0,0 +1,38 @@ +/** + * A long or deep program (an expression of thousands of terms, blocks in + * blocks) is converted: ja-notation walks its syntax trees with a stack of its + * own (L4 of the EPIC #1731). Ruby that prism cannot parse for its depth is + * told, and does not stop the editor. + */ +import { dnclToRuby } from '../../../../src/lib/dncl/dncl-to-ruby'; +import { rubyToDncl } from '../../../../src/lib/dncl/ruby-to-dncl'; +import { loadPrism } from '../../../../src/lib/prism-parser'; + +beforeAll(() => loadPrism()); + +const repeat = (n, f) => Array.from({ length: n }, (_, i) => f(i)); + +describe('long and deep programs (L4 of the EPIC #1731)', () => { + test('an expression of 5000 terms becomes Ruby', () => { + const { ruby, errors } = dnclToRuby(`x = ${repeat(5000, () => '1').join(' + ')}`); + expect(errors).toEqual([]); + expect(ruby).toBe(`@x = ${repeat(5000, () => '1').join(' + ')}`); + }); + + test('5000 parentheses in parentheses become Ruby', () => { + expect(dnclToRuby(`x = ${'('.repeat(5000)}1${')'.repeat(5000)}`).ruby).toBe('@x = 1'); + }); + + test('blocks 500 deep become Ruby', () => { + const dncl = [...repeat(500, (i) => `${' '.repeat(i)}もし x > 0 ならば:`), `${' '.repeat(500)}x = 1`]; + const { ruby, errors } = dnclToRuby(dncl.join('\n')); + expect(errors).toEqual([]); + expect(ruby.split('\n')).toHaveLength(1001); + }); + + test('Ruby of 1000 terms becomes Japanese', () => { + const { dncl, errors } = rubyToDncl(`@x = ${repeat(1000, () => '1').join(' + ')}`); + expect(errors).toEqual([]); + expect(dncl).toBe(`x = ${repeat(1000, () => '1').join(' + ')}`); + }); +}); diff --git a/packages/scratch-gui/test/unit/lib/dncl/dncl-number-notation.test.js b/packages/scratch-gui/test/unit/lib/dncl/dncl-number-notation.test.js index b48ba62ed3..5d773e5b73 100644 --- a/packages/scratch-gui/test/unit/lib/dncl/dncl-number-notation.test.js +++ b/packages/scratch-gui/test/unit/lib/dncl/dncl-number-notation.test.js @@ -170,8 +170,13 @@ describe('DNCL → Ruby: full-width digits and a name right after a number (issu else expect(result.ruby).not.toMatch(/[0-9]/); }); - test('full-width digits in a name stay in the name', () => { - expect(dnclToRuby('x1 = 1').ruby).toBe('@x1 = 1'); + // The full-width letters and digits are made half-width, in a name too (W8 of the EPIC #1731). + test('full-width letters and digits in a name are read as the half-width ones', () => { + expect(dnclToRuby('x1 = 1\nx1 = x1 + 1').ruby).toBe('@x1 = 1\n@x1 = @x1 + 1'); + }); + + test('full-width letters and digits in a string and a comment are kept', () => { + expect(dnclToRuby('表示する("x1") # x1 を表示する').ruby).toBe('console.puts("x1") # x1 を表示する'); }); test.each([ diff --git a/packages/scratch-gui/test/unit/lib/dncl/dncl-stage-variable.test.js b/packages/scratch-gui/test/unit/lib/dncl/dncl-stage-variable.test.js new file mode 100644 index 0000000000..8a298fc726 --- /dev/null +++ b/packages/scratch-gui/test/unit/lib/dncl/dncl-stage-variable.test.js @@ -0,0 +1,39 @@ +/** + * The Japanese mode has no stage (Issue #1706): it is written for a program + * that runs alone, as on a terminal. Ruby → Japanese does not write a variable + * or a list of the stage (`$a`) and tells it where it is, so the student is + * not given Japanese that cannot be turned back into Ruby. + */ +import { STAGE_VARIABLE_ERROR_ID } from '../../../../src/lib/dncl/dncl-validator'; +import { rubyToDncl } from '../../../../src/lib/dncl/ruby-to-dncl'; +import { loadPrism } from '../../../../src/lib/prism-parser'; + +beforeAll(() => loadPrism()); + +const lines = (...ls) => ls.join('\n'); + +const told = (line, column, name) => ({ + line, + column, + id: STAGE_VARIABLE_ERROR_ID, + name, + message: `ステージの変数(${name})は日本語モードでは書けません。Ruby モードでは書けます。`, +}); + +describe('Ruby → DNCL: the variables of the stage (issue #1706)', () => { + test('a variable of the stage is told where it is written and read', () => { + expect(rubyToDncl(lines('$a = 1', 'say($a)')).errors).toEqual([told(1, 1, '$a'), told(2, 5, '$a')]); + }); + + test('a list of the stage is told, not the command it calls', () => { + expect(rubyToDncl(lines('$a = []', '$a.push(1)')).errors).toEqual([told(1, 1, '$a'), told(2, 1, '$a')]); + }); + + test('a variable of the stage changed by an operator is told at its name', () => { + expect(rubyToDncl('$score += 1').errors).toEqual([told(1, 1, '$score')]); + }); + + test('a variable of the sprite is still written in the Japanese', () => { + expect(rubyToDncl(lines('@a = 1', 'say(@a)'))).toEqual({ dncl: lines('a = 1', '表示する(a)'), errors: [] }); + }); +}); diff --git a/packages/scratch-gui/test/unit/lib/dncl/dncl-validator.test.js b/packages/scratch-gui/test/unit/lib/dncl/dncl-validator.test.js index e389552386..3acc371b6f 100644 --- a/packages/scratch-gui/test/unit/lib/dncl/dncl-validator.test.js +++ b/packages/scratch-gui/test/unit/lib/dncl/dncl-validator.test.js @@ -79,9 +79,39 @@ describe('block errors of the DNCL-like form', () => { ]); }); - test('a wrong closer that still makes good Ruby is not an error', () => { - // The editor takes it (every closer becomes `end`), so it is left alone. - expect(errorsOf(['もし x > 0 ならば', ' 表示する(x)', 'を繰り返す'])).toEqual([]); + // A closer of another kind of block is an error even when the Ruby would be good (W3 of #1731). + test('a wrong closer is reported with the closer of the block', () => { + expect(errorsOf(['もし x > 0 ならば', ' 表示する(x)', 'を繰り返す'])).toEqual([ + { + line: 3, + column: 1, + message: '「もし〜ならば」は「を繰り返す」ではなく「を実行する」で閉じてください', + }, + ]); + }); + + test('a counting loop closed with another closer is reported', () => { + expect(errorsOf(['i を 1 から 3 まで 1 ずつ増やしながら', ' 表示する(i)', 'を実行する'])).toEqual([ + { + line: 3, + column: 1, + message: '「〜ずつ増やしながら」は「を実行する」ではなく「を繰り返す」で閉じてください', + }, + ]); + }); + + test('a block opened with : closed by a line of another kind names the standard form', () => { + expect(errorsOf(['x < 3 の間繰り返す:', ' x = x + 1', 'と定義する'])).toEqual([ + { + line: 3, + column: 1, + message: '「〜の間繰り返す:」は「と定義する」ではなく「を繰り返す」で閉じてください', + }, + ]); + }); + + test('end still closes any kind of block', () => { + expect(errorsOf(['x < 3 の間', ' x ← x + 1', 'end'])).toEqual([]); }); test('そうでなければ inside a loop of an if left open names the if', () => { diff --git a/packages/scratch-gui/test/unit/lib/ja-notation/build-and-write-ruby.test.js b/packages/scratch-gui/test/unit/lib/ja-notation/build-and-write-ruby.test.js index ffb6ff1d4d..2b9fff3b1a 100644 --- a/packages/scratch-gui/test/unit/lib/ja-notation/build-and-write-ruby.test.js +++ b/packages/scratch-gui/test/unit/lib/ja-notation/build-and-write-ruby.test.js @@ -63,10 +63,11 @@ describe('jaToRuby with the table of Smalruby', () => { // The current conversion adds the step only at `を繰り返す`, so a counting // loop closed by `を実行する` never ends there; here every closing line adds it. - test('a counting loop closed by another closing word adds the step too', () => { - expect(smalruby('i を 1 から 3 まで 1 ずつ増やしながら\n 表示する(i)\nを実行する').ruby).toBe( - '@i = 1\nwhile @i <= 3\n console.puts(@i)\n @i += 1\nend', - ); + // The closing word must be the one of the block (W3 of the EPIC #1731). + test('a counting loop closed by another closing word is an error', () => { + const result = smalruby('i を 1 から 3 まで 1 ずつ増やしながら\n 表示する(i)\nを実行する'); + expect(result.ruby).toBeNull(); + expect(result.errors).toMatchObject([{ block: 'closer-kind', line: 3, expected: 'を繰り返す' }]); }); test.each([