Skip to content
Merged
37 changes: 23 additions & 14 deletions docs/dncl/README.md

Large diffs are not rendered by default.

9 changes: 9 additions & 0 deletions docs/dncl/design.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,6 +148,15 @@ Ruby や日本語のプログラムの `表示する` は、キャラクター
Ruby タブは Scratch の部分集合(Scratch でできることを Ruby で書く)なので、コンソールのブロックを Ruby で書く形は足さない。
- 将来の案: Ruby タブに「コンソールモード」のような切りかえを置き、そのときだけ `puts` / `gets` がコンソールを使う。いまは作らない。

### ステージの変数は書かない

- 日本語モードは、**ターミナルで動かすような、1 つで動くプログラム**を想定する。上のとおりステージはコンソールになり、
ステージという考え方はできるだけ薄くする。ステージの変数・リストや、ほかのスプライトの変数は使わない。
- そのため、Ruby の `$a`(ステージの変数・リスト)は日本語にしない。Ruby → 日本語では、`$a` の書いてある位置に
「ステージの変数($a)は日本語モードでは書けません。Ruby モードでは書けます。」の誤りを出す(ほかの日本語にできない Ruby と同じく、
日本語に切りかえず Ruby タブにとどまる)。日本語の変数は、いつもそのスプライトの変数(Ruby の `@a`)になる。
- ステージの変数を使うプログラムは Ruby タブで書く。

### 要素の追加

- 日本語の `A に 値 を追加する` で書けるようにする。
Expand Down
2 changes: 1 addition & 1 deletion docs/dncl/policy.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@
| 種類 | 扱い | 例 |
|---|---|---|
| **意味が変わるもの、または問題文で約束されるもの** | **設定にする**。選んだ設定はプログラムの冒頭に説明文として書く | 添字の始まり(0 から / 1 から)、コメントの記号(`#` か `//` か。`//` をコメントにしたときは、`//` を整数の割り算に使えない)、論理演算子の書き方 |
| **意味が変わらない、見た目だけの揺れ** | **設定にしない**。読むとき(貼り付けなど)はどの書き方も受け付ける。書くとき(ブロックや Ruby から日本語にするとき)は 1 つにそろえる | 行番号の形(`(01)` / `(1)`)、罫線の文字(`|` `⎿` / `|` `└`)、`:` と `:`、全角と半角の演算子(`+` `%` `<`)、引き算の記号(`-` / `−` / `–`)、`,` の後の空白 |
| **意味が変わらない、見た目だけの揺れ** | **設定にしない**。読むとき(貼り付けなど)はどの書き方も受け付ける。書くとき(ブロックや Ruby から日本語にするとき)は 1 つにそろえる | 行番号の形(`(01)` / `(1)`)、罫線の文字(`|` `⎿` / `|` `└`)、`:` と `:`、全角と半角の演算子(`+` `%` `<`)、文字列とコメントの外の全角英字・全角数字(`x1` `12`)、引き算の記号(`-` / `−` / `–`)、`,` の後の空白 |

- 書くときにそろえる 1 つは、上の「文法の決め方」で決める(新しい年度の問題に合わせる)。
- 専門的でない受験者には細かい設定が難しいので、**よくある設定のパターンを用意**する。
Expand Down
7 changes: 5 additions & 2 deletions docs/smalruby-dncl-spec.ja.md
Original file line number Diff line number Diff line change
Expand Up @@ -151,8 +151,8 @@ c = "「く」がある" # → "「く」がある" のまま
### 識別子に使える文字

- ASCII 英字(`a-z`, `A-Z`)、アンダースコア(`_`)、2 文字目からは数字(`0-9`)
- ひらがな・カタカナ・漢字(CJK 拡張の漢字も)と、`々` `〆` `ヶ` `ー` など名前に使う文字。全角英字・ギリシャ文字など、ほかの文字体系の文字も使えます(`佐々木`、`α`)
- 全角英字は半角とは別の名前です(`x` と `x` は別の変数)
- ひらがな・カタカナ・漢字(CJK 拡張の漢字も)と、`々` `〆` `ヶ` `ー` など名前に使う文字。ギリシャ文字など、ほかの文字体系の文字も使えます(`佐々木`、`α`)
- 全角英字・全角数字は半角として読みます(`x1` は `x1` と同じ変数)。文字列とコメントの中は書いたとおりです
- `☆` `①` のような記号は使えません

範囲の詳細は `docs/dncl/README.md` の「名前に使える文字」を参照してください。
Expand Down Expand Up @@ -382,6 +382,8 @@ DNCL 風の形の終端キーワードも読みます(読み込みだけ)。
| `を繰り返す` | `の間`(while ループ)、`ずつ増やしながら/減らしながら`(for ループ) |
| `と定義する` | `関数〜`(関数定義) |

ブロックの種類に合わない終端キーワードで閉じると誤りです(`もし〜ならば` を `を繰り返す` で閉じるなど。「「もし〜ならば」は「を繰り返す」ではなく「を実行する」で閉じてください」)。`end` はどのブロックも閉じます。

## 7. 関数

### 関数定義
Expand Down Expand Up @@ -652,6 +654,7 @@ Monaco エディタで以下の要素がハイライトされます:
| 等価比較 | `==`(条件式の中の `=` も比較として読むが、`==` を推奨) |
| 割り算 | `/`(小数になる割り算)、`÷`(商の整数値)。`//` も商の整数値として使える(コメントの記号を `//` にしたときを除く)。負の数の商は小さいほうへ丸める(`-7 ÷ 2` は `-4`) |
| 全角の記号 | `%` `≦` `≧` `≠` `+` などの全角の演算子、`:` `,` も読む。書き出しは半角(余りだけ `%`) |
| 全角英字・全角数字 | 文字列とコメントの外では半角として読む(`x1` は `x1`、`12` は `12`) |
| 入力 | `【外部からの入力】`(答えをそのまま使う)と `【整数を入力】`(整数にする) |
| 要素の追加 | `A に 値 を追加する`(最後に足す。例題に無いので、方針の決め方の ⑤ で決めた) |
| 戻り値 | `値 を返す`(例題に無いので、方針の決め方の ⑤ で決めた) |
Expand Down
71 changes: 52 additions & 19 deletions packages/ja-notation/src/lexer.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,18 +4,19 @@

/**
* The first character of a name: an ASCII letter, `_`, or a letter of any
* script (`\p{L}`: kana, kanji with the CJK extensions, `々`, `〆`, `ヶ`, full-width
* letters, Greek, ...), a letter number (`\p{Nl}`: `〇`, `Ⅱ`), and the marks of
* the kana blocks (`゛`, `・`). Ruby takes any of them in a name, and Ruby →
* script (`\p{L}`: kana, kanji with the CJK extensions, `々`, `〆`, `ヶ`,
* Greek, ...), a letter number (`\p{Nl}`: `〇`, `Ⅱ`), and the marks of the
* kana blocks (`゛`, `・`). Ruby takes any of them in a name, and Ruby →
* Japanese writes the name as it is (`@佐々木` → `佐々木`), so the Japanese reads
* back every name it writes (Issue #1714). A full-width letter is a name of its
* own (`x` is not `x`): Ruby has them as two variables.
* back every name it writes (Issue #1714). A full-width letter or digit out of
* a string and a comment is read as the half-width one before (`x1` is
* `x1`; see `halfWidthAlphanumerics`), as the auto-correct of the Ruby mode does.
*/
const NAME_HEAD_RE = /[A-Za-z_\p{L}\p{Nl}\u3040-\u309F\u30A0-\u30FF]/u;

/**
* A character after the first one of a name: the same, the digits (`\p{Nd}`,
* full-width ones too) and the combining marks (`\p{M}`: `か` and `゙`).
* A character after the first one of a name: the same, the digits (`\p{Nd}`)
* and the combining marks (`\p{M}`: `か` and `゙`).
*/
const NAME_TAIL_RE = /[A-Za-z0-9_\p{L}\p{Nl}\p{Nd}\p{M}\u3040-\u309F\u30A0-\u30FF]/u;

Expand Down Expand Up @@ -129,20 +130,33 @@ const stringEnd = (text, start) => {
return -1;
};

/** A digit (the full-width ones are made half-width before, see `halfWidthAlphanumerics`). */
const DIGIT_RE = /[0-9]/;

/** The full-width letters and digits (`A`–`Z`, `a`–`z`, `0`–`9`). */
const FULL_WIDTH_ALPHANUMERIC_RE = /[A-Za-z0-9]/g;

/**
* A digit, half-width or full-width (`12` is `12`): only how it looks differs, so a number
* written with full-width digits is read as the number (docs/dncl/policy.md: 見た目だけの揺れ).
* In a name (`x1`) a full-width digit stays a part of the name, as Ruby has it.
* The text with its full-width letters and digits made half-width (`x1 = 12` →
* `x1 = 12`). Only how they look differs (docs/dncl/policy.md: 見た目だけの揺れ), and
* the Ruby mode makes them half-width too (its auto-correct), so the Japanese reads
* them as the half-width ones: `x` is the name `x`, `if` would be `if`. Each is one
* UTF-16 unit, as the half-width one, so the columns stay the same.
* @param {string} text - The code (no string and no comment in it).
* @returns {string} The text.
*/
const DIGIT_RE = /[0-90-9]/;
const halfWidthAlphanumerics = (text) =>
text.replace(FULL_WIDTH_ALPHANUMERIC_RE, (ch) => String.fromCharCode(ch.charCodeAt(0) - 0xfee0));

/**
* The number as the Ruby writes it: the full-width digits and `.` as half-width ones.
* @param {string} text - The number as written.
* The number as the Ruby writes it: `.` as `.`.
* @param {string} text - The number as written (its digits are half-width).
* @returns {string} The number.
*/
const halfWidthNumber = (text) =>
text.replace(/[0-9.]/g, (ch) => (ch === '.' ? '.' : String.fromCharCode(ch.charCodeAt(0) - 0xfee0)));
const halfWidthNumber = (text) => text.replace(/./g, '.');

/** The characters that start a string or a comment, which are not made half-width. */
const STRING_OR_COMMENT_RE = /["'「“##]|\/\//g;

/**
* The end of a number that starts at `text[start]`, and whether it is written
Expand Down Expand Up @@ -185,7 +199,10 @@ const longOperatorAt = (text, i) => LONG_OPERATORS.find((op) => text.startsWith(

/**
* Read one line of code into tokens. A token is `{type, text, column}` and,
* for some types, `value`:
* for some types, `value`. The full-width letters and digits out of the strings
* and the comments are read as the half-width ones (`halfWidthAlphanumerics`):
* the `text` of a name or a number has the half-width ones. The strings and the
* comments are kept as they are written.
*
* - `number` (`value`: the number as written, with half-width digits), `badNumber`, `string` (`text`
* is the literal with its quotes), `name`, `op` (`value`: the operator, a
Expand All @@ -198,15 +215,27 @@ const longOperatorAt = (text, i) => LONG_OPERATORS.find((op) => text.startsWith(
* - `forbidden`: `@` or `$`, which Japanese names do not have.
* - `unknown`: any other character, and a string that is not closed.
*
* `column` is the 0-based index in `text` plus `offset`.
* @param {string} text - The code of the line (without its line number and indent).
* `column` is the 0-based index in `source` plus `offset`.
* @param {string} source - The code of the line (without its line number and indent).
* @param {number} [offset] - What to add to each column.
* @param {string} [commentMarker] - The comment mark of the settings (`#` or
* `//`). With `//`, `//` starts a comment (and `#` still does) instead of
* being the integer quotient.
* @returns {Array<object>} The tokens.
*/
const tokenize = (text, offset = 0, commentMarker = '#') => {
const tokenize = (source, offset = 0, commentMarker = '#') => {
let text = source;
// The code is made half-width up to the next string or comment, each time the reading gets there.
let halfWidthTo = 0;
const makeHalfWidthFrom = (start) => {
STRING_OR_COMMENT_RE.lastIndex = start + 1;
const next = STRING_OR_COMMENT_RE.exec(text);
halfWidthTo = next ? next.index : text.length;
text =
text.substring(0, start) +
halfWidthAlphanumerics(text.substring(start, halfWidthTo)) +
text.substring(halfWidthTo);
};
const tokens = [];
const push = (type, start, end, extra = {}) =>
tokens.push({ type, text: text.substring(start, end), column: start + offset, ...extra });
Expand All @@ -217,6 +246,10 @@ const tokenize = (text, offset = 0, commentMarker = '#') => {
i++;
continue;
}
if (i >= halfWidthTo && !/["'「“##]/.test(ch) && !text.startsWith('//', i)) {
makeHalfWidthFrom(i);
continue;
}
if (ch === '#' || ch === '#') {
// A full-width `#` is read as `#` (only how it looks differs).
tokens.push({ type: 'comment', text: `#${text.substring(i + 1)}`, column: i + offset });
Expand Down
76 changes: 67 additions & 9 deletions packages/ja-notation/src/parse.js
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,8 @@ const STOP_NAMES = new Set([
* @param {object} [detail] - What a caller may need to say the error in its own
* words: `name` (the token), `opener` / `direction` / `colon` / `word` (the
* block of a block error), `block` (`stray-closer`, `closer-indent`,
* `stray-branch`) and `branch`.
* `closer-kind`, `stray-branch`), `expected` (the closing word of the block
* of a `closer-kind`) and `branch`.
* @returns {object} The error.
*/
const makeError = (code, message, line, column = 0, detail = {}) => ({ code, message, line, column, ...detail });
Expand Down Expand Up @@ -914,6 +915,33 @@ const tokenError = (tokens) => {
return null;
};

/**
* The closing word of a block in the DNCL-like form.
* @param {object} node - The block (`If`, `While`, `For`, `FunctionDef`).
* @returns {string} `を実行する`, `を繰り返す` or `と定義する`.
*/
const closerOf = (node) => {
if (node.type === 'If') return 'を実行する';
if (node.type === 'FunctionDef') return 'と定義する';
return 'を繰り返す';
};

/**
* Where in the open blocks is the nearest one a closing word closes, when only
* blocks closed by a line of their own are inside it (so the word closes it and
* the blocks inside it were forgotten), or -1.
* @param {Array<object>} stack - The open blocks (the innermost last).
* @param {string} word - The closing word.
* @returns {number} The index in the stack, or -1.
*/
const matchingCloser = (stack, word) => {
for (let i = stack.length - 1; i >= 0; i--) {
if (stack[i].mode !== 'closer') return -1;
if (closerOf(stack[i].node) === word) return i;
}
return -1;
};

/**
* Read the Japanese (the standard form, the DNCL-like form or both mixed)
* into a JaAST.
Expand Down Expand Up @@ -947,6 +975,16 @@ const parse = (source) => {
...(entry.node.type === 'For' ? { direction: entry.node.direction } : {}),
colon: entry.colon,
});
// A block closed by a line of its own that was not closed.
const unclosed = (entry) => {
const word = closerOf(entry.node);
errors.push(
makeError('unclosed', `「${word}」で閉じていません`, entry.node.line, entry.node.column, {
...openerOf(entry),
word,
}),
);
};
const emptyBlock = (entry) =>
errors.push(
makeError(
Expand Down Expand Up @@ -1021,6 +1059,33 @@ const parse = (source) => {
);
return;
}
// The closing word must be the one of the block (Issue #1731, W3): a
// `もし` closed by `を繰り返す` is told. `end` closes any block, as in Ruby.
if (block.word !== END_WORD && block.word !== closerOf(top.node)) {
// A block forgotten to be closed inside the one the word closes (a
// `もし` in a loop closed by `を繰り返す`) is told as not closed.
const index = matchingCloser(stack, block.word);
if (index < 0) {
stack.pop();
const expected = closerOf(top.node);
errors.push(
makeError(
'syntax',
`「${block.word}」ではなく「${expected}」で閉じてください`,
entry.line,
column,
{ ...openerOf(top), block: 'closer-kind', word: block.word, expected },
),
);
return;
}
while (stack.length > index + 1) unclosed(stack.pop());
const found = stack.pop();
close(found, entry.line, entry.line);
found.node.closeOwnLine = true;
lastCodeLine = entry.line;
return;
}
stack.pop();
close(top, entry.line, entry.line);
top.node.closeOwnLine = true;
Expand Down Expand Up @@ -1081,14 +1146,7 @@ const parse = (source) => {
} else if (top.mode === 'indent') {
emptyBlock(top);
} else {
const word =
top.node.type === 'If' ? 'を実行する' : top.node.type === 'FunctionDef' ? 'と定義する' : 'を繰り返す';
errors.push(
makeError('unclosed', `「${word}」で閉じていません`, top.node.line, top.node.column, {
...openerOf(top),
word,
}),
);
unclosed(top);
}
}
return errors.length > 0 ? { program: null, errors } : { program, errors };
Expand Down
Loading
Loading