Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .github/workflows/changelog.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,9 @@ jobs:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
- uses: actions/setup-node@v7
with:
cache: yarn
node-version: 22
- run: node --test scripts/changelog.test.js
# The scripts import semver. Package builds aren't needed for them.
- run: yarn install --immutable --mode=skip-build
- run: node --test scripts/changelog.test.js scripts/magicNumbers.test.js
Comment thread
cknitt marked this conversation as resolved.
- run: node scripts/changelog.js check
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -584,7 +584,7 @@ The repository's `npm-release` GitHub environment should have required reviewers
To build a new version and release it on NPM, follow these steps:

1. Verify that the version number is already set correctly for the release. (It should have been incremented after releasing the previous version.)
1. Run `yarn changelog:preview` to review pending notes, then `yarn changelog:release` to collect fragments into `CHANGELOG.md`, remove "(Unreleased)", and delete consumed fragments. Create a PR containing these changes, review the combined release notes, and apply the `changelog:release` label. Existing unreleased entries are preserved. If more fragments are merged before publishing, the `Publish` workflow refuses to run; repeat this step to add them to the same version. Don't run `yarn changelog:release` while a `Publish` run is in progress: it could add notes for changes that the run is not publishing.
1. Run `yarn changelog:preview` to review pending notes, then `yarn changelog:release` to collect fragments into `CHANGELOG.md`, remove "(Unreleased)", and delete consumed fragments. It also bumps the `.cmi`, `.cmt` and `.ast` magic numbers in `compiler/ext/config.ml` when their sources changed since the previous release tag, so fetch the tags first (`git fetch --tags`). Create a PR containing these changes, review the combined release notes, and apply the `changelog:release` label. Existing unreleased entries are preserved. If more fragments are merged before publishing, the `Publish` workflow refuses to run; repeat this step to add them to the same version. Don't run `yarn changelog:release` while a `Publish` run is in progress: it could add notes for changes that the run is not publishing.
1. Once that PR is merged and built successfully, run the `Publish` workflow from the GitHub Actions tab. Use the "Run workflow" branch dropdown to select the release branch (normally `master`, or the relevant maintenance branch) and indicate whether this is the current stable release. The version is read from that branch's `package.json`, and the npm tag is derived automatically:
- A current stable release is published as `latest`.
- A stable maintenance release for an older major is published as `latest-<major>`, for example `latest-12`.
Expand Down
2 changes: 1 addition & 1 deletion changes/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ A bullet needs no PR link: `yarn changelog:release` appends the link of the pull

`yarn changelog:preview` prints pending fragments grouped by category, newest PR first, without changing files. It does not include entries already in `CHANGELOG.md`.

During release preparation, run `yarn changelog:release`. It verifies that the first version heading matches `package.json` and is unreleased, validates every fragment before writing, adds fragments above the existing entries of each section, newest PR first, removes `(Unreleased)`, and deletes consumed files. Review and commit all changes in the release PR. Historical entries and existing unreleased notes are preserved. Releases with no fragments can still finalize existing notes. The publish workflow refuses to release while fragments remain in `changes/`. If fragments are merged after the release PR but before publishing, run `yarn changelog:release` again: as long as Publish has not created the version's `v<version>` tag, it adds them to the finalized version. Once the tag exists, it refuses, and the fragments wait for the next version after `yarn changelog:open`. With no fragments left, running it again fails. Don't run it while a Publish run is in progress, because the tag only appears once that run gets to tagging, and the notes could then describe changes the run is not publishing.
During release preparation, run `yarn changelog:release`. It verifies that the first version heading matches `package.json` and is unreleased, validates every fragment before writing, adds fragments above the existing entries of each section, newest PR first, removes `(Unreleased)`, and deletes consumed files. It also bumps the magic numbers in `compiler/ext/config.ml` whose sources changed since the previous release tag, unless they already differ from that tag, so running it again does not bump them twice. CI rejects pull requests without the `changelog:release` label that bump one. Review and commit all changes in the release PR. Historical entries and existing unreleased notes are preserved. Releases with no fragments can still finalize existing notes. The publish workflow refuses to release while fragments remain in `changes/`. If fragments are merged after the release PR but before publishing, run `yarn changelog:release` again: as long as Publish has not created the version's `v<version>` tag, it adds them to the finalized version. Once the tag exists, it refuses, and the fragments wait for the next version after `yarn changelog:open`. With no fragments left, running it again fails. Don't run it while a Publish run is in progress, because the tag only appears once that run gets to tagging, and the notes could then describe changes the run is not publishing.

After publishing, run `yarn changelog:open` once `package.json` has the next version. It adds an empty unreleased section with every category heading.

Expand Down
1 change: 1 addition & 0 deletions changes/release-magic-numbers.breaking.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
- Give `.cmi` and `.cmt` files ReScript magic numbers (`ResCmi…`, `ResCmt…`) instead of OCaml-style ones, and bump all magic numbers at release time instead of in pull requests.
2 changes: 2 additions & 0 deletions compiler/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ commands below are relative to the repository root unless stated otherwise.
- Add a source fixture exercising the new syntax to `tests/syntax_tests/data/ast-mapping/`. Every file there is run through `res_parser -test-ast-conversion` (which round-trips the parsetree through the frozen v0 AST before printing) as part of `make test-syntax`; the printed output must match the snapshot in its `expected/` directory.
- For exact-identity invariants (locations, attributes) or the v0 wire shape itself, add cases to `tests/ounit_tests/ounit_ast_mapper0_tests.ml`, which tests `ast_mapper_to0`/`ast_mapper_from0` directly.

- **Don't bump magic numbers** - Changing a serialized type (parsetree, typed tree, `.cmi` signature) needs no edit of the magic numbers in `compiler/ext/config.ml`. `yarn changelog:release` bumps them when `compiler/ml`, `compiler/ext` or `compiler/flow_parser` changed since the previous release, and the changelog check rejects pull requests that bump them.

- **Missing test coverage** - Always add tests for syntax, lambda, and end-to-end behavior

- **Test early and often** - Add tests immediately after modifying each compiler layer to catch problems early, rather than waiting until all changes are complete
Expand Down
9 changes: 7 additions & 2 deletions compiler/ext/config.ml
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
let cmi_magic_number = "Caml1999I040"
(* Pull requests never change the magic numbers below: [yarn changelog:release]
bumps their trailing counters when the format sources changed since the
previous release (see scripts/magicNumbers.js). The .cmi and .cmt numbers
must have the same length, since a .cmt reader compares its first bytes with
both. *)
let cmi_magic_number = "ResCmi040"

(* Magic numbers for marshaled values of the *current* parsetree, whose layout
changes across compiler versions. *)
Expand All @@ -13,7 +18,7 @@ and ast0_impl_magic_number = "Caml1999M022"

and ast0_intf_magic_number = "Caml1999N022"

and cmt_magic_number = "Caml1999T048"
and cmt_magic_number = "ResCmt048"

let get_load_path () = (Compiler_request_state.current ()).load_path
let set_load_path path = (Compiler_request_state.current ()).load_path <- path
9 changes: 7 additions & 2 deletions compiler/ml/cmi_format.ml
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,9 @@ let input_cmi ic =
let flags = input_value ic in
{cmi_name = name; cmi_sign = sign; cmi_crcs = crcs; cmi_flags = flags}

(* Earlier compilers wrote OCaml-style magic numbers *)
let legacy_magic_prefix = "Caml1999I"

let read_cmi_channel filename ic =
try
let buffer =
Expand All @@ -50,6 +53,8 @@ let read_cmi_channel filename ic =
if buffer < Config.cmi_magic_number then "an older" else "a newer"
in
raise (Error (Wrong_version_interface (filename, msg)))
else if String.starts_with ~prefix:legacy_magic_prefix buffer then
raise (Error (Wrong_version_interface (filename, "an older")))
else raise (Error (Not_an_interface filename)));
let cmi = input_cmi ic in
cmi
Expand Down Expand Up @@ -111,8 +116,8 @@ let report_error ppf = function
filename
| Wrong_version_interface (filename, older_newer) ->
fprintf ppf
"%a@ is not a compiled interface for this version of OCaml.@.It seems to \
be for %s version of OCaml."
"%a@ is not a compiled interface for this version of ReScript.@.It seems \
to be for %s version of ReScript."
Location.print_filename filename older_newer
| Corrupted_interface filename ->
fprintf ppf "Corrupted compiled interface@ %a" Location.print_filename
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@
"changelog:preview": "node scripts/changelog.js preview",
"changelog:release": "node scripts/changelog.js release",
"changelog:open": "node scripts/changelog.js open",
"test-changelog": "node --test scripts/changelog.test.js",
"test-changelog": "node --test scripts/changelog.test.js scripts/magicNumbers.test.js",
"test-commit-records": "node --test scripts/commitRecords.test.js scripts/recordMarkdown.test.js"
},
"files": [
Expand Down
22 changes: 18 additions & 4 deletions scripts/changelog.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,12 @@ import { execFileSync } from "node:child_process";
import fs from "node:fs/promises";
import path from "node:path";
import { fileURLToPath } from "node:url";
import {
checkMagicNumbers,
configPath,
releaseMagicNumbers,
showAt,
} from "./magicNumbers.js";

export const categories = {
breaking: "#### :boom: Breaking Change",
Expand Down Expand Up @@ -280,6 +286,7 @@ export async function run(
command,
root = process.cwd(),
pullRequestOf = name => addedByPullRequest(root, name),
magicNumbersFor = version => releaseMagicNumbers(root, version),
) {
if (!["check", "preview", "release", "open"].includes(command)) {
throw new Error(
Expand Down Expand Up @@ -308,6 +315,11 @@ export async function run(
files.push({ status: diff[i], name: diff[i + 1] });
}
checkPullRequest(files, pullRequest);
checkMagicNumbers(
showAt(root, pullRequest.base.sha, configPath),
await fs.readFile(path.join(root, configPath), "utf8"),
pullRequest.labels.map(label => label.name),
);
} else if (process.env.GITHUB_EVENT_NAME === "push") {
// Once merged, every fragment must resolve to its pull request, so a
// merge without "(#N)" in its subject fails here rather than at release.
Expand Down Expand Up @@ -339,10 +351,12 @@ export async function run(
linked.length > 0 &&
isFinalized(changelog, version) &&
releaseTagExists(repository.url, version);
await fs.writeFile(
target,
prepareRelease(changelog, version, linked, { tagged }),
);
const released = prepareRelease(changelog, version, linked, { tagged });
// The release bumps the magic numbers of the formats that changed, so that
// pull requests never touch them.
const config = await magicNumbersFor(version);
await fs.writeFile(target, released);
if (config !== null) await fs.writeFile(path.join(root, configPath), config);
for (const fragment of fragments) {
await fs.unlink(path.join(root, "changes", fragment.name));
}
Expand Down
25 changes: 20 additions & 5 deletions scripts/changelog.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ import {
const note =
"- Fix recovery. https://github.com/rescript-lang/rescript/pull/8667";
const historical = "# 12.0.0\n\nOld notes.\n";
// Bumping the magic numbers needs a checkout of the repository.
const noMagicNumbers = async () => null;
const changelog = `# Changelog\n\n# 13.0.0 (Unreleased)\n\n${Object.values(categories).join("\n\n")}\n\n${historical}`;

test("validates category names and bullet content", () => {
Expand Down Expand Up @@ -203,7 +205,7 @@ test("validates before mutation, sorts fragments, consumes only on release", asy
await write("z.fix.md", note);
await write("a.fix.md", note.replace("recovery", "formatting"));
await write("bad.unknown.md", note);
await assert.rejects(run("release", root));
await assert.rejects(run("release", root, undefined, noMagicNumbers));
assert.equal(
await fs.readFile(path.join(root, "CHANGELOG.md"), "utf8"),
changelog,
Expand All @@ -216,9 +218,9 @@ test("validates before mutation, sorts fragments, consumes only on release", asy
(await readFragments(root)).map(fragment => fragment.name),
["a.fix.md", "z.fix.md"],
);
await run("release", root);
await run("release", root, undefined, noMagicNumbers);
assert.deepEqual(await fs.readdir(path.join(root, "changes")), ["README.md"]);
await assert.rejects(run("release", root));
await assert.rejects(run("release", root, undefined, noMagicNumbers));
});

test("links bullets without a link to the pull request that added them", () => {
Expand Down Expand Up @@ -285,15 +287,28 @@ test("releases fragments without a link with their pull request's link", async t
await fs.writeFile(path.join(root, "changes", "lost.fix.md"), "- Lost.");
const numbers = { "new.fix.md": 9000, "lost.fix.md": null };
await assert.rejects(
run("release", root, name => numbers[name]),
run("release", root, name => numbers[name], noMagicNumbers),
/changes\/lost\.fix\.md/,
);
assert.equal(
await fs.readFile(path.join(root, "CHANGELOG.md"), "utf8"),
changelog,
);
await fs.unlink(path.join(root, "changes", "lost.fix.md"));
await run("release", root, name => numbers[name]);
await fs.mkdir(path.join(root, "compiler", "ext"), { recursive: true });
await run(
"release",
root,
name => numbers[name],
async version => {
assert.equal(version, "13.0.0");
return "bumped";
},
);
assert.equal(
await fs.readFile(path.join(root, "compiler", "ext", "config.ml"), "utf8"),
"bumped",
);
assert.ok(
(await fs.readFile(path.join(root, "CHANGELOG.md"), "utf8")).includes(
`${categories.fix}\n\n- New fix. https://github.com/rescript-lang/rescript/pull/9000\n${note}\n`,
Expand Down
144 changes: 144 additions & 0 deletions scripts/magicNumbers.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,144 @@
import { spawnSync } from "node:child_process";
import fs from "node:fs/promises";
import path from "node:path";
import semver from "semver";

export const configPath = "compiler/ext/config.ml";

// The libraries that define the serialized parsetree and typed tree types,
// and everything those types refer to: `ml` and the libraries it depends on.
export const formatSources = [
"compiler/ml",
"compiler/ext",
"compiler/flow_parser",
];

export const magicNumbers = [
"cmi_magic_number",
"cmt_magic_number",
"ast_impl_magic_number",
"ast_intf_magic_number",
];

const definition = name => new RegExp(`\\b${name} = "([^"]*)"`);

export function readMagicNumber(config, name) {
return definition(name).exec(config)?.[1] ?? null;
}

// Increments the trailing counter and keeps its width: ResCmi040 -> ResCmi041.
export function incrementMagicNumber(value) {
const match = /^(.*?)(\d+)$/.exec(value);
if (!match) throw new Error(`${value}: expected a trailing counter`);
const next = String(Number(match[2]) + 1);
if (next.length > match[2].length) {
throw new Error(`${value}: the counter is out of digits`);
}
return match[1] + next.padStart(match[2].length, "0");
}

// Bumps every magic number that is still the one of the previous release,
// when the format sources changed since then. One that differs was already
// bumped, by an earlier run for the same release or by hand.
export function bumpMagicNumbers(config, previousConfig, sourcesChanged) {
if (!sourcesChanged) return config;
let output = config;
for (const name of magicNumbers) {
const value = readMagicNumber(config, name);
if (value === null) throw new Error(`${configPath}: ${name} not found`);
if (readMagicNumber(previousConfig ?? "", name) !== value) continue;
output = output.replace(
definition(name),
`${name} = "${incrementMagicNumber(value)}"`,
);
}
return output;
}

// The highest release tag below the version being released. The nearest tag
// is not enough: a maintenance branch merged into master brings its newer
// tags, such as v12.3.2 after v13.0.0-alpha.6, with that line's magic numbers.
export function previousRelease(tags, version) {
let previous = null;
for (const tag of tags) {
const parsed = tag.startsWith("v") ? semver.parse(tag.slice(1)) : null;
if (parsed === null || !semver.lt(parsed, version)) continue;
if (previous === null || semver.gt(parsed, previous.parsed)) {
previous = { tag, parsed };
}
}
return previous?.tag ?? null;
}

// The previous release tag among the tags reachable from HEAD.
export function previousReleaseTag(root, version) {
const result = spawnSync("git", ["tag", "--merged", "HEAD", "--list", "v*"], {
cwd: root,
encoding: "utf8",
});
const tag =
result.status === 0
? previousRelease(result.stdout.split("\n").filter(Boolean), version)
: null;
if (tag === null) {
throw new Error(
`No previous release tag found; fetch the tags (git fetch --tags) first.\n${result.stderr}`,
);
}
return tag;
}

function changedSince(root, tag) {
const result = spawnSync(
"git",
["diff", "--quiet", tag, "HEAD", "--", ...formatSources],
{ cwd: root },
);
if (result.status === 0) return false;
if (result.status === 1) return true;
throw new Error(`git diff ${tag} HEAD failed: ${result.stderr}`);
}

export function showAt(root, revision, file) {
const result = spawnSync("git", ["show", `${revision}:${file}`], {
cwd: root,
encoding: "utf8",
});
return result.status === 0 ? result.stdout : null;
}

const counterPrefix = value => /^(.*?)\d*$/.exec(value)[1];

// A pull request may switch a magic number to another prefix, but only the
// release preparation bumps a counter.
export function checkMagicNumbers(baseConfig, config, labels) {
if (baseConfig === null || labels.includes("changelog:release")) return;
const bumped = magicNumbers.filter(name => {
const before = readMagicNumber(baseConfig, name);
const after = readMagicNumber(config, name);
return (
before !== null &&
after !== null &&
before !== after &&
counterPrefix(before) === counterPrefix(after)
);
});
if (bumped.length) {
throw new Error(
`${configPath}: don't bump ${bumped.join(", ")}; yarn changelog:release bumps the magic numbers when a release is prepared.`,
);
}
}

// The new contents of config.ml for the release, or null when no magic
// number needs a bump.
export async function releaseMagicNumbers(root, version) {
const tag = previousReleaseTag(root, version);
const config = await fs.readFile(path.join(root, configPath), "utf8");
const output = bumpMagicNumbers(
config,
showAt(root, tag, configPath),
changedSince(root, tag),
);
return output === config ? null : output;
}
Loading
Loading