Repository navigation
Add verify-compilers-locally skill - #2156
Merged
Merged
Conversation
Captures the install -> run-local-CE -> compile-test loop for vetting new compiler PRs: ce_install the compilers (plus toolchain deps), layer the config as local.properties (never --env amazon), drive the compile API across every id, report a pass/fail table, tear down. compile_matrix.py is the reusable harness: --source-file/--args for interpreted DSLs, and --execute to check the execution path (didExecute + execResult.code), which is separate from disassembly -- a compiler can disassemble fine yet run the wrong binary. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a new Claude skill and a small Python harness to standardize the “verify new compilers locally” workflow against a real local Compiler Explorer (CE) instance, including optional execution-path validation.
Changes:
- Introduces
verify-compilers-locallyskill documentation capturing the end-to-end local verification procedure and common gotchas. - Adds
compile_matrix.py, a CLI helper to drive CE’s compile API across multiplelang:compilerIdcases and print a pass/fail matrix (with an--executemode).
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
| .claude/skills/verify-compilers-locally/SKILL.md | Documents the local install/config/run/teardown procedure and verification gotchas. |
| .claude/skills/verify-compilers-locally/compile_matrix.py | Adds a reusable CLI harness to compile (and optionally execute) a test program across many compiler IDs via the CE API. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+57
to
+69
| asm = res.get("asm") or [] | ||
| exec_res = res.get("execResult") or {} | ||
| exec_stdout = " ".join(x.get("text", "") for x in (exec_res.get("stdout") or [])).strip() | ||
| # The build/setup stderr (e.g. a failed interpreter spawn) surfaces at the top level. | ||
| setup_err = " ".join(x.get("text", "") for x in (res.get("stderr") or [])).strip() | ||
| return { | ||
| "code": res.get("code"), | ||
| "asm_lines": sum(1 for a in asm if (a.get("text") or "").strip()), | ||
| "did_execute": exec_res.get("didExecute"), | ||
| "exec_code": exec_res.get("code"), | ||
| "stdout": exec_stdout, | ||
| "setup_err": setup_err, | ||
| } |
Comment on lines
+144
to
+145
| cases = [(case.partition(":")[0], case.partition(":")[2]) for case in opts.cases] | ||
| all_ok = run_execution(cases, sources, opts) if opts.execute else run_disassembly(cases, sources, opts) |
| ### 4. Launch CE locally | ||
| Needs `node_modules` (run `npm ci` in the checkout/worktree if absent). Then: | ||
| ```bash | ||
| nohup npm run dev -- --port 10240 > /tmp/ce_dev.log 2>&1 & |
Comment on lines
+95
to
+96
| pkill -f "app.ts --port 10240" | ||
| rm -f <ce-checkout>/etc/config/*.local.properties # so the next `npm run dev` isn't AOCC-only |
partouf
reviewed
Jun 4, 2026
| ``` | ||
|
|
||
| ### 2. Install the compilers (+ toolchain deps) | ||
| Default dest is `/opt/compiler-explorer` (must be writable). Install the group: |
Member
There was a problem hiding this comment.
perhaps important to note to never install a complete group with things like gcc, clang or rust
Member
There was a problem hiding this comment.
Could list first and select the exact versions to install instead
Install exact versions picked from `ce_install list` instead of whole groups. Track the dev server PID for teardown rather than pkill -f. compile_matrix.py now counts devices[].asm, reports execResult.stderr on execution failures, validates LANG:ID args, and catches concrete exceptions instead of suppressing BLE001. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A Claude Code skill that captures the loop for vetting new compiler PRs: install the compilers locally (plus toolchain deps), run a local CE instance against the real config, compile-test every id, report a pass/fail table, then tear down.
What's here
SKILL.md-- the procedure and the hard-won gotchas (never--env amazonlocally; layer config aslocal.properties;--gcc-toolchainGCC must be installed; disassembly passing != execution working).compile_matrix.py-- the reusable harness. DrivesPOST /api/compiler/<id>/compileacross a list oflang:idcases and prints a table. Flags:--source/--source-file(interpreted DSLs whose source is an example file),--args, and--execute(checksdidExecute+execResult.code, a separate path from disassembly).Why
Built and refined while adopting AOCC, CuTe DSL, and Lua. It earned its keep: it caught a CuTe DSL placeholder false-pass, and the
--executemode exists because every Lua version disassembled correctly while all executed with the wrong interpreter -- a disassembly-only check was all-green.make static-checkspasses.