Skip to content

ci: auto-sort and validate systems.csv on PRs#894

Open
fredericsimard wants to merge 5 commits into
masterfrom
chore/sort-validate-systems-csv/2026-07-24
Open

ci: auto-sort and validate systems.csv on PRs#894
fredericsimard wants to merge 5 commits into
masterfrom
chore/sort-validate-systems-csv/2026-07-24

Conversation

@fredericsimard

Copy link
Copy Markdown
Contributor

What

Adds automated checks that run on every pull request that changes systems.csv, plus a data sort so the catalog starts in a sorted state.

New workflow: .github/workflows/systems_csv_pr_checks.yml

Job Behavior
sort Re-sorts systems.csv and pushes the result back to the PR branch as a separate chore: sort systems.csv commit. Same-repo branches only — fork PRs are skipped (the default GITHUB_TOKEN cannot push to forks).
validate-structure Fails on malformed rows (wrong field count / broken quoting) and empty required columns (Country Code, Name, Location, System ID, URL, Auto-Discovery URL). Empty Supported Versions → warning; the 3 Authentication columns are optional. Emits inline PR annotations.
check-new-urls For URLs newly added by the PR (both URL and Auto-Discovery URL columns), follows redirects and requires a final HTTP 200–299. Pre-existing URLs are ignored.

New scripts (scripts/)

  • sort-systems-csv.sh — mirrors the canonical (head -n 1; LC_ALL=en_US.UTF-8 sort -t, -k1,1 -k2,2) < systems.csv. Supports --check and a GNUSORT=gsort override for macOS.
  • validate-systems-csv.js — structure validator (Node, no deps).
  • check-new-urls.js — new-URL HTTP checker (Node, no deps; retries once on transient network errors).

Other changes

  • chore: sort systems.csv — applies the sort now (3 rows reordered).
  • README.md — the "keep this list alphabetized" note now explains the automated sort in plain and technical terms, with local-repro instructions.

Notes / decisions

  • Fork PRs: auto-sort commit is skipped (can't push to forks); structure and URL checks still run.
  • URL check covers both URL columns per request; operator homepages occasionally return 403/429 to CI bots, which would fail a PR — worth watching in practice.

Test evidence (local)

  • Sort: idempotent; --check correctly flags unsorted input.
  • Structure: current file passes with 1 warning (empty Supported Versions); synthetic malformed rows correctly flagged.
  • URL check: pre-existing URLs ignored, new URLs from both columns checked, passes on 2xx / fails on non-2xx.

🤖 Generated with Claude Code

fredericsimard and others added 3 commits July 24, 2026 14:56
Adds a GitHub Actions workflow that runs on any pull request changing
systems.csv, plus the scripts it drives:

- sort: re-sorts systems.csv (header kept in place, remaining rows sorted
  by Country Code then Name with GNU sort under en_US.UTF-8) and pushes the
  result back to the PR branch as a separate `chore: sort systems.csv`
  commit. Same-repo branches only; fork PRs are skipped because the default
  GITHUB_TOKEN cannot push to forks.
- validate-structure: fails on malformed rows (wrong field count / broken
  quoting) and on empty required columns (Country Code, Name, Location,
  System ID, URL, Auto-Discovery URL). An empty Supported Versions is a
  non-fatal warning; the three Authentication columns are optional.
- check-new-urls: for URLs newly added by the PR (both the URL and
  Auto-Discovery URL columns), follows redirects and requires a final
  HTTP 200-299.

scripts/sort-systems-csv.sh mirrors the canonical one-liner and supports a
--check mode plus a GNUSORT override for macOS (gsort). README documents the
now-automated sorting in plain and technical terms.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Apply the canonical sort so the catalog starts in a sorted state; going
forward the systems.csv PR checks workflow keeps it sorted automatically.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ fredericsimard
❌ github-actions[bot]
You have signed the CLA already but the status is still pending? Let us recheck it.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed by subsequent commits

Pull request overview

Adds automated pull-request checks around systems.csv to keep the catalog consistently sorted and to validate new/changed entries, with supporting scripts and contributor documentation updates.

Changes:

  • Introduces a new PR workflow to auto-sort systems.csv, validate CSV structure/required fields, and HTTP-check newly added URLs.
  • Adds three new helper scripts under scripts/ to implement sorting, structural validation, and new-URL checking.
  • Applies an initial sort to systems.csv and updates README contributor guidance with local reproduction instructions.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
systems.csv Reorders a handful of rows to match the new canonical sort order.
scripts/validate-systems-csv.js Adds a Node-based structure/required-field validator emitting GitHub Actions annotations.
scripts/sort-systems-csv.sh Adds an in-repo canonical sort implementation (header preserved; GNU sort keys).
scripts/check-new-urls.js Adds a Node-based checker that verifies newly introduced URLs return final 2xx after redirects.
README.md Updates contributor instructions to describe the automated sort and local reproduction steps.
.github/workflows/systems_csv_pr_checks.yml New workflow wiring sorting + validation + URL checks on PRs touching systems.csv.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread scripts/validate-systems-csv.js
Comment thread scripts/validate-systems-csv.js Outdated
Comment thread .github/workflows/systems_csv_pr_checks.yml
Comment thread README.md Outdated
Comment thread scripts/check-new-urls.js Outdated
Comment thread scripts/check-new-urls.js
Comment thread scripts/check-new-urls.js
The CI auto-sort (Linux/glibc) produced a different order than a local
macOS sort under the same en_US.UTF-8 locale, so the two would fight in a
loop. Switch the canonical sort to LC_ALL=C (byte-wise), which is identical
on macOS BSD sort and Linux GNU sort; re-sort systems.csv accordingly and
update the README to describe the byte-wise ordering and drop the macOS
coreutils requirement.

Also addresses adversarial PR review on the systems.csv checks:

- check-new-urls.js: add an SSRF guard. URLs come from untrusted fork PRs
  and are fetched by the runner, so refuse to connect to loopback, private,
  link-local, CGNAT, and ULA addresses (and localhost), resolving hostnames
  and re-validating every redirect hop. Blocks probing of the runner network
  and cloud metadata endpoints (e.g. 169.254.169.254).
- check-new-urls.js: cap newly added URLs checked per PR (MAX_NEW_URLS=200).
- validate-systems-csv.js: escape %/CR/LF in GitHub Actions annotations so
  multi-line errors (e.g. header mismatch) are not truncated; fix the
  blank-line comment to match behavior.
- workflow: gate check-new-urls on validate-structure.
- README: fix GitHub branding ("Github-less" -> "GitHub-less").

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 6 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

scripts/check-new-urls.js:163

  • The SSRF guard can be bypassed via DNS rebinding: hostIsSafe() resolves the hostname and checks the returned IPs, but http(s).request() will do its own DNS lookup at connection time (which may return a different/private address). Add a lookup option to the request that validates the resolved address with isBlockedIp() before connecting, so the safety check is enforced at the actual socket connection step as well.
      {
        method: 'GET',
        headers: { 'User-Agent': USER_AGENT, Accept: '*/*' },
        timeout: TIMEOUT_MS,
      },

Comment thread scripts/sort-systems-csv.sh Outdated
Comment thread .github/workflows/systems_csv_pr_checks.yml Outdated
The previous sort used `sort -t, -k1,1 -k2,2`, which is not aware of CSV
quoting: a Name quoted because it contains a comma was sorted by the text up
to the first raw comma, and (under LC_ALL=C) its leading `"` byte pushed the
row to the top of its country block. Two rows were affected -- "Veo
University of Illinois, Urbana-Champaign" (US) and "Styr & Ställ (Sweden,
Göteborg)" (SE).

Replace the bash sorter with a small CSV-quoting-aware Node sorter
(scripts/sort-systems-csv.js) that parses quoted fields, sorts by the real
Country Code then Name using byte-wise (LC_ALL=C-equivalent) comparison, and
re-emits each row verbatim. Implementing the comparison in Node keeps the
result identical on macOS and the Linux CI runner. Re-sort systems.csv so the
two quoted rows move to their correct alphabetical positions; update the
workflow to run the Node sorter and the README to match.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@fredericsimard
fredericsimard requested a review from richfab July 24, 2026 20:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants