shacl-runner can only validate built HTML, and its validator silently cannot run SHACL-SPARQL shapes

[bdelanghe]

Two limits in gates/shacl-runner.mjs that together mean a non-site artifact cannot use the gate at all. Found while trying to wire the kit to a database mirror (bounded-systems/front-desk-scheduler#3), where both bite immediately.

Measurements below are from that session, 2026-08-04.

1. The only input mode is a directory of built HTML

node gates/shacl-runner.mjs <shapes.ttl> <htmlDir>

It walks <htmlDir> for .html, extracts <script type="application/ld+json"> blocks, and validates those. For a consumer whose artifact is a dataset rather than a site — a mirror, an export, a manifest — there is no input mode at all. The alternative is emitting throwaway HTML solely to carry JSON-LD past the runner's front door, which is fabricating an artifact to satisfy a signature.

The vocabulary side is already general: $SHACL_CONTEXT overrides the built-in schema.org context and the gate never fetches over the network, so a custom namespace works fine. It is only the input shape that is site-specific.

Ask: a dataset input mode — --jsonld <file>, --turtle <file>, or reading n-quads on stdin — leaving the HTML-scraping path exactly as it is for existing consumers.

2. rdf-validate-shacl cannot run SHACL-SPARQL, at any pinned version

The kit pins rdf-validate-shacl ^0.5.10. Tested directly against a shapes file using sh:sparql:

rdf-validate-shacl@0.5.10  → Error: Cannot find validator for constraint component
rdf-validate-shacl@0.6.5   →   http://www.w3.org/ns/shacl#SPARQLConstraintComponent

Upgrading the pin does not fix it. In the runner this surfaces through main().catch() as ✗ shacl-runner: error — …, which reads like an infrastructure fault rather than "your shapes use a feature this validator does not implement".

Any shape expressing a cross-row invariant — referential agreement, acyclicity, "this status implies that relationship" — needs sh:sparql. Core SHACL cannot say those things. So today the gate can enforce per-node structural constraints and nothing relational.

Ask: swap to shacl-engine (1.1.2). It is Zazuko's, same as @zazuko/env-node which the kit already depends on, and it implements SHACL-SPARQL constraints and targets. Verified: with a spec-correct shapes file it returns verdicts identical to pyshacl 0.40.1 — 4 results each on the same cyclic fixture.

⚠️ If you take the swap, the fixture is not optional

shacl-engine's Validator ships core validations only. SPARQL support is opt-in:

import { targetResolvers, validations } from 'shacl-engine/sparql.js'
new Validator(shapes, { factory, targetResolvers, validations })

Omit those and it does not warn and does not throw — it silently skips every sh:sparql shape. Measured, on a graph whose only defect is a dependency cycle:

shacl-engine, default config      → conforms: TRUE     ← cycle silently ignored
shacl-engine, with sparql opt-in  → conforms: false, 4 results
pyshacl 0.40.1                    → Conforms: False, 4 results

Note this is a regression in failure mode: rdf-validate-shacl throws on sh:sparql, loudly. shacl-engine without the opt-in passes, quietly. For a gate — whose entire value is that a pass means something — trading a loud failure for a silent one is the wrong direction unless it is pinned down.

So the swap should land with a negative fixture the gate is required to fail: a known-cyclic graph, asserted to produce conforms: false. Without it, a refactor can drop the two options and every downstream consumer keeps seeing green.

Why this is worth doing here rather than working around

A consumer blocked on this can always run its own validator locally, and that is the fallback. But then the kit is the sites' conformance story rather than the org's, and each non-site consumer reinvents the reporting surface. The two changes above are what make the gate reusable for artifacts that were never a website.

Happy to do the work if the direction is right — the validator swap in particular touches every existing consumer's behaviour, which is why this is an issue and not a PR.

[bdelanghe]

Both halves merged; closing. (It stayed open past the second merge because the PR bodies said "Completes #58" rather than a keyword GitHub acts on — noting that rather than silently fixing it.)

#59 — the input mode. --turtle / --jsonld take a dataset directly, so an artifact that was never a website can use the gate. HTML path untouched; a flag with no path exits 2 rather than silently falling back to scanning HTML.

#60 — the validator. Swapped rdf-validate-shacl → shacl-engine. SHACL-SPARQL constraints now execute, so cross-row invariants — referential agreement, acyclicity, "this status implies that relationship" — are enforceable. Report format and exit codes unchanged.

What guards it

The hazard the swap introduces is worse than the bug it fixes, taken alone: shacl-engine skips every sh:sparql shape unless targetResolvers/validations are passed, and it does so silently — where the old validator at least threw. So it landed with a required-to-fail cyclic fixture, and I checked that guard actually guards by removing the opt-in:

✓ sh:sparql shapes + conforming data → conforms:true   ← STILL PASSED
✗ sh:sparql violation → exit 1 (opt-in wired)          ← only this caught it

A positive test alone would not have noticed.

Two things found along the way, both now fixed

The reproducible path had stopped being built. flake.nix declares all 29 ck-* gates as hash-pinned CLIs, but nothing in CI built it — flake.nix said 0.8.0 against package.json's 0.11.0, and npmDepsHash had never been checked against the lockfile it claims to pin. The new flake job runs nix build, asserts version agreement, and runs the built binary:

built CLI: conforming passes, cyclic is rejected

It has since caught two real lockfile/derivation inconsistencies, both on changes made after it was added.

The deps lane was red on main. deps.yml declares Failure posture: HARD-FAIL — adopted deliberately in #53 — and it had drifted red with 7 advisories. I twice called that "pre-existing, safe to ignore" before reading the config; it is a gate this repo chose. Fixed via overrides (fast-uri → 3.1.5, ip-address → 10.4.0, undici → 6.28.0), not the osv-scanner.toml [[IgnoredVulns]] hatch — that is for accepted or unfixable findings, and all seven were fixable today.

On the dependency footprint

shacl-engine hard-depends on Comunica: 297 → 758 packages. I originally raised this as blocking and leaned toward keeping both engines behind a dynamic import. That was wrong — determinism here is a property of the hash-pinned Nix artifact, not of the dependency count, and the added packages carry zero new advisories. Two engines would have meant two code paths and two answers to "what does this gate enforce".

Downstream: bounded-systems/front-desk-scheduler#3 is unblocked — the shared runner can now consume that mirror's RDF and execute the D1/D2/D3 shapes its sh:sparql constraints need.

---

Generated by Claude Code