Skip to content

shacl-runner: swap to shacl-engine so SHACL-SPARQL shapes actually run (#58) - #60

Merged
bdelanghe merged 5 commits into
mainfrom
claude/whats-next-ow7hqm
Aug 4, 2026
Merged

shacl-runner: swap to shacl-engine so SHACL-SPARQL shapes actually run (#58)#60
bdelanghe merged 5 commits into
mainfrom
claude/whats-next-ow7hqm

Conversation

@bdelanghe

@bdelanghe bdelanghe commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Completes #58. One engine, and the reproducible artifact built and exercised in CI.

What it fixes

rdf-validate-shacl cannot execute SPARQL-based constraints at any version in range:

rdf-validate-shacl@0.5.10 (the pin)  → Error: Cannot find validator for constraint
rdf-validate-shacl@0.6.5  (latest)   →   component sh#SPARQLConstraintComponent

Core SHACL cannot traverse a property path, so every cross-row invariant — referential agreement, acyclicity, "this status implies that relationship" — requires sh:sparql. The gate could enforce per-node structure and nothing relational, and announced that by throwing in a way that reads like an infrastructure fault rather than an unsupported feature.

shacl-engine is the same RDF/JS stack — same @zazuko/env-node the kit already depends on — and implements SHACL-SPARQL constraints and targets. Report format and exit codes are unchanged; printViolations normalises the two engines' differing result shapes (path as segments vs a single term, constraintComponent vs sourceConstraintComponent) so consumers reading these logs see no difference.

⚠️ The hazard, and the guardrail

shacl-engine ships core validations only. Without targetResolvers and validations from shacl-engine/sparql.js, it does not warn and does not throw — it silently skips every sh:sparql shape:

default config          → conforms: TRUE     ← the cycle is invisible
with the sparql opt-in  → conforms: false, cycle reported

That is a worse failure mode than the validator it replaces, which at least threw loudly. So the swap lands with a required-to-fail cyclic fixture — and I verified that guard actually guards, by removing the opt-in and watching what happened:

✓ gates/shacl-runner: sh:sparql shapes + conforming data → conforms:true   ← STILL PASSES
✗ gates/shacl-runner: sh:sparql violation → exit 1 (opt-in wired)          ← catches it

The conforming test still passed with SPARQL disabled. Only the negative test caught it — which is the entire argument for having one.

On the dependency footprint

shacl-engine hard-depends on @comunica/query-sparql-rdfjs-lite, so the Comunica query engine comes with it: 297 → 758 lockfile packages. It is not a peer or optional dependency, so there is no way to take shacl-engine's core validations without it.

I originally raised this as a blocking question and leaned toward keeping both engines with a dynamic import. That was the wrong call, for two measured reasons:

  • Determinism here is a property of the artifact, not of the dependency count. The kit ships its gates as Nix CLIs and npmDepsHash content-addresses the entire tree, so a consuming site's vendored gate is hash-verified rather than re-resolved at install.
  • The added packages carry no new known vulnerabilities. OSV reports exactly the same 7 advisories (fast-uri, ip-address, undici) as main — all pre-existing, none from Comunica.

And two engines would have meant two code paths, two result shapes, and two answers to "what does this gate enforce?" One engine is the point.

The reproducible path had stopped being built

Following from that, the flake had to actually work — and it didn't. flake.nix declares all 29 ck-* gates as reproducible CLIs, but:

  • nothing in CI ever built it
  • flake.nix said 0.8.0 while package.json said 0.11.0 — three minors of drift
  • npmDepsHash had never been checked against the lockfile it claims to pin, so this PR's lockfile change made it detectably wrong rather than newly wrong

A reproducible-build guarantee nobody executes decays in exactly one direction. The new flake job runs nix build (which fails loudly on a stale hash and prints the correct one — that is how the pinned value here was obtained), asserts version agreement, and then runs the built binary:

built CLI: conforming passes, cyclic is rejected

Building is not evidence the artifact works. That line is the Nix-built ck-shacl-runner validating a conforming graph and refusing a cyclic one through a SHACL-SPARQL property path — so the binary a consumer vendors is verified, not just the source tree.

Verification

✓ sample shapes + HTML → conforms:true          (existing, unchanged)
✓ --turtle dataset → conforms:true              (from #59)
✓ violating dataset → exit 1, violations named  (from #59)
✓ --turtle with no path → usage error           (from #59)
✓ sh:sparql shapes + conforming data → conforms:true   ← proves it RUNS, not just fails
✓ sh:sparql violation → exit 1 (opt-in wired)          ← required-to-fail

32 passed. build green. version-check green.

Known-red, and deliberately not fixed here

osv / osv-scan fails on 7 advisories in fast-uri, ip-address and undici. All are pre-existing — present in main's lockfile, and this branch's additions contribute none of them. They are real and fixable, but a three-package lockfile bump inside an engine swap would make one diff about two unrelated things. Worth its own PR.

Also unexplained and worth someone's attention: standard / osv passes the same lockfile that osv / osv-scan fails. Two scanners disagreeing about one input is worth understanding regardless of which is right.

🤖 Generated with Claude Code

https://claude.ai/code/session_01MqhgW5RCRn6wWfjKvDsxTq

#58)

The second half of #58. rdf-validate-shacl cannot execute SPARQL-based
constraints at any version in range — it throws:

    Cannot find validator for constraint component sh#SPARQLConstraintComponent

Core SHACL cannot traverse a property path, so every cross-row invariant —
referential agreement, acyclicity, "this status implies that relationship" —
needs sh:sparql. The gate could therefore enforce per-node structure and nothing
relational, and said so by failing in a way that reads like an infrastructure
fault rather than an unsupported feature.

shacl-engine is the same RDF/JS stack (same @zazuko/env-node the kit already
depends on) and implements SHACL-SPARQL constraints and targets. Report format
and exit codes are unchanged; printViolations normalises the two engines'
differing result shapes (path as segments vs a term, constraintComponent vs
sourceConstraintComponent) so consumers reading these logs see no difference.

THE HAZARD, AND THE GUARDRAIL
-----------------------------
shacl-engine ships CORE validations only. Without targetResolvers and
validations from shacl-engine/sparql.js it does not warn and does not throw — it
silently skips every sh:sparql shape. Measured on the new cyclic fixture:

    default config        → conforms: TRUE   (the cycle is invisible)
    with the sparql opt-in → conforms: false, cycle reported

That is a worse failure mode than the validator it replaces, which at least
threw loudly. So the swap lands with a required-to-fail fixture, and that guard
was itself verified by removing the opt-in: the negative test goes red while the
CONFORMING sparql test still passes — which is exactly why a positive test alone
would not have caught it.

DEPENDENCY FOOTPRINT — the reason this is a draft
-------------------------------------------------
shacl-engine hard-depends on @comunica/query-sparql-rdfjs-lite, so the whole
Comunica engine arrives with it: 297 → 758 packages (+468). It is not a peer or
optional dependency, so there is no way to take shacl-engine's core validations
without it. For a kit that consuming sites vendor hash-pinned, that is a real
supply-chain cost and wants a decision rather than a merge.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MqhgW5RCRn6wWfjKvDsxTq
claude added 2 commits August 4, 2026 04:13
…ting

The kit ships its gates as hash-pinned Nix CLIs — npmDepsHash content-addresses
the whole dependency tree, which is what makes a consuming site's vendored gate
deterministic rather than re-resolved at install.

Nothing built it, and the decay was already visible: flake.nix said 0.8.0 while
package.json said 0.11.0, three minors of drift with nothing to report it. The
engine swap in this PR makes it acute — changing package-lock.json invalidates
npmDepsHash, and without a build there is no signal at all.

Adds a `flake` job that runs `nix build`, which fails loudly on a stale hash and
prints the correct one. It also asserts version agreement between flake.nix and
package.json, and — because building is not evidence the artifact works — runs
the BUILT ck-shacl-runner against the fixtures, including requiring it to reject
the cyclic graph. A gate that only passes from the source tree says nothing
about the binary a consumer actually vendors.

npmDepsHash is a placeholder here; the job's failure carries the real value and
it is pinned in the follow-up commit.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MqhgW5RCRn6wWfjKvDsxTq
The value the new `flake` job reported. Swapping rdf-validate-shacl for
shacl-engine changes package-lock.json, which changes the content hash of the
fetched npm dependencies:

    specified: sha256-AAAA…  (placeholder)
    got:       sha256-Y6daAa9ySvlQhl3pbmPtfnx4JUyZrfKdnihAoZV7tH4=

Worth noting the previous hash had been stale in a quieter way: nothing built
the derivation, so the value in tree was simply never checked against the
lockfile it claims to pin.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MqhgW5RCRn6wWfjKvDsxTq
@bdelanghe
bdelanghe marked this pull request as ready for review August 4, 2026 04:19
claude added 2 commits August 4, 2026 04:35
I had twice called this failure pre-existing and safe to ignore. That was wrong
about intent. deps.yml states the posture plainly:

    Failure posture: HARD-FAIL. This repo's adoption grace was removed once its
    findings cleared (conformance-kit#53) — a known vulnerability now reds the lane.

So a green deps lane is this repo's stated merge condition, not optional. It was
red on main too, which is why it read as background noise rather than as the gate
it was adopted to be.

All three packages are deep transitive deps with fixes published, so `overrides`
is the lever:

    fast-uri    3.1.4  → 3.1.5    (via stylelint → table → ajv)
    ip-address  10.2.0 → 10.4.0   (via sigstore → … → socks)
    undici      6.27.0 → 6.28.0   (via jsonld → @digitalbazaar/http-client)

`npm audit` and `npm audit --omit=dev` both report 0 vulnerabilities; the suite
is unchanged at 32 passed.

Note this is deliberately NOT the escape hatch deps.yml offers. An
osv-scanner.toml [[IgnoredVulns]] entry is for an accepted or unfixable finding,
and all seven of these are fixable today — suppressing them would record a
judgement nobody made.

Landing it here rather than in a separate PR because the swap cannot honestly
merge while the lane the repo hard-gates on is red.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MqhgW5RCRn6wWfjKvDsxTq
The overrides that cleared the OSV findings changed package-lock.json, so the
fetched npm dependencies hash differently:

    specified: sha256-Y6daAa9y…   (the shacl-engine tree)
    got:       sha256-MltNN6K9…   (with fast-uri/ip-address/undici bumped)

Second time today the flake job has caught this, on a change made after it was
added — which is the argument for it. Before, a lockfile edit left the
derivation unbuildable with nothing anywhere reporting it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MqhgW5RCRn6wWfjKvDsxTq
@bdelanghe
bdelanghe merged commit 892957c into main Aug 4, 2026
9 checks passed
@bounded-systems-front-desk bounded-systems-front-desk Bot moved this from Todo to Done in Front Desk Aug 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants