Skip to content

fix(e2e): unblock the e2e stack after v2.37.0 — seed hash, AI stub opt-in, no-redirect PKCE error - #196

Merged
krazymir merged 1 commit into
mainfrom
claude/e2e-test-shard-setup-d6c5a1
Oct 19, 2026
Merged

krazymir merged 1 commit into
mainfrom
claude/e2e-test-shard-setup-d6c5a1

Conversation

@krazymir

@krazymir krazymir commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Problem

The e2e shard stack fails during provisioning, before the suite starts:

{"level":"error","msg":"artifact integrity mismatch — version left unpublished", ... "expected":"sha-1.0.0", ...}
publish sweep: 0 version(s) published into .../.ci-e2e-git/shard-1
still unpublished (6): global/lint-fixer@2.2.0, global/lint-fixer@2.3.0, global/pdf-tools@1.0.0, ...

Since v2.37.0 (§49.12) the publish sweep re-hashes the stored artifact bytes against skill_versions.artifact_sha256 and refuses to publish on a mismatch. db/seed.dev.sql recorded placeholder hashes ('sha-'||semver) for the seeded hosted versions, while seed-bundles.mjs uploads a freshly generated bundle for each — the two can never agree, so every seeded version stays unpublished and publish-once.mjs exits non-zero. The same sequence runs in the GitHub e2e job.

The hash cannot be corrected after upload: the skill_versions immutability guard pins artifact_sha256 on UPDATE.

Fix

Seed the hosted versions with a NULL artifact_sha256. Nothing honest exists to record at seed time (the bytes don't exist yet), and the integrity check deliberately skips rows with no recorded hash. The seed comment and the spec's e2e-provisioning bullet (§ roadmap item 18) document the rule.

Verification

Ran the Jenkins provisioning sequence on a throwaway postgres:16-alpine + MinIO stack: all migrations → the fixed seed → seed-bundles.mjs → publish-once.mjs.

  • 6 bundles uploaded, publish sweep: 6 version(s) published, exit code 0
  • all six rows git_published = true; repos created under global/ and team-a/
  • 0 artifact_integrity_mismatch system events

Not run here: the Playwright suite itself (this change only unblocks its provisioning step).

No APP_VERSION bump: dev-only seed data and CI behavior, no app change.

Follow-up: the three remaining quick-run failures

With provisioning fixed, the Jenkins quick run got through all 22 tests and left three failures, all caused by the v2.37.0 hardening rather than by the seed:

  • ai-draft / ai-quality-draft — the specs run a stub AI provider on 127.0.0.1; §49.11 now refuses loopback base URLs (422 ai_base_url_private). The Playwright webServer env sets AI_ALLOW_PRIVATE_BASE_URL=1, which covers both the Jenkins shard script and the GitHub e2e job.
  • mcp-connect › missing PKCE challenge — expected a redirect with error=invalid_request; §49.5 made /oauth/authorize render the error page and never redirect before consent. The test now asserts a 200 with no Location, the "PKCE is required" message, the URL still on /oauth/authorize, and no Approve button.

No app code changes. Web typecheck passes and both files parse; the specs themselves were not run locally.

Third commit: the random 10 % quick run sampled a new slice

achievements.spec.ts failed on aria-valuemax: expected "24", received "26". The badge catalog grew twice since the spec was written (Encore in v2.19.0, Hype in v2.38.0) and only one of its five counts had been bumped. Every count and N-of-M regex now derives from ACHIEVEMENT_TOTAL (@skilly/shared/achievements), which the shared unit test already pins, so the next badge cannot strand it again.

A static sweep of the rest of the suite against the v2.37.0 and v2.38.0 route changes (visibility-first 404s, same-site download counting, audit-trim floor, DCR scheme refusal, Hype) found no other stale assertion: no spec asserts a 403, a download count or a trim, and the sharing spec runs as the platform admin. Because the quick run samples 10 % at random, a green run is not proof for the whole suite; a full run is the only way to be sure.

🤖 Generated with Claude Code

…e2e publish sweep passes

The e2e shard stack (ci/e2e-shard.sh and the GitHub e2e job) applies db/seed.dev.sql,
uploads a generated bundle per seeded hosted version (seed-bundles.mjs) and runs the
one-shot publish sweep (publish-once.mjs). Since v2.37.0 the sweep re-hashes the stored
bytes against skill_versions.artifact_sha256 (SKILLY_SPEC.md section 49.12) and refuses to
publish on a mismatch. The dev seed recorded placeholder hashes ('sha-1.0.0' etc.) that can
never equal the hash of the uploaded bundle, so every seeded version was left unpublished
and publish-once exited non-zero:

  artifact integrity mismatch - version left unpublished ... expected "sha-1.0.0"
  still unpublished (6): global/lint-fixer@2.2.0, ... team-a/secret-helper@0.9.0

The row is immutable once inserted (the skill_versions guard pins artifact_sha256), so the
uploader cannot fill the real hash in afterwards. Seed the hosted versions with a NULL hash
instead: nothing honest exists to record at seed time, and the integrity check deliberately
skips rows without a recorded hash. The spec's e2e-provisioning bullet now states this rule.

Verified on a throwaway Postgres/MinIO stack with the Jenkins sequence: all 6 seeded
versions publish, publish-once exits 0, no artifact_integrity_mismatch system events.

No APP_VERSION bump: dev-only seed data and CI behavior, no app change.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@krazymir
krazymir merged commit af24c77 into main Oct 19, 2026
9 checks passed
@krazymir
krazymir deleted the claude/e2e-test-shard-setup-d6c5a1 branch October 19, 2026 11:32
@krazymir krazymir changed the title fix(seed): record no artifact hash for seeded hosted versions so the e2e publish sweep passes fix(e2e): unblock the e2e stack after v2.37.0 — seed hash, AI stub opt-in, no-redirect PKCE error Oct 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants