Repository navigation
fix(e2e): unblock the e2e stack after v2.37.0 — seed hash, AI stub opt-in, no-redirect PKCE error - #196
Merged
Conversation
…e2e publish sweep passes
The e2e shard stack (ci/e2e-shard.sh and the GitHub e2e job) applies db/seed.dev.sql,
uploads a generated bundle per seeded hosted version (seed-bundles.mjs) and runs the
one-shot publish sweep (publish-once.mjs). Since v2.37.0 the sweep re-hashes the stored
bytes against skill_versions.artifact_sha256 (SKILLY_SPEC.md section 49.12) and refuses to
publish on a mismatch. The dev seed recorded placeholder hashes ('sha-1.0.0' etc.) that can
never equal the hash of the uploaded bundle, so every seeded version was left unpublished
and publish-once exited non-zero:
artifact integrity mismatch - version left unpublished ... expected "sha-1.0.0"
still unpublished (6): global/lint-fixer@2.2.0, ... team-a/secret-helper@0.9.0
The row is immutable once inserted (the skill_versions guard pins artifact_sha256), so the
uploader cannot fill the real hash in afterwards. Seed the hosted versions with a NULL hash
instead: nothing honest exists to record at seed time, and the integrity check deliberately
skips rows without a recorded hash. The spec's e2e-provisioning bullet now states this rule.
Verified on a throwaway Postgres/MinIO stack with the Jenkins sequence: all 6 seeded
versions publish, publish-once exits 0, no artifact_integrity_mismatch system events.
No APP_VERSION bump: dev-only seed data and CI behavior, no app change.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The e2e shard stack fails during provisioning, before the suite starts:
Since v2.37.0 (§49.12) the publish sweep re-hashes the stored artifact bytes against
skill_versions.artifact_sha256and refuses to publish on a mismatch.db/seed.dev.sqlrecorded placeholder hashes ('sha-'||semver) for the seeded hosted versions, whileseed-bundles.mjsuploads a freshly generated bundle for each — the two can never agree, so every seeded version stays unpublished andpublish-once.mjsexits non-zero. The same sequence runs in the GitHube2ejob.The hash cannot be corrected after upload: the
skill_versionsimmutability guard pinsartifact_sha256on UPDATE.Fix
Seed the hosted versions with a NULL
artifact_sha256. Nothing honest exists to record at seed time (the bytes don't exist yet), and the integrity check deliberately skips rows with no recorded hash. The seed comment and the spec's e2e-provisioning bullet (§ roadmap item 18) document the rule.Verification
Ran the Jenkins provisioning sequence on a throwaway
postgres:16-alpine+ MinIO stack: all migrations → the fixed seed →seed-bundles.mjs→publish-once.mjs.publish sweep: 6 version(s) published, exit code 0git_published = true; repos created underglobal/andteam-a/artifact_integrity_mismatchsystem eventsNot run here: the Playwright suite itself (this change only unblocks its provisioning step).
No
APP_VERSIONbump: dev-only seed data and CI behavior, no app change.Follow-up: the three remaining quick-run failures
With provisioning fixed, the Jenkins quick run got through all 22 tests and left three failures, all caused by the v2.37.0 hardening rather than by the seed:
ai-draft/ai-quality-draft— the specs run a stub AI provider on127.0.0.1; §49.11 now refuses loopback base URLs (422 ai_base_url_private). The PlaywrightwebServerenv setsAI_ALLOW_PRIVATE_BASE_URL=1, which covers both the Jenkins shard script and the GitHub e2e job.mcp-connect› missing PKCE challenge — expected a redirect witherror=invalid_request; §49.5 made/oauth/authorizerender the error page and never redirect before consent. The test now asserts a 200 with noLocation, the "PKCE is required" message, the URL still on/oauth/authorize, and no Approve button.No app code changes. Web typecheck passes and both files parse; the specs themselves were not run locally.
Third commit: the random 10 % quick run sampled a new slice
achievements.spec.tsfailed onaria-valuemax: expected "24", received "26". The badge catalog grew twice since the spec was written (Encore in v2.19.0, Hype in v2.38.0) and only one of its five counts had been bumped. Every count and N-of-M regex now derives fromACHIEVEMENT_TOTAL(@skilly/shared/achievements), which the shared unit test already pins, so the next badge cannot strand it again.A static sweep of the rest of the suite against the v2.37.0 and v2.38.0 route changes (visibility-first 404s, same-site download counting, audit-trim floor, DCR scheme refusal, Hype) found no other stale assertion: no spec asserts a 403, a download count or a trim, and the sharing spec runs as the platform admin. Because the quick run samples 10 % at random, a green run is not proof for the whole suite; a full run is the only way to be sure.
🤖 Generated with Claude Code