Skip to content

Check every recipe screenshot for overflow and broken images - #281

Merged
Programcode merged 2 commits into
mainfrom
test/screenshot-layout-check
Oct 30, 2026
Merged

Programcode merged 2 commits into
mainfrom
test/screenshot-layout-check

Conversation

@Programcode

Copy link
Copy Markdown
Owner

Agents keep verify-debatelab screenshots as evidence without opening them. In October one shipped a broken LaunchIgniter badge in its own screenshot, and a coordinator caught it later. Every evidence.shot() now also checks the page it just captured:

  • the page is no wider than the screen (overflow ≤ 1 px);
  • every visible image loaded. Lazy images that haven't been fetched yet are skipped; an image that tried and got nothing fails.

The checks are soft assertions, so one recipe reports every bad screen. The recipe fixtures already checked overflow, but only at the end of a test; now each screenshot is checked when it's taken. SKILL.md documents the check and says to keep opening screenshots, because it can't see clipped text, overlaps or wrong colours.

Running every recipe also turned up a stale assertion on main. loc-switcher looked for a "404" h1, but the not-found page now shows the translated message as its h1 and "404" as an eyebrow. #279 fixed the same assertion in e2e/localization.spec.ts.

Verification

  • npm run verify passed at this head (10 steps, 126 s).
  • Every recipe, run with this check on the reduced local stack from Make local Supabase verification parallel-safe with two slots #280:
    • desktop-chromium: 79 passed, 2 skipped;
    • mobile-chromium: 78 passed, 2 skipped, 1 failed. The failure is lib-edit-while-switching (a not.toBeChecked race), which passed in the previous mobile run on the same code; it isn't overflow or images.
  • No screenshot tripped the overflow or broken-image check.

This changes test tooling only, so it's low-risk under AGENTS.md.

🤖 Generated with Claude Code

Programcode and others added 2 commits October 30, 2026 11:35
Agents kept screenshots as evidence without opening them; in October one
shipped a broken LaunchIgniter badge in its own screenshot. Each
evidence.shot() now soft-asserts that the page is no wider than the
screen and that every visible image loaded, so the recipe reports every
bad screen instead of relying on someone to look.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The not-found page now shows "404" as an eyebrow and the translated
message as its h1, so the recipe failed looking for a "404" heading.
#279 fixed the same assertion in e2e/localization.spec.ts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings October 30, 2026 13:29

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

@coderabbitai

coderabbitai Bot commented Oct 30, 2026

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 57 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration
Commits

Reviewing files that changed from the base of the PR and between f35ac23 and bb3408b.

Files selected for processing: 3
  • .claude/skills/verify-debatelab/SKILL.md
  • .claude/skills/verify-debatelab/recipes/fixtures.ts
  • .claude/skills/verify-debatelab/recipes/localization.spec.ts
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@Programcode

Copy link
Copy Markdown
Owner Author

Review round 1 (Claude, bb3408b; Codex is at its usage cap): nothing blocking.

  • Valid, accepted as is. A broken image inside a closed <details> would be flagged though it isn't on screen. No <details> in the app holds an <img> today (SparringSetup, ResearchedSources, FAQ, ToolPageLayout, WorkspaceVersions), and a false positive would be loud and easy to read, not a silent pass.
  • Valid, accepted as is. Lazy images below the fold are never fetched during a full-page screenshot, so they are never checked. The helper's comment already says unfetched lazy images are skipped.

CI is green and npm run verify passed at this head. Merging under the low-risk rule in AGENTS.md (test tooling only).

@Programcode
Programcode merged commit c571c88 into main Oct 30, 2026
6 checks passed
@Programcode
Programcode deleted the test/screenshot-layout-check branch October 30, 2026 13:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants