Repository navigation
test(agentic): cover every tool and page, fail on hidden errors, fix what it found - #110
Conversation
Upgrade e2e to 0.17.0 and @e2e-dev/web to 0.12.0 for init scripts and failOnSkippedFailure. An init script records uncaught errors, console.error and Supabase 5xx answers; every test fails on them. Add a journey per tool and a sweep of every page in English, Spanish and Hungarian on both viewports. Run CI per viewport on app changes, and a weekly bug bash with each persona. Fix the p > div > p nesting in Attack results that the new check caught. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…tic flows The agentic suite's page-error check found that empty submits on Generate, Attack, Brainstorm and Factoids threw an unhandled ZodError and showed no message: @hookform/resolvers 3 cannot read zod 4 errors. Upgrade it to 5.4.0, the newest release without the @typeschema peer chain npm rejects, and type FactoidsForm's input and output apart. agentic/forms.e2e.ts pins all four. The replay cache could not record Settings because its three selects had no accessible name. Associate their labels, and find the output language select by name in the two specs that used a CSS container. OpenRouter withdrew the free Qwen actor, so every cache miss and exploration failed. Default the actor to openai/gpt-6-luna. Split CI into page-sweep and journey legs per viewport, fix the truncated workflow, and commit the re-recorded replay cache. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
React Flow 11 gives its zoom, fit-view and lock buttons fixed English names, which the agentic page sweep flagged in Spanish and Hungarian. Render the same four as ControlButtons with translated names, and pass the MiniMap a translated aria-label. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…re fail-safe Codex review: the translated zoom buttons stayed enabled at the zoom limits, unlike React Flow's own. The error-capture fetch wrapper now cannot throw into the app's request. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…st the edge The page sweep's judge saw the Hungarian landing hero cut off on a 393 px phone: the hero grid had no explicit column, so its auto track grew to its widest word (416 px) while an overflow-x: hidden wrapper hid the page-level overflow. grid-cols-1 caps the track. The fixtures now fail on visible text that crosses the screen edge, which the page-width check alone cannot see. The sweep's judgment lists the technical loanwords the Spanish copy uses on purpose. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… edge Codex review round 2: the edge guard measured element boxes, so nowrap text cut off inside a box at the edge passed, a wide box holding short text failed, and text clipped away entirely (a panned canvas node, an off-canvas drawer) failed. Measure each text node and clip it through every clipping ancestor; exempt the pannable Flowchart canvas. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…rage # Conflicts: # package-lock.json
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
ℹ️ Recent review info⚙️ Run configuration
📒 Files selected for processing (8)
🚧 Files skipped from review as they are similar to previous changes (3)
Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review. 📝 WalkthroughWalkthroughThe pull request expands agentic end-to-end coverage and updates fixtures, replay records, runner configuration, and CI execution. It also adds multilingual page checks, localized Flowchart controls, and several form and page adjustments. ChangesAgentic E2E coverage and execution
Application UI updates
Estimated code review effort: 4 (Complex) | ~45 minutes
Note 🎁 Summarized by CodeRabbit FreeYour organization is on the Free plan. CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please upgrade your subscription to CodeRabbit Essentials by visiting https://app.coderabbit.ai/settings/billing. Comment |
The clipped-text guard from b1cba08 flagged the mobile header's truncated motion: its text box runs past the screen, but its own ellipsis box cuts it off at 360 px. Flag text only when what shows of it reaches the screen edge. Nine injected layouts pin the rule; the guard still fails on main's mobile hero. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
@coderabbitai review |
|
…e guard Codex review round 3: any clip or clip-path exempted text outright, and a clip on one axis clipped both. Clip each axis only where its overflow clips, clip to the box for clip and clip-path, and skip only a zero-area clip (sr-only). agentic/edge-guard.e2e.ts pins the guard on eleven injected layouts, sized to the viewport so both targets run them. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
@coderabbitai review |
|
…areas Codex review round 4: a fixed or absolute box escapes the overflow of ancestors outside its containing block, a clip-path inset or a zero-height clip can hide a whole box, and two guard cases passed even with clipping removed. Clip only along the containing-block chain, compute clip and inset areas, and rebuild every case so its text crosses the screen edge before clipping. Removing clipping fails 7 cases; removing the containing-block rule fails the fixed-escape case. Text cut short inside the screen stays out of scope: the judge checks it. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…page claims Codex review round 5: inset(10%) took the height for its horizontal cuts, and backdrop-filter and the translate, rotate and scale properties also make a containing block for fixed boxes. Two new guard cases fail on the previous guard. The page sweep covers the student and public pages, not /app/admin or /share/:code, and the docs now say so. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
@coderabbitai review |
✅ Action performedReview finished.
|



What this changes
The agentic suite from #99 was well built but thin, and it had never run in CI. This PR widens it to every tool and to every page a student or visitor reaches. It also makes the suite fail on errors the screen hides, and fixes the app bugs it found on its first runs.
The suite
failOnSkippedFailure. The replay cache re-recorded, as it must after a minor bump.agentic/support/capture-errors.jsruns before the app in every document. It records uncaught errors, rejected promises,console.error(React warnings and the route error boundary included) and 5xx answers from Supabase. Every test fails on them.overflow-x: hiddenwrapper would otherwise hide. The guard measures each text node and clips it the way the browser does: along its containing-block chain, and byclipandclip-path. Text clipped away entirely (an off-canvas drawer, a panned canvas node, ansr-onlylabel) or inside a sideways scroller does not count. Text cut short inside the screen, such as an ellipsis or a scroll area, is out of its scope on purpose, and the judge checks it.agentic/edge-guard.e2e.tspins the rules on 17 injected layouts. Each one puts its text across the edge before clipping: removing clipping fails 7 of them, removing the containing-block rule fails the fixed-position case, and the two cases added inbdd73dbfail on the guard before it.agentic/pages.e2e.ts): all 14 student pages and 5 public pages, in English, Spanish and Hungarian, on desktop and mobile, for 114 tests. It skips/app/admin(admins only) and/share/:code(no client code creates a share link yet). Selectors checklang, a heading, and that noundefined,NaNor[object Object]leaked into the page. The judge checks for error notices, raw translation keys, untranslated interface text and clipped text. It replaceshungarian.e2e.ts, which covered 7 pages in one language.agentic/forms.e2e.tscovers empty submits on the four AI forms.qwen/qwen3.8-27b:free, so every cache miss and every exploration failed withThis model is unavailable for free. The actor is nowopenai/gpt-6-luna. It only runs on a cache miss or in an exploration.agentic-e2e.yml):src/,public/, Edge Functions, migrations, dependencies), not only the suite.pagesandrepro, so an untagged test still runs.docs/standards/security.md.docs/testing.md, the verify-debatelab skill, AGENTS.md, and the vendored e2e skill, refreshed from 0.17.0 with the DebateLab block re-added.App bugs the suite found, fixed here
ZodError.@hookform/resolvers3 cannot read zod 4's errors and rethrows them. The fix is an upgrade to 5.4.0, the newest release without the@typeschemapeer chain that npm rejects. It is pinned exactly, because a caret range resolves to 5.9.1 whenever the lockfile is regenerated.FactoidsFormnow types its input and output separately, because its schema coerces and defaults fields.grid-cols-1caps it.<div>inside a<p>(p > div > p). React's DOM-nesting warning caught it.account-data.spec.tsfind the output-language dropdown by name, which pins the fix.Not changed
[Meta] The requested model was not foundon roughly one call in a hundred. CI's default retry absorbs it; locally, use--last-failed.Verification
All of this ran locally. GitHub Actions cannot run on this account until its billing block is cleared, and every Agentic E2E run since #99 stopped after 2 seconds with no steps. None of this is a CI pass.
bdd73db, everything except the page sweep ran on both viewports and passed 179 of 180. That covers the 32 journey and form tests, all 34 guard cases (17 layouts on each viewport), and a guard-only probe of all 114 page, language and viewport combinations. The one failure was the transient provider error, and it passed on--last-failed.b45a069: 140 of 146 passed in 49 minutes for $0.11. Of the 6 failures, 2 were the transient provider error, 3 were the judge flagging Spanish loanwords (prompt, tokens, chats, email), and 1 was the hero overflow. After the loanword wording and the hero fix, those pages and Flowchart passed at5276486(the 2 provider errors among them passed on--last-failed). Later commits change only the fixtures' edge guard, test files and docs.CI=1) passed 24 of 24 in 3 minutes for under a cent. 28 flows replayed, and 4 fell back to the actor.main's hero and passes with the fix.npm run verifyon the merged head9339f03passed docs, lint (withknipand the repo rules from Review standards and lint checks from past PR reviews #108), both typechecks, 273 unit tests, 30 collaboration tests, 318 Edge tests, build, service worker, and 29 desktop@smokePlaywright tests. The account recipe passed 5 of 5 on desktop and on mobile Chromium, ande2e/full-stack/account-data.spec.tspassed 4 of 4. Atbdd73db, the agentic typecheck, lint anddocs:checkpass../.superset/db-gate.sh) passed atbdd73db: pgTAP 419 of 419, and full-stack Playwright 71 passed with 1 skipped.c61a4c2. Round 2 found two medium issues in the first version of the edge guard (it measured element boxes and ignored clipping), fixed inb1cba08. Round 3 found the guard flagging an ellipsis-truncated motion (my mobile journey run caught it too, fixed in8e7a499) and two clipping gaps, fixed in8a5e455. Round 4 found two clipping gaps (fixed-position escape, clip areas) and two guard cases that did not exercise their rule, all fixed in0199c6e. It also flagged text cut short inside the screen without an ellipsis, which I left out of the guard's scope: scroll areas, carousels and line clamps clip there by design, and the judge checks cut-off text. Round 5 found two low gaps (percentage insets, more containing-block properties) and an overstated "every page" claim, all fixed inbdd73db. Round 6 onbdd73dbfound no code issues; its one note, stale guard counts in this description, is fixed here.Not verified: live mode (
up --live), theLIVEbranches of the journeys, WebKit, and any GitHub Actions run.🤖 Generated with Claude Code