Skip to content

Act on the fresh review: accuracy, accessibility, and the stale Markdown copy - #4

Merged
vahid-ahmadi merged 1 commit into
mainfrom
fix/review-followups
Aug 12, 2026
Merged

vahid-ahmadi merged 1 commit into
mainfrom
fix/review-followups

Conversation

@vahid-ahmadi

Copy link
Copy Markdown
Contributor

Acts on a fresh independent review of the article. Mechanical and clear-cut fixes only — the editorial calls are listed at the bottom, untouched.

Wrong or misleading

  • "They fall into one of five types" — plural subject, singular predicate. The Markdown copy had it right; the page had regressed.
  • "without switching tools" contradicted the section immediately above it, which is entirely about the model switching between 21 tools. The point is that the reader does not leave the conversation.
  • The worked-example chip did not match the request the prose quotes. It omitted "Child Benefit" and the requested output, while stage 1 claimed to extract "budgetary impact" from "the user's exact words" and stage 2 bound gov.hmrc.child_benefit.amount.eldest from wording that was never shown.
  • gateway route and full lifecycle rendered as <code> chips styled identically to validate_reform and run_household_simulation, so they read as public tool names. They are not tools; the row is now omitted when a question kind has none.
  • Tax revenue rising on a benefit increase reads as a sign error. Now states that the High Income Child Benefit Charge is why.
  • content/current-article.md had drifted badly — missing the PolicyBench paragraph, "The boundary", and "What you can ask" entirely, and carrying a different results table — while the README called it "the canonical editorial reference". Regenerated from the page. The README now says the page is canonical, because a hand-synced duplicate will drift again.

Accessibility

  • --text-light was #7a9494, 3.2:1 on white, under the 4.5:1 minimum, and it carried the results-table sign-convention caveat plus every card meta line. Darkened at the token, which fixes six call sites at once.
  • The six stages were <span>s, so a screen-reader user could not navigate them by heading, and the sections had no accessible name. Now <h3> with aria-labelledby; the CSS already gave .step-title its own size, so nothing moves visually.
  • The boundary diagram rendered its labels at about 3.5px on a phone — max-width: 100% prevented .agent-flow-container's overflow-x: auto from ever engaging. A min-width lets it scroll.
  • Keyboard focus on a tool chip was styled identically to selection, so two chips read as active while tabbing.
  • The question panel's aria-live region re-announced what the tab change had already announced.

Also

Capped the measure at 78ch (running text was ~140 characters a line), added a publication date to a piece whose figures are pinned to a dataset release, dropped the worked-example timeline's hover affordances since it has no click handler, set lang="en-GB", and removed instructional filler ("Explore these categories in the interactive below", "(more on those later)") and meta-commentary about the article's own structure.

Deliberately not changed

  • The CTA host still points at policyengine-uk-chat.vercel.app.
  • No evaluation of UK Chat itself. PolicyBench measures unaided US models — the premise, not the conclusion — and Limitations does not acknowledge the gap.
  • The gate runs on a model-generated plan. A confident misreading with the right parameter path passes cleanly and returns a correct number answering the wrong question. The closing caveat covers ambiguity, not confident misinterpretation.

The reviewer also could not verify the article's claims about the chat service — the 30-round cap, the five gate outcomes, the SSE event types, "no generic agent framework in the request path" — against policyengine-uk-chat, which is a separate repo. Worth checking before publication.

bun run check passes.

Wrong or misleading:
- "They fall into one of five types" — plural subject, singular predicate. The
  Markdown had it right; the page had regressed.
- "without switching tools" contradicted the section above it, which is about
  the model switching between 21 tools. The point is that the reader does not
  leave the conversation.
- The worked-example chip omitted "Child Benefit" and the requested output,
  while stage 1 claimed to extract both from "the user's exact words" and
  stage 2 bound a Child Benefit parameter from wording that was not shown.
- "gateway route" and "full lifecycle" rendered as code chips styled exactly
  like validate_reform, so they read as public tool names. They are not.
- Tax revenue rising on a benefit increase reads as a sign error; say that the
  High Income Child Benefit Charge is why.
- content/current-article.md had drifted badly: three sections missing and a
  different results table. Regenerated from the page, and the README no longer
  calls it canonical, because a hand-synced duplicate will drift again.

Accessibility:
- --text-light was #7a9494, 3.2:1 on white, below the 4.5:1 minimum, and it
  carried the results-table caveat and every card meta line.
- The six stages were spans, so they could not be navigated by heading, and
  their sections had no accessible name.
- The boundary diagram scaled its labels to about 3.5px on a phone because
  max-width: 100% stopped the container's overflow-x from ever engaging.
- Keyboard focus on a tool chip looked identical to selection, so two chips
  read as active at once while tabbing.
- The question panel's live region re-announced content the tab change had
  already announced.

Also: capped the measure at 78ch (it ran to ~140 characters a line), added a
publication date to a piece whose figures are version-pinned, dropped the
timeline's hover affordances since it has no click handler, set lang="en-GB",
and removed instructional filler and meta-commentary.

Left alone deliberately: the CTA host, the absence of a UK Chat evaluation,
and the gap between a deterministic gate and the model-generated plan it runs
on. Those are editorial calls.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@vercel

vercel Bot commented Aug 12, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
policyengine-uk-chat-article Ready Ready Preview Aug 12, 2026 1:48pm

Request Review

@vahid-ahmadi
vahid-ahmadi merged commit 388133a into main Aug 12, 2026
3 checks passed
@vahid-ahmadi
vahid-ahmadi deleted the fix/review-followups branch August 12, 2026 13:53

This branch was successfully deployed

1 active deployment
Preview — 9fd15441 Deployed Aug 12, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant