Repository navigation
Answer safety hardening: source tier rule, superseded sources out of evidence, national guidance line, tests and ledger - #3743
Conversation
…answer loader Tier-rule tests for #BFCTST (WA, national, other states, supplementary within a relevance band) and synthetic gating tests for registered-public-answer-options.ts (audit DOC-18). Ledger inbox: close #HD1C6W, #F3B2XC and #VXB8XA as already fixed on main, and bring #ZZ4RAP and #BFCTST up to date. No answer behaviour change. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Jchk5iTkeUyPRUoCHyWei6
|
Warning Review limit reachedYour organization has reached its usage spending cap. Adjust your spending cap in the billing tab. Next included review available in 14 minutes. View limit details
Comment |
|
This pull request has been ignored for the connected project Preview Branches by Supabase. |
![gitar-bot[bot]](https://avatars.githubusercontent.com/in/827041?s=60&v=4){kind=small}
Owner rule of 2026-09-25 (#BFCTST), approved for build 2026-10-21. Outdated documents are removed once, before every answer route sees the ranked results, instead of only being penalised in ranking. review_due and unknown sources are unchanged. A read-only live count on 2026-10-21 found no indexed document marked outdated, so no current answer should change. Two fallback tests that used an outdated row only as a convenient governance-gap trigger now use poor extraction, which raises the same marker. Also: capture the KEMH lithium breastfeeding passage for #ZZ4RAP, close #E6142T, record the owner's cloZAPine decision, and log the missing WA valproate guideline. RAG impact: behaviour change - outdated sources leave the evidence set; canary pair pending Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Jchk5iTkeUyPRUoCHyWei6
BigSimmo
changed the title
Owner decision 2026-10-21 (mockup approved, Live site visual checks item 8): when a WA guideline and a national guideline differ on a point, the answer still follows WA and one plain line under the prose names the national source, linking to its page. The line only comes from an owner-signed entry in src/data/reviewed-guidance-differences.json, which ships empty, so no live answer changes until the owner signs off a difference. Partial support, refusals, ungrounded answers and expired entries never show it. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Jchk5iTkeUyPRUoCHyWei6
BigSimmo
changed the title
…ld reject A signed entry with an out-of-range year or overlong title would have built a note that fails clientAnswerFieldsSchema, and the client rejects the whole answer in that case. Entries are now active only when their note already passes that schema, so a bad entry is dropped, as documented. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Jchk5iTkeUyPRUoCHyWei6
Code Review ✅ Approved 1 closed / 1 findings🔴 High risk · Filtering outdated documents from the shared RAG evidence set changes clinical answer grounding across routes and could leave queries without usable evidence. Adds a superseded-source filter so outdated documents leave answer evidence, a national guidance difference line under answers, and tests for source tier ordering and the answer loader. The registry validator mismatch that could drop whole answers over one bad entry was addressed. No open issues found. ✅ 1 closed✅ Bug: Registry validator is looser than client schema, so one bad entry drops the whole answer
VerificationRules No rules evaluated Functional validation Not enabledAuto-approval Approved For more verification details, comment OptionsAuto-apply is off → Gitar will not commit updates to this branch. Comment with these commands to change the behavior for this request:
Was this helpful? React with 👍 / 👎 | Gitar |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
There was a problem hiding this comment.
BigSimmo has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Claude encountered an error after 2s —— View job I'll analyze this and get back to you. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 622432df21
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…idance line in copies Review follow-ups on the superseded-source rule and the national guidance line: - Drop outdated rows before every text and memory-card fast-path selection, so an outdated lexical match cannot short-circuit the lanes that would find current guidance. - Refuse to summarise a document marked outdated instead of generating from it. - Copied answers carry the national guidance line directly after the answer text. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Jchk5iTkeUyPRUoCHyWei6
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Jchk5iTkeUyPRUoCHyWei6
…ts within budget rag.ts had grown past its 4362-line no-growth budget. The not-indexed and superseded-document refusals now come from documentSummaryUnavailableAnswer in rag-document-summary-context.ts; behaviour is unchanged. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Jchk5iTkeUyPRUoCHyWei6
|
Lighthouse budget is red on 410c02d. I think the cause is outside this PR.
Generated by Claude Code |
|
Auto-fix isn't included in your plan ↗ |
Requested by Josh · project thread
Summary
tests/australian-source-priority.test.ts: pins the owner tier rule from #BFCTST. At equal relevance the order is WA validated, then Australian national, then other states, then supplementary. Relevance always wins across bands. Supplementary padding is dropped once two Australian documents suffice.tests/registered-public-answer-options.test.ts(new): synthetic gating tests forsrc/lib/registered-public-answer-options.ts(audit DOC-18).src/lib/rag/superseded-evidence.tsremovesdocument_status: "outdated"rows once insrc/lib/rag/rag.ts, before every answer route.review_dueandunknownare unchanged.src/lib/rag/guidance-difference-note.tsattaches it insrc/lib/answer-response.ts. It is passed throughclientAnswerFieldsSchemaand rendered byNaturalLanguageAnswer.src/data/reviewed-guidance-differences.json. That file ships empty, so no live answer changes until the owner signs off a difference.tests/fixtures/kemh-lithium-breastfeeding-live-excerpt.json: KEMH passage captured read-only for #ZZ4RAP case breastfeeding-lithium. It is not wired yet because the owner is correcting the draft.RAG impact: behaviour change. Superseded (outdated) sources leave the answer evidence set, and the live corpus has none today. The guidance line has no retrieval effect and ships with an empty list. Canary pair: owner-approved, capped at US$4. The baseline is the 2026-10-21 14:09Z main canary (run 39614340500). The after run is post-merge.
Verification
tests/guidance-difference-note.dom.test.tsx: 7 passed.Tests 845 passed (845).rag-answer-fallback: 179 passed. Tier and loader tests: 23 passed.npm run eval:rag:offline: 984 passed.npm run eval:rag:adversarial:offline: 25 passed.tsc --noEmitclean, eslint clean, prettier applied.Risk and rollout
Notes
npm run issues:reconcileafter merge.🤖 Generated with Claude Code
https://claude.ai/code/session_01Jchk5iTkeUyPRUoCHyWei6