Skip to content

Security and compliance: bot auto-merge gate, worker scan window, TGA limbs (d) and (e) - #3742

Merged
BigSimmo merged 11 commits into
mainfrom
claude/project-thread-5zy0n0
Oct 22, 2026
Merged

BigSimmo merged 11 commits into
mainfrom
claude/project-thread-5zy0n0

Conversation

@BigSimmo

@BigSimmo BigSimmo commented Oct 21, 2026 •

Copy link
Copy Markdown
Owner

Requested by Josh · project thread

Summary

Four changes. The owner approved each one on 21 Oct 2026, first with "Go ahead with all recommendations please" and then the quote cards and reseal with "Go ahead with, 2 and 3 and 4 5 and 6". On 22 Oct the owner also approved them on this PR in his own words, including the reseal.

  • Bot auto-merge gate (#YF0XSQ). PR policy now fails a clinical-risk PR while auto-merge is armed by any account other than the repository owner. This enforces the 2026-10-20 decision that Gitar never arms clinical-risk PRs. On 13 Oct, gitar-bot armed PR 3507 and it merged on bot approvals alone. Turning auto-merge off re-runs the check and it passes. If the check can't identify the account that armed it, it fails closed. docs/agents/pull-request-workflow.md records this.
  • Worker image scan advisory until 4 Nov 2026, on main only. HIGH and CRITICAL findings in the worker image no longer fail the image scan on main until 4 Nov. A red worker scan was holding up production deploys under Railway's Wait for CI while Backport native worker parser and mount security fixes #3701 is finished. Release branches, dispatches from any other ref, app image findings and unassessed scans still fail, and the window ends by itself. Decision: docs/decisions/2026-10-21-worker-scan-advisory-until-4-nov.md.
  • TGA exemption limb (d). Both answer prompts gain one line: do not make the diagnosis or the treatment decision for a specific patient, and leave that to the treating clinician. The legacy prompt-cache revision rolled. The response cache is keyed by prompt version with a 5-minute default TTL, so stale answers age out within minutes. Decision: docs/decisions/2026-10-21-answers-leave-diagnosis-to-clinician.md.
  • TGA exemption limb (e): exact quotes at medium trust. Medium-trust answers now carry up to 2 exact quote cards. High trust still carries 3, and low trust carries none. A medium answer that is authority-capped still carries none, so an unverified-authority source never gets exact-quote presentation. The only live surface is the source drawer's cited passage, which now shows the exact quote rather than the snippet. A local screenshot check found no visible difference with demo content.
  • docs/clinical-hazard-controls.json was resealed (833f34b) after the owner's approval. That records the reviewed digests for src/lib/answer-render-policy.ts and its test and removes the temporary drift exceptions.

Areas touched: Delivery and assurance, Knowledge and records, Answers

RAG impact: behaviour change. The answer prompt text and the render caps changed, but retrieval and ranking did not. The owner approved one before and after eval canary pair, capped at US$2. The canary only runs from main, so the before run goes on main before merge and the after run follows the merge. A regression in the after run means an immediate revert of this PR.

Verification

  • Checks run: node scripts/pr-policy.mjs --self-test passed, including five new cases. npx vitest run tests/pr-policy*.test.ts passed 39 tests. tests/trivy-image-scan.test.ts, tests/container-ci-contract.test.ts and tests/ci-audit-contracts.test.ts passed 148 tests, including the advisory-window and main-only cases. tests/answer-render-policy.test.ts passed. The prompt and answer tests (rag-prompt-injection, rag-answer-composition-prompt, rag-answer-fallback) passed. The client payload, clinical safety, trust, dashboard helper, demo boundary and decision tests passed. check:clinical-hazard-controls passed after the reseal. check:env-parity and check:repo-awareness-snapshot passed. The pre-push static guard (lint and typecheck) passed.
  • Verification not run: the eval canary pair after merge. The before run, 39630009061, is green and recorded on this PR. The after run is provider-backed and runs from main only.

Risk and rollout

  • Risk: medium. The prompt change can alter answer wording for patient-scenario questions, which the canary pair checks. The worker scan window lowers a security gate for 14 days, for the worker image on main only. The auto-merge gate affects only clinical-risk PRs that a non-owner arms.
  • Rollback: revert this PR. The worker window also ends by itself on 4 Nov 2026.
  • Provider or production effects: none from the PR itself. Merging changes production answer prompts and medium-trust rendering once Railway deploys. The canary pair is one approved, capped run.

Clinical Governance Preflight

  • Source-backed claims still require linked source verification before clinical use
  • No patient-identifiable document workflow was introduced or expanded without explicit governance approval
  • Supabase target remains PsychSift Production (sjrfecxgysukkwxsowpy)
  • Service-role keys and private document access remain server-only
  • Demo/synthetic content remains clearly separated from real clinical sources
  • Source metadata, review status, and outdated/unknown-source behavior remain conservative
  • Deployment classification/TGA SaMD impact was checked when clinical decision-support behavior changed

The prompt line and the quote cards keep the amended TGA exemption limbs (d) and (e) from 1 Nov 2026. See docs/clinical-hazard-analysis.md, "Amended exemption from 1 November 2026".

🤖 Generated with Claude Code

https://claude.ai/code/session_01W7EHqBKdEqicWVBdYqv4W5

…XSQ)

PR policy now fails a clinical-risk PR while auto-merge on it was armed by
any account other than the repository owner. This enforces the 2026-10-20
decision that Gitar never arms clinical-risk PRs (gitar-bot armed PR 3507,
which then merged on bot approvals alone). The owner may still arm it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7EHqBKdEqicWVBdYqv4W5
@BigSimmo BigSimmo self-assigned this Oct 21, 2026
@coderabbitai

coderabbitai Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

Your organization has reached its usage spending cap. Adjust your spending cap in the billing tab.

Next included review available in 17 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available. Your 91 included PR review attempts over the past 7 days set your current allowance at 1 review per hour.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Repository: BigSimmo/PsychSift/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Team
  • Run ID: 7c37398d-c21d-4bec-950f-20373f102f64
📥 Commits

Reviewing files that changed from the base of the PR and between 92d37ba and 035c6cd.

📒 Files selected for processing: 12
  • .github/workflows/pr-policy.yml
  • data/repo-awareness-snapshot.json
  • docs/agents/pull-request-workflow.md
  • docs/clinical-hazard-controls.json
  • docs/decisions/2026-10-21-answers-leave-diagnosis-to-clinician.md
  • docs/decisions/2026-10-21-worker-scan-advisory-until-4-nov.md
  • scripts/pr-policy.mjs
  • scripts/trivy-image-scan.mjs
  • src/lib/answer-render-policy.ts
  • src/lib/rag/rag-answer-instructions.ts
  • tests/answer-render-policy.test.ts
  • tests/trivy-image-scan.test.ts
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@supabase

supabase Bot commented Oct 21, 2026

Copy link
Copy Markdown

This pull request has been ignored for the connected project sjrfecxgysukkwxsowpy because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.


Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

gitar-bot[bot]
gitar-bot Bot previously approved these changes Oct 21, 2026

@gitar-bot gitar-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gitar has auto-approved this PR (configure)

…decision to the clinician

Owner decisions 2026-10-21:
- HIGH/CRITICAL findings in the worker image are advisory on main until
  4 Nov 2026 so a red worker scan stops holding up deploys. App image
  findings and unassessed scans still fail; the window ends by itself.
- Both answer prompts gain the TGA limb (d) line: no diagnosis or treatment
  decision for a specific patient. Legacy prompt-cache revision rolled.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7EHqBKdEqicWVBdYqv4W5
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7EHqBKdEqicWVBdYqv4W5
@BigSimmo BigSimmo changed the title fix(pr-policy): block bot-armed auto-merge on clinical-risk PRs Security and compliance: bot auto-merge gate, worker scan window, TGA limb (d) prompt line Oct 21, 2026
gitar-bot[bot]
gitar-bot Bot previously approved these changes Oct 21, 2026

@gitar-bot gitar-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gitar has auto-approved this PR (configure)

… records

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7EHqBKdEqicWVBdYqv4W5
gitar-bot[bot]
gitar-bot Bot previously approved these changes Oct 21, 2026

@gitar-bot gitar-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gitar has auto-approved this PR (configure)

…mb e)

Medium-trust answers now show the exact sentences relied on, capped at 2.
Authority-capped medium answers still show none. DEMO_ANSWER_CONFIDENCE=medium
renders supported demo answers at medium trust for local screenshots with no
provider call. Built for owner screenshot review; merges only after approval.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7EHqBKdEqicWVBdYqv4W5
@BigSimmo BigSimmo changed the title Security and compliance: bot auto-merge gate, worker scan window, TGA limb (d) prompt line Security and compliance: bot auto-merge gate, worker scan window, TGA limbs (d) and (e) Oct 21, 2026
gitar-bot[bot]
gitar-bot Bot previously approved these changes Oct 21, 2026

@gitar-bot gitar-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gitar has auto-approved this PR (configure)

…ng owner screenshot approval

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7EHqBKdEqicWVBdYqv4W5
gitar-bot[bot]
gitar-bot Bot previously approved these changes Oct 21, 2026

@gitar-bot gitar-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gitar has auto-approved this PR (configure)

…l of medium-trust quotes

The local screenshot check showed no visible difference with demo content, so
the switch served no one. Owner approved the change on 21 Oct.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7EHqBKdEqicWVBdYqv4W5
…-5zy0n0

# Conflicts:
#	tests/trivy-image-scan.test.ts
gitar-bot[bot]
gitar-bot Bot previously approved these changes Oct 21, 2026

@gitar-bot gitar-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gitar has auto-approved this PR (configure)

Copy link
Copy Markdown
Owner Author

Canary pair: "before" baseline recorded. This is Eval Canary run 39630009061 on main 8c03823, owner-dispatched on 21 Oct 2026 at 15:55Z. It went green.

  • Golden retrieval: 36 of 36 cases passed, all at docRecall@5 = 1.00 and contentRecall@5 = 1.00.
  • Answer-quality subset: 44 cases. Grounded supported rate 1, unsupported correct rate 1, citation failure rate 0, numeric grounding failure rate 0, route ceiling failures 0. One case was marked failed (1) within thresholds: admission-discharge-coverage-paraphrase. There were 2 generation fallbacks (summary-discharge-guidance, nocc-requirements). P95 latency was 11396 ms.
  • Estimated cost: US$0.12 (cap US$2 for the pair).

The "after" run follows the merge, because the canary only runs from main. It must show doc and content recall still at 1.00, no per-case rr regression, and no new failed answer cases. If any regress, this PR is reverted at once.


Generated by Claude Code

@BigSimmo
BigSimmo marked this pull request as ready for review October 21, 2026 17:03
@BigSimmo
BigSimmo enabled auto-merge October 21, 2026 17:04
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-22T01:06:53.358253Z b634c8b Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BigSimmo has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.

@claude

claude Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Claude encountered an error after 2s —— View job


I'll analyze this and get back to you.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 831ff94ac9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/trivy-image-scan.mjs Resolved
A release branch push or a manual dispatch from another ref now blocks on
worker HIGH or CRITICAL findings, as the Codex review asked.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7EHqBKdEqicWVBdYqv4W5
gitar-bot[bot]
gitar-bot Bot previously approved these changes Oct 21, 2026

@gitar-bot gitar-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gitar has auto-approved this PR (configure)

@gitar-bot

gitar-bot Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor
Code Review ✅ Approved

🔴 High risk · Making worker-image HIGH/CRITICAL findings advisory on main until Nov 4 could let vulnerable images pass CI and reach production.

Adds a PR policy gate that blocks bot-armed auto-merge on clinical-risk PRs, a time-limited advisory window for HIGH/CRITICAL worker image scan findings until 4 Nov 2026, prompt lines for TGA exemption limb (d), and up to 2 exact quote cards at medium trust for limb (e). No issues found.

Verification

Rules No rules evaluated

Functional validation Not enabled

Set up

Auto-approval Approved

For more verification details, comment gitar display:verbose.

Options

Auto-apply is off → Gitar will not commit updates to this branch.
Display: compact → Showing compact verification results.

Comment with these commands to change the behavior for this request:

Auto-apply Compact
gitar auto-apply:on         
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Gitar

@BigSimmo
BigSimmo disabled auto-merge October 21, 2026 17:21
@BigSimmo

Copy link
Copy Markdown
Owner Author

PR sweep (22 Oct): I've turned off auto-merge here. This PR changes owner sign-off areas (PR policy, seals, clinical content or visible UI). The PR body summarises the owner's approvals, but there is no owner approval written in the owner's own words on the PR. Auto-merge stays off until the owner posts one. Sweep agents won't re-arm it.

Copy link
Copy Markdown
Owner Author

Lighthouse budget failed on b634c8b: desktop-root LCP +217 ms (+22.7%) and TBT +76 ms against the baseline. This PR doesn't touch the home page or any client bundle. The last commit only changed scripts/trivy-image-scan.mjs and its test. The same lane passed on the previous head, 831ff94, so this looks like runner timing noise, like the slow-runner failures seen earlier today. I'll re-run the failed job once when the workflow finishes. The baseline stays as it is.


Generated by Claude Code

@BigSimmo
BigSimmo marked this pull request as draft October 21, 2026 18:02
@BigSimmo

Copy link
Copy Markdown
Owner Author

Moved to draft by Grok Bot while it waits for the owner's own-words approval, because gitar-bot keeps re-arming auto-merge on disarmed PRs. Mark ready again once he posts approval here.

@BigSimmo
BigSimmo marked this pull request as ready for review October 22, 2026 01:04

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BigSimmo has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b634c8bec4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread docs/clinical-hazard-controls.json Resolved
…exceptions

Ran npm run governance:seal-hazard-controls, which reseals the reviewed digests for
src/lib/answer-render-policy.ts and tests/answer-render-policy.test.ts and clears the
two driftExceptions entries, per the owner's decision and the Codex P1 review thread.
@BigSimmo

Copy link
Copy Markdown
Owner Author

I approve this PR, including the medium-trust quote cards, the TGA prompt line and the worker scan window to 4 Nov."

@BigSimmo

Copy link
Copy Markdown
Owner Author

"I approve resealing the clinical hazard controls on #3742 and removing the path-wide drift exception."

Copy link
Copy Markdown
Owner Author

Static PR checks failed on 833f34b at check:ledger-write-discipline. The five docs/outstanding-issues-inbox/applied/*.json files it named came from main (00317f9), not from this PR. The check compared the merge ref against the stale PR base, 8c03823. I merged origin/main (92d37ba) in 035c6cd. Locally, the ledger check now passes for 92d37ba..HEAD, check:clinical-hazard-controls passes, and the trivy and answer-render tests pass (99 tests).


Generated by Claude Code

@BigSimmo
BigSimmo merged commit 472bee2 into main Oct 22, 2026
32 checks passed
@BigSimmo
BigSimmo deleted the claude/project-thread-5zy0n0 branch October 22, 2026 02:34

Copy link
Copy Markdown
Owner Author

Canary pair, "after" run: no regression. No revert. Eval Canary 39742639771 ran on main 4a79008, which includes this PR, and went green.

  • Golden retrieval: 36 of 36 passed. docRecall@5 and contentRecall@5 are 1.00 on every case. Per-case rr@10 and contentRR@10 are identical to the baseline, 39630009061, with zero differences across all 36 cases.
  • Answer-quality subset: 44 cases, with substantive grounded rate 1. Only admission-discharge-coverage-paraphrase is marked failed (1), the same case as the baseline. There are no new failed cases.
  • Generation fallbacks went from 2 to 3. The new one is neuroleptic-side-effect-escalation (numeric_band_conflict_source_withheld), and the case passed. That reason code comes from the source-conflict handling in Answer safety hardening: source tier rule, superseded sources out of evidence, national guidance line, tests and ledger #3743, which also landed in this run, so it's for the Answer safety thread to judge.
  • P95 latency was 8464 ms, against a baseline of 11396 ms. Estimated cost was US$0.13, so the pair came to about US$0.25 against the US$2 cap.

Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants