Skip to content

OCR caution, scanner warning, table-fact labels and deleted-document titles (split from #3707) - #3715

Merged
gitar-bot[bot] merged 14 commits into
mainfrom
grok/3707b-ocr-scanner-tablefacts-titles
Oct 21, 2026
Merged

gitar-bot[bot] merged 14 commits into
mainfrom
grok/3707b-ocr-scanner-tablefacts-titles

Conversation

@BigSimmo

@BigSimmo BigSimmo commented Oct 21, 2026 •

Copy link
Copy Markdown
Owner

Split out of #3707 at the owner's request (21 Oct 2026, 14:42 AWST). This is part (b). #3707 keeps part (a), the Retry route and heartbeat.

Moved unchanged from #3707 (Claude's commits):

  • OCR caution: answers caution when a cited source has pages read from a scan by OCR. The document cautions stay inside the 8-warning cap. (src/lib/source-governance.ts, src/lib/types.ts, tests/source-governance.test.ts)
  • Scanner warning: the worker warns at start-up when the malware scanner module is not set (worker/admin-source-runtime.ts, worker/main.ts, runbook). Includes the decision record to keep the upload malware scan as is.
  • Table-fact label carry-forward (ING-B-09): a merged row label is carried into clinical_parameter (worker/table-facts.ts).
  • Titles: a deleted document's title is dropped from the storage clean-up ledger (src/lib/ingestion.ts, scripts/cleanup-storage.ts, privacy impact assessment). This is the app-side half of the Clear deleted documents' titles from older clean-up ledger rows (database) #3706 backfill.
  • The owner-approved temporary drift exception for tests/source-governance.test.ts (docs/clinical-hazard-controls.json).

RAG impact: behaviour change. The OCR caution is governance only. ING-B-09 changes table facts for newly indexed documents only. The canary pair belongs around the re-index that first applies it.

Blocked: owner reseal needed (clinical)

Two unit tests fail, and an agent can't clear either:

  • clinical-doc-line-anchors: docs/clinical-hazard-analysis.md:93 cites source-governance.ts:364 for hasDangerSourceGovernanceWarning, which is now at line 393. Both files are sealed hazard-control paths, so refreshing the anchor changes a sealed digest.
  • challenger-m5-2-empirical-stress: it expects at least 15 unexcepted sealed control paths, and the drift exception brings it to 14. Lowering that number would loosen a ratchet, which is not allowed.

Clearing these needs the owner to re-review and reseal (governance:seal-hazard-controls) src/lib/source-governance.ts, tests/source-governance.test.ts and docs/clinical-hazard-analysis.md. The anchor refresh can then land with the reseal. The owner confirmed the wording on #3707 (05:18 UTC, 21 Oct). Stays a draft until then.

Evidence (local): table-facts, ingestion, worker-admin-source-runtime, source-governance and worker-shadow-extraction tests pass. The only failures are the two above.


Summary by cubic

Adds an OCR caution (not a refusal) when a cited source has pages read by OCR, a worker start-up warning when the malware scanner module is unset, a merged table-row label carried into clinical_parameter for newly indexed documents, and drops a deleted document's title from the storage clean-up ledger. The grouped OCR caution holds one reserved slot so the 8-warning cap can never drop it or crowd out document-specific warnings, and the decision record keeps the upload malware scan unchanged.

Table-fact label carry-forward now inherits only into cells the Python extractor confirmed as covered by a merged span (PyMuPDF None provenance), and resets at an empty separator row, a row with its own label, a section heading, or a page/segment break. The owner-approved hazard-controls reseal landed with the anchor refresh and regenerated hazard-register snapshot, clearing the previously failing clinical-doc-line-anchors and challenger-m5-2-empirical-stress checks.

Written for commit 6d61d29. Summary will update on new commits.

View guided diff Turn on auto-fix

… deleted-document titles (split from #3707)

Owner-approved split of #3707 (21 Oct 2026, 14:42 AWST). Moved unchanged
from #3707 (Claude's commits af2dfc0, 2158725 caution part, 2abb635,
3752f2e, 4804078, 02bea13, 0f7b568):
- scanned-page (OCR) source caution, kept inside the warning cap
- worker start-up warning when the malware scanner module is not set,
  plus the decision record to keep the upload scan as is
- table-fact merged row label carried into clinical_parameter (ING-B-09)
- deleted document's title dropped from the storage clean-up ledger
- the owner-approved temporary drift exception for source-governance tests
@supabase

supabase Bot commented Oct 21, 2026

Copy link
Copy Markdown

This pull request has been ignored for the connected project sjrfecxgysukkwxsowpy because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.


Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

@coderabbitai

coderabbitai Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

Your organization has reached its usage spending cap. Adjust your spending cap in the billing tab.

Next included review available in 56 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available. Your 87 included PR review attempts over the past 7 days set your current allowance at 1 review per hour.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Repository: BigSimmo/PsychSift/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Team
  • Run ID: 80c58e28-de00-4e99-ab8d-9c6a6aaa48a1
📥 Commits

Reviewing files that changed from the base of the PR and between 42509e9 and 6d61d29.

📒 Files selected for processing: 24
  • data/hazard-register-snapshot.json
  • data/repo-awareness-snapshot.json
  • docs/clinical-hazard-analysis.md
  • docs/clinical-hazard-controls.json
  • docs/decisions/2026-10-21-no-malware-scan-before-upload-storage.md
  • docs/privacy-impact-assessment.md
  • docs/worker-deploy-runbook.md
  • scripts/cleanup-storage.ts
  • src/lib/ingestion.ts
  • src/lib/source-governance.ts
  • src/lib/types.ts
  • tests/extracted-document-validation.test.ts
  • tests/ingestion.test.ts
  • tests/source-governance.test.ts
  • tests/table-facts.test.ts
  • tests/worker-admin-source-runtime.test.ts
  • worker/admin-source-runtime.ts
  • worker/image-captioning.ts
  • worker/image-classification.ts
  • worker/main.ts
  • worker/python/extract_pdf_assets.py
  • worker/python/test_extract_pdf_assets_merged_cells.py
  • worker/python/test_extract_pdf_assets_raster_tables.py
  • worker/table-facts.ts
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

gitar-bot[bot]
gitar-bot Bot previously approved these changes Oct 21, 2026

@gitar-bot gitar-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gitar has auto-approved this PR (configure)

@BigSimmo

Copy link
Copy Markdown
Owner Author

Owner approval (typed, relayed via Grok Bot, Oct 21 2026 15:46 AWST), verbatim: "I approve resealing #3715's hazard controls with the anchor refresh."

…nalysis anchor refresh (source-governance.ts:364 -> :393)

Owner approval typed 2026-10-21 15:46 AWST on #3715. Sealed with npm run governance:seal-hazard-controls.
@BigSimmo

Copy link
Copy Markdown
Owner Author

Owner approvals (typed, relayed via Grok Bot, Oct 21 2026 15:49 AWST), verbatim:

  1. "I approve the full 30-path hazard-control reseal for OCR caution, scanner warning, table-fact labels and deleted-document titles (split from #3707) #3715, clearing the 16 earlier drift exceptions."
  2. "I approve OCR caution, scanner warning, table-fact labels and deleted-document titles (split from #3707) #3715's OCR caution wording and carrying merged row labels down in table facts."

Comment thread docs/clinical-hazard-controls.json Resolved

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BigSimmo has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-21T08:21:06.538076Z 4bc2374 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@claude

claude Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Claude encountered an error after 2s —— View job


I'll analyze this and get back to you.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4bc23745b4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread worker/table-facts.ts Resolved
Comment thread worker/table-facts.ts Resolved
gitar-bot[bot]
gitar-bot Bot previously approved these changes Oct 21, 2026

@gitar-bot gitar-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gitar has auto-approved this PR (configure)

@BigSimmo

Copy link
Copy Markdown
Owner Author

I approve restricting #3715's row-label carry-down to cells the parser marks as merged.

gitar-bot[bot]
gitar-bot Bot previously approved these changes Oct 21, 2026

@gitar-bot gitar-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gitar has auto-approved this PR (configure)

@BigSimmo
BigSimmo disabled auto-merge October 21, 2026 10:04
@BigSimmo

Copy link
Copy Markdown
Owner Author

Auto-merge disabled again: gitar-bot re-armed this after Helper 3 (the owner) disarmed it for the pending OCR-cap fix. Per the owner's rule, no bot re-arms a disarmed PR. Helper 3 will re-arm it once everything is in and green.

gitar-bot[bot]
gitar-bot Bot previously approved these changes Oct 21, 2026

@gitar-bot gitar-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gitar has auto-approved this PR (configure)

@BigSimmo

Copy link
Copy Markdown
Owner Author

I approve resealing #3715's hazard controls to include the grouped OCR warning that the warning cap never drops, and updating reviewedCommit/reviewedAt to that reseal.

gitar-bot[bot]
gitar-bot Bot previously approved these changes Oct 21, 2026

@gitar-bot gitar-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gitar has auto-approved this PR (configure)

@cubic-dev-ai

cubic-dev-ai Bot commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

Auto-fix isn't included in your plan ↗

Upgrade to Team

@gitar-bot gitar-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gitar has auto-approved this PR (configure)

@gitar-bot

gitar-bot Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor
CI failed: Lighthouse budget check failed on three total-blocking-time regressions (desktop root, mobile document search, mobile root); the mobile breaches were confirmed 3/3 times, but the desktop-root confirmation was lost to a NO_NAVSTART runtime error. The analyzer judged this unrelated to the PR, which touches OCR, table-fact, and governance code, not the tested pages.

Overview

One unique error template across 7 analyzed logs, with a single failing job (119188672916). The failure is a Lighthouse performance budget breach on the check:lighthouse-budget step. The analyzer reports no evidence linking it to this PR's OCR/table-facts/governance changes.

Failures

Lighthouse TBT budget regression on root and document-search pages (confidence: high)

  • Type: test
  • Affected jobs: 119188672916
  • Related to change: no
  • Root cause: Measured total-blocking-time exceeded tolerance on three cells: desktop-root 124 ms vs 64 ms baseline (+60 ms, +93%); mobile-documents-search 994 ms vs 679 ms (+315 ms, +46%); mobile-root 1382 ms vs 919 ms (+464 ms, +50.5%). The two mobile breaches were confirmed in all 3 samples. The desktop-root breach could not be confirmed because Lighthouse reported NO_NAVSTART, yet it was still counted as a regression. No change to the tested pages or their rendering appears in the supplied PR context.
  • Suggested fix: Profile main-thread work on the root and document-search pages, reduce blocking JavaScript or rendering work driving TBT, and rerun npm run verify:lighthouse. Rerun the desktop-root measurement to get a valid confirmation. Do not raise the budget unless the performance change is intentional and justified.

Summary

  • Change-related failures: None. The analyzer's verdict for the only finding is no.
  • Infrastructure/flaky failures: The desktop-root result is uncertain because of the NO_NAVSTART trace-recording error, so it may warrant a rerun before triage. It is not classified as flaky by the analyzer.
  • Recommended action: Treat as a pre-existing or unrelated performance regression. Investigate the TBT increase on the root and document-search pages separately, and rerun the desktop-root measurement to resolve the unconfirmed breach.
Code Review ✅ Approved 1 closed / 1 findings

🔴 High risk · Merged-cell carry-forward changes clinical table-fact labels for reindexed documents; incorrect span provenance could associate thresholds with the wrong parameter.

Adds a grouped OCR caution that reserves a slot within the 8-warning cap, a worker start-up warning when the malware scanner module is unset, merged table-row label carry-down for newly indexed documents, and removal of deleted document titles from the storage clean-up ledger. No open findings; the change is approved.

✅ 1 closed
Closed: Security: Reseal clears all 16 drift exceptions, not just the 3 approved files

📄 docs/clinical-hazard-controls.json:161-175
The PR says the owner's re-review and reseal covers src/lib/source-governance.ts, tests/source-governance.test.ts and docs/clinical-hazard-analysis.md. But governance:seal-hazard-controls (sealReviewedPathDigests in scripts/check-clinical-hazard-controls.mjs:589-603) re-digests every reviewed path and runs delete sealed.driftExceptions. So this commit also marks as reviewed 13 other sealed paths whose exceptions still said the owner had not reviewed them. Examples: src/lib/service-urgent-routing.ts and tests/services-catalog.test.ts (audit 3, "owner hazard re-review and reseal (H8) pending"), src/lib/answer-verification.ts with its test (PR #3546/#3628), clinical-safety.ts, answer-render-policy.ts, source-authority-registry.ts, patient-profile-panel.tsx, dictionary-source-dispositions.json, rag-source-block.ts and docs/clinical-governance.md. Unless the owner's typed approval names all 16 paths, this records clinical sign-off the owner never gave. It also clears the expiring exceptions that would have flagged these files for review. It is also why challenger-m5-2 now passes: every exception was dropped, not only the one this PR added. Fix: either get the owner's explicit sign-off on every resealed path and record it (decision entry), or reseal only the three approved digests and restore the other 13 driftExceptions entries.

Verification

Rules No rules evaluated

Functional validation Not enabled

Set up

Auto-approval Approved

For more verification details, comment gitar display:verbose.

Tip

Comment Gitar fix CI or enable auto-apply: gitar auto-apply:on

Options

Auto-apply is off → Gitar will not commit updates to this branch.
Display: compact → Showing compact verification results.

Comment with these commands to change the behavior for this request:

Auto-apply Compact
gitar auto-apply:on         
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Gitar

@gitar-bot
gitar-bot Bot merged commit 81ba5ae into main Oct 21, 2026
77 of 82 checks passed
@gitar-bot
gitar-bot Bot deleted the grok/3707b-ocr-scanner-tablefacts-titles branch October 21, 2026 11:37
BigSimmo added a commit that referenced this pull request Oct 21, 2026
Conflicts were only EXTRACTOR_VERSION labels (PR raster-math-layout-v13 vs main
merged-cell-provenance-v11); kept the PR's v13 label, which already postdates
main's, with no further bump. Brings in #3715.
BigSimmo added a commit that referenced this pull request Oct 21, 2026
…519926

The second merge resolved extract_pdf_assets.py with the branch side wholesale,
dropping main's merged_cells / table_merged_cells hunks (#3715). Re-run the
three-way merge for that file; only the version label conflicted.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant