Skip to content

Faster keyword search, a 90 second limit per search database call, and DSM "depression" search - #3560

Merged
BigSimmo merged 13 commits into
mainfrom
claude/ranking-taps-tests-hox0ma
Oct 15, 2026
Merged

BigSimmo merged 13 commits into
mainfrom
claude/ranking-taps-tests-hox0ma

Conversation

@BigSimmo

@BigSimmo BigSimmo commented Oct 15, 2026 •

Copy link
Copy Markdown
Owner

Summary

Before: searching DSM for "depression" did not find Major depressive disorder. Keyword search on the live corpus can take 30 to 48 seconds. Since PR #3554, search database calls have no time limit at all, including the query-correction call.

After: "depression", "major depression" and "clinical depression" find Major depressive disorder first, and "chronic depression" finds Persistent depressive disorder. Each search database call now stops after 90 seconds instead of never. A search that makes several calls in sequence can still take longer in total, and a whole-search deadline is left as a follow-up. The keyword-only retrieval function now keeps only the chunks that match the query before it builds candidate rows. Before, it built a row for every public chunk and threw the non-matches away afterwards.

How:

  • DSM search aliases are added in src/lib/dsm.ts. This is the local DSM list ranker, not the protected RAG path.
  • src/lib/rag/rag.ts passes timeoutMs: 90_000 to the search admin client. This answers the CodeRabbit finding on Fix source-number taps and stop search cutting off keyword results at 30 seconds #3554 about the unbounded correct_clinical_query_terms call. The correction call also carries the caller's abort signal now. When that call fails, times out or is cancelled, the empty short-circuit result is not cached.
  • New migration 20261015180000_prefilter_text_only_governed_retrieval.sql re-creates match_governed_candidate_chunks_v3 with one extra WHERE predicate. It sorts after main's 20261015160000 and Open shifts: post a gap once, even when the answer is lost and the post is retried #3557's 20261015174500. The predicate only applies when query_embedding is null, so hybrid and vector calls are unchanged. With a null embedding only text-matched rows could ever be returned, so the output set is identical and only the work to reach it shrinks. supabase/schema.sql, supabase/drift-manifest.json and the migration seal are updated to match. After merging Open shifts: post a gap once, even when the answer is lost and the post is retried #3557 in, the manifest was regenerated on the pinned local Postgres image, and only this function's def_hash differs from main.
  • A new schema test pins that the function body differs from 20260901130000 only by that predicate.
  • Two unreviewed bot commits (a77cdb0 and 6c326b2) were merged with their changes discarded.

Areas touched: Data platform and access, Clinical reference content, Answer engine
RAG impact: behaviour change — canary pair 38766713061 (post-#3554 baseline, matches pre-regression run 37226742946 case for case) -> post-merge run to follow

Verification

  • npm run test:focused -- --files src/lib/rag/rag.ts: 60 files, 1222 tests passed
  • After merging Open shifts: post a gap once, even when the answer is lost and the post is retried #3557: supabase-schema, migration-history guards, search-health coverage, drift and migration immutability tests, 8 files and 269 tests passed
  • npm run check:migration-immutability passed (312 shipped migrations unchanged) and npm run check:migration-role passed
  • Lint, Prettier and typecheck clean on changed files
  • Baseline live eval canary 38766713061 on main a5d4ee9: document_recall@5=1, content_recall@5=1, mrr@10=0.8921, failed_cases=0
  • Not run: EXPLAIN against live. This container cannot reach Supabase.

Risk and rollout

  • Risk: the migration replaces a live retrieval function, and merging applies it to the live clinical database within seconds. The owner gave an explicit yes in words to this database change on 15 Oct 2026 ("Yes put live", and "Fix the clash put 3560 live" after Open shifts: post a gap once, even when the answer is lost and the post is retried #3557 landed).
  • Rollback: a single revert commit with a new migration that restores the 20260901130000 body. The predicate is purely additive.
  • Provider or production effects: a post-merge eval canary dispatch (about US$1 to 2) checks that no case drops in rank. Any regression means an immediate single-commit revert.

Clinical Governance Preflight

  • Source-backed claims still require linked source verification before clinical use
  • No patient-identifiable document workflow was introduced or expanded without explicit governance approval
  • Supabase target remains PsychSift Production (sjrfecxgysukkwxsowpy)
  • Service-role keys and private document access remain server-only
  • Demo/synthetic content remains clearly separated from real clinical sources
  • Source metadata, review status, and outdated/unknown-source behavior remain conservative
  • Deployment classification/TGA SaMD impact was checked when clinical decision-support behavior changed

The DSM aliases only change which catalogue entry a search lands on. No diagnostic text, criteria or sign-off state changes.

Notes

  • The "assessment-documentation" answer-quality case still fails. It has failed the same way since at least 7 Oct, because the expected MHSP.AssessmentDocumentation.pdf is never retrieved. So the keyword timeouts are not the cause. It is tracked under #J8SJQ9.

🤖 Generated with Claude Code

https://claude.ai/code/session_01VkKjYakMrMXofU8foQCgbq

DSM search has no stemming, so 'depression' never matched MDD's text
('depressed', 'depressive') and the diagnosis was missing from the
results while records that only mention depression in passing filled
the list. Add a small curated alias list: depression, major depression
and clinical depression for MDD, chronic depression for persistent
depressive disorder. Local catalogue search only, no retrieval change.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VkKjYakMrMXofU8foQCgbq
…lding rows

match_governed_candidate_chunks_v3 kept the text match as a select-list
column only, so a keyword-only call (match_document_chunks_text_v3)
built labels, summary, images and metadata for every eligible public
chunk before discarding the non-matching ones. On the live corpus that
took 30-48 s per call. Add one WHERE predicate that drops non-matching
chunks first when there is no embedding. Hybrid calls are unchanged,
and keyword-only output is unchanged because those calls can only rank
text-matched rows.

Approved by the owner on 15 Oct 2026 (open as draft, live only when he
marks it ready), with a live canary pair around it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VkKjYakMrMXofU8foQCgbq
…limit

PR #3554 removed the 30 s deadline so slow keyword searches could finish.
A finite 90 s limit keeps that while bounding a stalled call, including
the query-correction RPC that does not carry the caller's abort signal.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VkKjYakMrMXofU8foQCgbq
@supabase

supabase Bot commented Oct 15, 2026

Copy link
Copy Markdown

This pull request has been ignored for the connected project sjrfecxgysukkwxsowpy due to reaching the limit of concurrent preview branches.
Go to Project Integrations Settings ↗︎ if you wish to update this limit.


Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

@coderabbitai

coderabbitai Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Warning

Review limit reached

Your organization has reached its usage spending cap. Adjust your spending cap in the billing tab.

Next included review available in 54 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available. Your 91 included PR review attempts over the past 7 days set your current allowance at 1 review per hour.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Repository: BigSimmo/PsychSift/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Team
  • Run ID: fa24183b-e3fd-43f0-a947-be377155f49b
📥 Commits

Reviewing files that changed from the base of the PR and between e1bb8c3 and 08f258d.

📒 Files selected for processing: 9
  • src/lib/dsm.ts
  • src/lib/rag/rag.ts
  • supabase/applied-migration-hashes.json
  • supabase/drift-manifest.json
  • supabase/migrations/20261015180000_prefilter_text_only_governed_retrieval.sql
  • supabase/schema.sql
  • tests/dsm.test.ts
  • tests/rag-unsupported-short-circuit-cache.test.ts
  • tests/supabase-schema.test.ts

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository: BigSimmo/PsychSift/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Team
  • Run ID: 8a859d05-bd74-46a0-9801-a8e84b24d2fc
📥 Commits

Reviewing files that changed from the base of the PR and between c5fc332 and e1bb8c3.

📒 Files selected for processing: 9
  • src/lib/dsm.ts
  • src/lib/rag/rag.ts
  • supabase/applied-migration-hashes.json
  • supabase/drift-manifest.json
  • supabase/migrations/20261015180000_prefilter_text_only_governed_retrieval.sql
  • supabase/schema.sql
  • tests/dsm.test.ts
  • tests/rag-unsupported-short-circuit-cache.test.ts
  • tests/supabase-schema.test.ts

Included review availability: This review used your included allowance. 0 included reviews remain after this review. Your included PR review attempts over the past 7 days set your current allowance at 1 review per hour.

📝 Walkthrough

The pull request adds clinical diagnosis aliases, changes RAG search timeout and correction handling, and adds a governed retrieval SQL function with text-only prefiltering, clinical-site candidates, and hybrid ranking.

Changes: Diagnosis search aliases

Layer / File(s) Summary
Clinical aliases and ranking expectations
src/lib/dsm.ts, tests/dsm.test.ts
The alias map adds normalized terms for Major depressive disorder and Persistent depressive disorder. Tests check which diagnosis ranks first for three search terms.

Search request handling

Layer / File(s) Summary
Search deadline and correction handling
src/lib/rag/rag.ts, tests/rag-unsupported-short-circuit-cache.test.ts
The search admin client now uses a 90-second timeout. The correction RPC receives the caller’s abort signal. Tests check that a correction error does not cache the empty short-circuit result and that a successful correction does.

Governed retrieval

Layer / File(s) Summary
Function contract and document prefilter
supabase/migrations/20261015180000_prefilter_text_only_governed_retrieval.sql, supabase/schema.sql, tests/supabase-schema.test.ts
The function defines its result shape and filters document candidates. When no query embedding is provided, candidates require a text match against chunk or title text. Schema and tests check the prefilter and effective function definition.
Clinical-site candidates and hybrid ranking
supabase/migrations/20261015180000_prefilter_text_only_governed_retrieval.sql, supabase/applied-migration-hashes.json, supabase/drift-manifest.json
The function filters site candidates against release and record checks, combines document and site candidates, and returns bounded hybrid-ranked results. Migration and drift manifests record the update.

Priority: ➖ Normal

Estimated code review effort: 4 (Complex) | ~45 minutes

Change: Bug fix

Suggested reviewers: claude

Merge Risk: ⚪ Minimal · up to e1bb8

The change adds diagnosis search aliases, a 90-second search timeout, and a keyword-only retrieval prefilter that should not change results. No concrete defect remains. Apply the database migration in the planned approved window and confirm that the search plan performs as expected.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 20.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 5 functions across 5 files. (4 skipped: 4… Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Title check Passed The title clearly summarizes the three primary changes: faster keyword search, a 90-second search limit, and improved DSM depression search.
Description check Passed The description includes the required Summary, Verification, Risk and rollout, and Notes sections. It documents the changes, verification results, known limitations, migration risk, rollback plan, pro…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 20.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 5 functions across 5 files. (4 skipped: 4 unsupported.)

✨ Finishing Touches: 💡 1
📝 Generate docstrings: 💡
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

gitar-bot[bot]
gitar-bot Bot previously approved these changes Oct 15, 2026

@gitar-bot gitar-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gitar has auto-approved this PR (configure)

@codeant-ai

codeant-ai Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

🏁 CodeAnt Quality Gate Results

Commit: 08f258d2
Scan Time: 2026-10-15 23:53:34 UTC

✅ Overall Status: PASSED

Quality Gate Details

Quality Gate Status Details
Secrets ✅ PASSED 0 secrets found
Duplicate Code ✅ PASSED 0.0% duplicated
SAST ✅ PASSED No security issues
Bugs ✅ PASSED Rating S: No bugs
IAC ✅ PASSED No IAC issues

View Full Results

@BigSimmo
BigSimmo marked this pull request as ready for review October 15, 2026 17:33

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BigSimmo has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.

@codeant-ai

codeant-ai Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

🤖 CodeAnt AI — Review Status

Status Commit Started (UTC) Finished (UTC)
✅ Incremental review completed 08f258d Oct 15, 2026 · 23:36 23:38
✅ Incremental review completed e1bb8c3 Oct 15, 2026 · 18:49 18:52
✅ Reviewed your PR 7383d79 Oct 15, 2026 · 17:33 17:40

@codeant-ai

codeant-ai Bot commented Oct 15, 2026

Copy link
Copy Markdown

Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-15T18:22:51.397329Z 7f86952 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@codeant-ai codeant-ai Bot added the size:L This PR changes 100-499 lines, ignoring generated files label Oct 15, 2026
@qodo-code-review

qodo-code-review Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

PR Summary by Qodo

Speed up keyword retrieval and improve depression searches

🐞 Bug fix ✨ Enhancement 🧪 Tests 🕐 40+ Minutes

Grey Divider

AI Description

• Prefilter text-only document candidates to avoid building rows that cannot appear in results.
• Bound RAG database calls at 90 seconds and propagate cancellation to query correction.
• Add DSM depression aliases and tests for ranking and migration scope.
Diagram

graph TD
  Q["Search query"] --> D["DSM alias ranker"] --> DR["DSM results"]
  Q --> R["RAG admin client"] --> G["Governed candidate RPC"] --> F{"Embedding null?"}
  F -->|"yes: text match"| T["Document prefilter"] --> RR["Ranked chunks"]
  F -->|"no: unchanged"| RR
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. Separate text-only candidate query
  • ➕ Could specialize the keyword query and its execution plan further.
  • ➖ Duplicates governed eligibility and ranking logic, increasing the risk of behavioral drift.

Recommendation: Keep the conditional predicate in the existing governed function: it avoids constructing non-matching document rows without splitting the governed retrieval contract. Consider a separate keyword query only if production measurements show the prefilter is insufficient.

Files changed (8) +511 / -14

Enhancement (2) +439 / -0
20261015180000_prefilter_text_only_governed_retrieval.sqlPrefilter keyword-only governed document candidates +429/-0

Prefilter keyword-only governed document candidates

• Re-creates the governed candidate function with a predicate that excludes non-text-matching public document chunks when no embedding is supplied. Hybrid calls retain their existing candidate path; the migration preserves function access restrictions and ownership.

supabase/migrations/20261015180000_prefilter_text_only_governed_retrieval.sql

schema.sqlMirror the governed retrieval prefilter +10/-0

Mirror the governed retrieval prefilter

• Adds the same text-only document predicate to the checked-in effective schema.

supabase/schema.sql

Bug fix (2) +26 / -6
dsm.tsAdd curated depression search aliases +9/-0

Add curated depression search aliases

• Adds aliases for Major depressive disorder and Persistent depressive disorder to the local DSM ranker. Plain-language depression searches can now reach the intended diagnoses without changing RAG retrieval.

src/lib/dsm.ts

rag.tsBound search database calls and cancel query correction +17/-6

Bound search database calls and cancel query correction

• Sets a 90-second timeout on the search admin client instead of leaving its calls unbounded. Passes the caller's abort signal to query correction and stops an aborted search before its empty result can be cached.

src/lib/rag/rag.ts

Tests (2) +39 / -2
dsm.test.tsCover depression diagnosis ranking +8/-0

Cover depression diagnosis ranking

• Asserts that plain and major depression rank Major depressive disorder first, while chronic depression ranks Persistent depressive disorder first.

tests/dsm.test.ts

supabase-schema.test.tsPin the effective function and predicate-only change +31/-2

Pin the effective function and predicate-only change

• Checks that the final schema mirrors the new migration. Compares its function body with the earlier migration to ensure the only substantive SQL change is the text-only prefilter.

tests/supabase-schema.test.ts

Other (2) +7 / -6
applied-migration-hashes.jsonSeal the new retrieval migration +3/-2

Seal the new retrieval migration

• Records the new migration's hash in the applied-migration seal.

supabase/applied-migration-hashes.json

drift-manifest.jsonRefresh the replayed schema manifest +4/-4

Refresh the replayed schema manifest

• Updates the schema hash and governed candidate function definition hash after replaying the migration.

supabase/drift-manifest.json

@qodo-code-review

qodo-code-review Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

Grey Divider


Action required

No findings for this group


Remediation recommended

No findings for this group


Informational

No findings for this group

Grey Divider


Resolved findings

1. Timeouts cache false search misses ✓ Resolved
Description
The correction RPC discards its error, and throwIfAborted(args.signal) checks only the caller’s
signal, not the new admin-client timeout. If correction reaches the 90-second deadline, its null
response falls through to an empty unsupported result that can be written to the search cache.
Code

src/lib/rag/rag.ts[R1351-1354]

+      const { data: corrected } = await correctionQuery;
+      // A cancelled call resolves with an error rather than throwing; stop before the empty
+      // short-circuit result below is written to the search cache.
+      throwIfAborted(args.signal);
Relevance

●●● Strong

Timeout RPC errors can silently cache false misses; conservative RAG fallback precedent favors
preserving errors.

PR-#3554
PR-#2967

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The rag skill requires conservative fallback behavior. The new bounded client can return a timeout
as an RPC error without aborting the caller’s signal, while the changed correction path checks only
that signal before reaching the empty-result cache write.

src/lib/rag/rag.ts[1344-1354]
src/lib/rag/rag.ts[1366-1380]
src/lib/supabase/admin.ts[33-40]
tests/supabase-admin-timeout.test.ts[45-74]
Skill: rag

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
A correction RPC timeout returns an error with null data, but the search treats it as a successful uncorrected query and may cache an empty result.

## Fix Focus Areas
- src/lib/rag/rag.ts[1346-1354]
- src/lib/rag/rag.ts[1366-1380]

## Recommended Fix
Inspect the correction RPC error after awaiting it. On timeout or other RPC failure, prevent the unsupported empty-result cache write and propagate or explicitly handle the failure; add a focused test for a timed-out correction.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


2. Keyword searches still scan chunks ✓ Resolved
Description
The new document_candidates predicate ORs chunk-text and document-title matches across two tables,
repeating a query shape the repository’s earlier lexical-retrieval audit found could not use either
existing GIN index for candidate selection. Broad keyword-only searches can therefore still scan the
chunk corpus before discarding nonmatches, even though the filter avoids building metadata for those
rows.
Code

supabase/migrations/20261015140000_prefilter_text_only_governed_retrieval.sql[225]

+          and (chunk.search_tsv @@ admitted.tsq or document.title_search_tsv @@ admitted.tsq)
Relevance

●●● Strong

Documented index limitation directly undermines the PR’s performance goal; repository accepts
concrete search-latency fixes.

PR-#2805
PR-#3061

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The keyword-only predicate applies a cross-table OR to joined candidate rows. An earlier repository
audit documents that the same disjunction caused a full document_chunks scan at roughly 70,000
chunks, and demonstrates separate index probes followed by a union; the schema contains both
text-search indexes. The added filter may reduce work after candidate selection, but does not remove
the documented scan risk.

supabase/migrations/20261015140000_prefilter_text_only_governed_retrieval.sql[217-227]
supabase/migrations/20260713100000_index_friendly_lexical_retrieval.sql[1-14]
supabase/migrations/20260615000000_retrieval_synopsis_adaptive_rag.sql[4-9]
supabase/migrations/20260528004000_split_document_title_search_tokens.sql[1-10]
supabase/migrations/20261015140000_prefilter_text_only_governed_retrieval.sql[218-227]
supabase/migrations/20260713100000_index_friendly_lexical_retrieval.sql[1-15]
supabase/migrations/20260713100000_index_friendly_lexical_retrieval.sql[47-78]
Skill: performance-review
Skill: supabase-postgres-best-practices

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The new keyword-only filter retains a cross-table OR that the repository’s prior lexical-retrieval audit identified as preventing index-driven candidate selection. Preserve retrieval results while narrowing candidate IDs through separate index-backed searches.

## Fix Focus Areas
- supabase/migrations/20261015140000_prefilter_text_only_governed_retrieval.sql[214-227]
- supabase/schema.sql[17755-17767]

## Recommended Fix
For keyword-only calls, obtain matching chunk IDs through separate indexed chunk-text and document-title probes, union the IDs, then build candidate rows and apply the existing eligibility and ranking logic. Preserve the unfiltered candidate path for calls with an embedding, along with its eligibility gates, ranking, and output. Update the schema snapshot, and verify result parity and execution plans for both paths against an approved representative database.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Context sources
✅ Compliance rules (platform): 9 rules
✅ Skills: 4 invoked
  drift
  migrate
  supabase
  review
✅ Cross-repo context — repo relationships
Review mode: Auto: 🧠 Deep: Dense push spans dashboard refactor, roster idempotency, and multiple UI behavior paths.

Grey Divider

Tip of the day
💡 Did you know, you can hide the parts of a finding you never read, like the evidence or the agent prompt

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Previous reviews

Review updated until commit 08f258d

Results up to commit 7383d79 ⚖️ Balanced


🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)


Action required

No findings for this group


Remediation recommended

No findings for this group


Informational

No findings for this group


Resolved findings

1. Keyword searches still scan chunks ✓ Resolved
Description
The new document_candidates predicate ORs chunk-text and document-title matches across two tables,
repeating a query shape the repository’s earlier lexical-retrieval audit found could not use either
existing GIN index for candidate selection. Broad keyword-only searches can therefore still scan the
chunk corpus before discarding nonmatches, even though the filter avoids building metadata for those
rows.
Code

supabase/migrations/20261015140000_prefilter_text_only_governed_retrieval.sql[225]

+          and (chunk.search_tsv @@ admitted.tsq or document.title_search_tsv @@ admitted.tsq)
Relevance

●●● Strong

Documented index limitation directly undermines the PR’s performance goal; repository accepts
concrete search-latency fixes.

PR-#2805
PR-#3061

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The keyword-only predicate applies a cross-table OR to joined candidate rows. An earlier repository
audit documents that the same disjunction caused a full document_chunks scan at roughly 70,000
chunks, and demonstrates separate index probes followed by a union; the schema contains both
text-search indexes. The added filter may reduce work after candidate selection, but does not remove
the documented scan risk.

supabase/migrations/20261015140000_prefilter_text_only_governed_retrieval.sql[217-227]
supabase/migrations/20260713100000_index_friendly_lexical_retrieval.sql[1-14]
supabase/migrations/20260615000000_retrieval_synopsis_adaptive_rag.sql[4-9]
supabase/migrations/20260528004000_split_document_title_search_tokens.sql[1-10]
supabase/migrations/20261015140000_prefilter_text_only_governed_retrieval.sql[218-227]
supabase/migrations/20260713100000_index_friendly_lexical_retrieval.sql[1-15]
supabase/migrations/20260713100000_index_friendly_lexical_retrieval.sql[47-78]
Skill: performance-review
Skill: supabase-postgres-best-practices

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The new keyword-only filter retains a cross-table OR that the repository’s prior lexical-retrieval audit identified as preventing index-driven candidate selection. Preserve retrieval results while narrowing candidate IDs through separate index-backed searches.

## Fix Focus Areas
- supabase/migrations/20261015140000_prefilter_text_only_governed_retrieval.sql[214-227]
- supabase/schema.sql[17755-17767]

## Recommended Fix
For keyword-only calls, obtain matching chunk IDs through separate indexed chunk-text and document-title probes, union the IDs, then build candidate rows and apply the existing eligibility and ranking logic. Preserve the unfiltered candidate path for calls with an embedding, along with its eligibility gates, ranking, and output. Update the schema snapshot, and verify result parity and execution plans for both paths against an approved representative database.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Results up to commit 7f86952 ⚖️ Balanced


🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)


Action required

No findings for this group


Remediation recommended

No findings for this group


Informational

No findings for this group


Resolved findings

1. Timeouts cache false search misses ✓ Resolved
Description
The correction RPC discards its error, and throwIfAborted(args.signal) checks only the caller’s
signal, not the new admin-client timeout. If correction reaches the 90-second deadline, its null
response falls through to an empty unsupported result that can be written to the search cache.
Code

src/lib/rag/rag.ts[R1351-1354]

+      const { data: corrected } = await correctionQuery;
+      // A cancelled call resolves with an error rather than throwing; stop before the empty
+      // short-circuit result below is written to the search cache.
+      throwIfAborted(args.signal);
Relevance

●●● Strong

Timeout RPC errors can silently cache false misses; conservative RAG fallback precedent favors
preserving errors.

PR-#3554
PR-#2967

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The rag skill requires conservative fallback behavior. The new bounded client can return a timeout
as an RPC error without aborting the caller’s signal, while the changed correction path checks only
that signal before reaching the empty-result cache write.

src/lib/rag/rag.ts[1344-1354]
src/lib/rag/rag.ts[1366-1380]
src/lib/supabase/admin.ts[33-40]
tests/supabase-admin-timeout.test.ts[45-74]
Skill: rag

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
A correction RPC timeout returns an error with null data, but the search treats it as a successful uncorrected query and may cache an empty result.

## Fix Focus Areas
- src/lib/rag/rag.ts[1346-1354]
- src/lib/rag/rag.ts[1366-1380]

## Recommended Fix
Inspect the correction RPC error after awaiting it. On timeout or other RPC failure, prevent the unsupported empty-result cache write and propagate or explicitly handle the failure; add a focused test for a timed-out correction.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Qodo Logo

@cubic-dev-ai

cubic-dev-ai Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Cancelled. View session →

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7383d79a63

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/lib/rag/rag.ts Resolved
Main gained 20261015160000, so the prefilter migration is renamed to
20261015180000 to stay in order, sealed, and the drift manifest is
regenerated from the merged schema. The schema test now mirrors the
effective (prefiltered) function and pins that it differs from
20260901130000 only by the one prefilter predicate.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VkKjYakMrMXofU8foQCgbq
@qodo-code-review

Copy link
Copy Markdown
Contributor

Qodo Fixer

✅ Committed (1) · ☑ Fixed (1)

Grey Divider

Commits pushed directly to this PR — no separate fix PR opened.

Process — 1 fixed
  • ☑ Fixed: Keyword searches still scan chunks

@gitar-bot
gitar-bot Bot enabled auto-merge (squash) October 15, 2026 17:38
gitar-bot[bot]

This comment was marked as resolved.

Qodo pushed a77cdb0 (split text-index probes via a UNION CTE) to this
branch unreviewed. It edits the old 20261015140000 file, which this branch
has since renamed, and leaves the drift manifest stale. Keep the reviewed
single-predicate prefilter; the per-row label/summary/image/jsonb work it
avoids is the measured cost. The index-probe idea can follow separately
with its own EXPLAIN evidence.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VkKjYakMrMXofU8foQCgbq
Comment thread supabase/schema.sql Resolved
Comment thread supabase/schema.sql Resolved
@BigSimmo
BigSimmo marked this pull request as draft October 15, 2026 17:39
@qodo-code-review

Copy link
Copy Markdown
Contributor

Code review by qodo was updated up to the latest commit 7f86952

@qodo-code-review

Copy link
Copy Markdown
Contributor

Qodo Fixer

✅ Committed (1) · ☑ Fixed (1)

Grey Divider

Commits pushed directly to this PR — no separate fix PR opened.

Process — 1 fixed
  • ☑ Fixed: Timeouts cache false search misses

@gitar-bot
gitar-bot Bot enabled auto-merge (squash) October 15, 2026 18:19
gitar-bot[bot]
gitar-bot Bot previously approved these changes Oct 15, 2026

@gitar-bot gitar-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gitar has auto-approved this PR (configure)

A timed-out or failed correction resolves with an error and null data,
which is not evidence the query has no match (Qodo finding on #3560).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VkKjYakMrMXofU8foQCgbq
It threw on any correction error, failing the whole search where the
previous behaviour ignored the error. The preceding commit instead keeps
the search and skips only the cache write, with a test.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VkKjYakMrMXofU8foQCgbq
gitar-bot[bot]
gitar-bot Bot previously approved these changes Oct 15, 2026

@gitar-bot gitar-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gitar has auto-approved this PR (configure)

@qodo-code-review

Copy link
Copy Markdown
Contributor

Code review by qodo was updated up to the latest commit e649e0f

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VkKjYakMrMXofU8foQCgbq
@codeant-ai codeant-ai Bot added size:XL This PR changes 500-999 lines, ignoring generated files and removed size:L This PR changes 100-499 lines, ignoring generated files labels Oct 15, 2026

Copy link
Copy Markdown
Owner Author

PR mergeability is failing because #3557 (Open shifts) merged first and changed the same files: supabase/schema.sql, supabase/drift-manifest.json and supabase/applied-migration-hashes.json. That is a real merge conflict.

The migration order is still correct. This PR's 20261015180000 sorts after #3557's 20261015174500.

I have not resolved the conflict yet. Resolving it means merging main and regenerating the drift manifest, and pushing that leads straight to a live database change, so it is waiting on the owner's explicit go-ahead. Until then, nothing in this PR has reached the live database.


Generated by Claude Code

Resolves the clash with #3557 (Open shifts). The migration seal keeps both
entries in version order, and the drift manifest is regenerated on the
pinned Postgres image. Only match_governed_candidate_chunks_v3's def_hash
differs from main.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VkKjYakMrMXofU8foQCgbq

@gitar-bot gitar-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gitar has auto-approved this PR (configure)

@gitar-bot

gitar-bot Bot commented Oct 15, 2026

Copy link
Copy Markdown
Contributor
Code Review ✅ Approved 3 closed / 3 findings

🔴 High risk · The migration replaces the production governed clinical retrieval function; a faulty prefilter could omit relevant clinical sources from keyword search.

Keyword search now prefilters text-matched chunks before building candidate rows, adds DSM depression aliases, and bounds search database calls at 90 seconds. Query-correction RPC now honors abort signals and no longer caches empty results after failures, and the drift manifest and schema definition issues are resolved. No open issues remain.

✅ 3 closed
✅ Bug: drift-manifest.json not regenerated after schema.sql function change

📄 supabase/schema.sql:17684-17698 📄 supabase/schema.sql:17792-17797 📄 supabase/migrations/20261015140000_prefilter_text_only_governed_retrieval.sql:126-140 📄 supabase/migrations/20261015140000_prefilter_text_only_governed_retrieval.sql:234-239
This commit rewrites the body of match_governed_candidate_chunks_v3 in both supabase/schema.sql and the migration, but it does not touch supabase/drift-manifest.json. The manifest still has schema_sha256: dad0927e…, which is the hash of the parent commit's schema.sql. The current file hashes to 6bd18da0…. The function's def_hash (090e9afd…) also still matches the old single-predicate body. check-drift-manifest-freshness, the drift-detection tests and the post-merge check:drift gate compare against these values, so they will report drift or fail. Fix: regenerate the manifest against the pinned local Postgres image, the same way commit b249be6 did, and commit the new schema_sha256 and def_hash together.

✅ Quality: Historical v3 definition in schema.sql edited with an unused join

📄 supabase/schema.sql:15950-15964 📄 supabase/schema.sql:16058
The bot commit also added the text_match_ids CTE and a left join text_match_ids matched to the original create function public.match_governed_candidate_chunks_v3 at schema.sql:15832. That is the earlier migration's body, not the 20261015140000 replacement. The added predicate (query_embedding is not null or matched.id is not null) was not added there, so the join filters nothing. The join still forces the text probes to run, and this section no longer mirrors the migration it came from. The later create or replace overrides the function at runtime, so the only effect is that schema.sql diverges from migration history and is harder to audit. Fix: revert the two hunks around lines 15950 and 16058, and keep the change only in the 17566+ create or replace block.

✅ Bug: Aborted correction RPC falls through and caches an empty result

📄 src/lib/rag/rag.ts:1346-1351 📄 src/lib/rag/rag.ts:1365-1376
Passing args.signal to correct_clinical_query_terms means a cancelled search now resolves this call with { data: null, error: AbortError }. postgrest-js returns the abort as an error and does not throw it. The code ignores error and does not check the signal afterwards. So corrected is not a string, and execution continues into the unsupported short-circuit, which calls dispatchSearchCacheWrite(args, [], …) and returns [].

Before this change, the correction call always ran to completion. The new behaviour is that a user cancelling a typo query stores an empty cached result. Later identical queries can then be served that cached empty result instead of the corrected search, until the cache entry expires. Every other await in this function is followed by throwIfAborted(args.signal). This path needs the same check.

Verification

Rules No rules evaluated

Functional validation Not enabled

Set up

Auto-approval Approved

For more verification details, comment gitar display:verbose.

Options

Auto-apply is off → Gitar will not commit updates to this branch.
Display: compact → Showing compact verification results.

Comment with these commands to change the behavior for this request:

Auto-apply Compact
gitar auto-apply:on         
gitar display:verbose         

Important

Your trial ends in 6 days — upgrade now to keep code review, CI analysis, auto-apply, custom automations, and more.

Was this helpful? React with 👍 / 👎 | Gitar

@qodo-code-review

Copy link
Copy Markdown
Contributor

Code review by qodo was updated up to the latest commit 08f258d

@BigSimmo
BigSimmo merged commit 36d1639 into main Oct 15, 2026
42 checks passed
@BigSimmo BigSimmo changed the title Faster keyword search, a 90 second search limit, and DSM "depression" search Faster keyword search, a 90 second limit per search database call, and DSM "depression" search Oct 15, 2026
@BigSimmo
BigSimmo deleted the claude/ranking-taps-tests-hox0ma branch October 15, 2026 23:53
BigSimmo pushed a commit that referenced this pull request Oct 16, 2026
Brings in #3560's 20261015180000 migration. The seal keeps every entry
unchanged and lists this PR's 20261015181500 after it. The drift manifest
takes main's header with schema_sha256 recomputed over the merged
schema.sql.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LuYbdEGXeMntVA9KMgtAEx
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

🔴 Large blast radius size:XL This PR changes 500-999 lines, ignoring generated files 🕐 40+ Minutes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants