Skip to content

ENG-2108 Add ranked search to QueryEngine with a vault-iteration fallback - #1284

Open
trangdoan982 wants to merge 1 commit into
mainfrom
eng-2108-add-ranked-search-to-queryengine-with-a-vault-iteration
Open

ENG-2108 Add ranked search to QueryEngine with a vault-iteration fallback#1284
trangdoan982 wants to merge 1 commit into
mainfrom
eng-2108-add-ranked-search-to-queryengine-with-a-vault-iteration

Conversation

@trangdoan982

@trangdoan982 trangdoan982 commented Aug 7, 2026

Copy link
Copy Markdown
Member

Scope check

  • Ran $scope-check against ENG-2108 and the final diff.
  • Scope beyond Done When: None.
Mapping

One file, one concern. Three deviations from the ticket's literal wording, all narrowing rather than expanding: createScorer was dropped as a meaningless one-line wrapper over prepareFuzzySearch; the Scorer type and the scorer? injection parameter were dropped as unused indirection, so prepareFuzzySearch is called directly; and filterCandidatesByNodeTypeIds is exported instead, because the ticket requires callers to memoize the filtered array — which is only possible if the filter is callable on its own. No new dependencies, no migrations, no unrelated refactors.

What this does

Adds the search seam for the Obsidian advanced node search panel (F1 of Advanced Node Search (Obsidian)). No UI — the panel is #1285, stacked on this.

QueryEngine.ts:

Export Runs Purpose
DiscourseNodeCandidate, RankedDiscourseNode types
getDiscourseNodeCandidates() once per open every DG node as { file, title, nodeTypeId }
filterCandidatesByNodeTypeIds(...) on filter change node-type narrowing, OR across types
rankDiscourseNodesByTitle({...}) per keystroke filter, score, sort

Three decisions worth reviewing

Datacore first, vault iteration as fallback — via delegation, so it is not visible in this diff. The call chain:

getDiscourseNodeCandidates()          ← new here
  └─ getFilesWithNodeTypeId()         ← existing, unchanged, so not in the diff
       ├─ Datacore installed  → `@page and exists(nodeTypeId)`
       └─ absent or throws    → fallbackGetFilesWithNodeTypeId()  (vault iteration)

Reusing that method rather than writing a second fallback here is why NF3 holds for free.

Verification

check-types clean, eslint 0 problems on the changed file.

The reviewable demo is #1285, stacked on this branch: https://www.loom.com/share/877db7d2de834548a81bca1750bbbecb

🤖 Generated with Claude Code

@linear-code

linear-code Bot commented Aug 7, 2026

Copy link
Copy Markdown

ENG-2108

@supabase

supabase Bot commented Aug 7, 2026

Copy link
Copy Markdown

This pull request has been ignored for the connected project zytfjzqyijgagqxrzbmz because there are no changes detected in packages/database/supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.


Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

@vercel

vercel Bot commented Aug 7, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
discourse-graph Skipped Skipped Aug 9, 2026 5:28pm

Request Review

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 potential issue.

View 1 additional finding in Devin Review.

Open in Devin Review

Comment on lines +6 to +9
**These are not automated.** `apps/obsidian` has no test runner yet — no vitest, no
`test` script, no existing test files — and standing one up was deliberately kept out
of ENG-2108. This file is the spec to port when that lands, and the manual checklist
until then.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 New search ranking code ships without automated tests

The new ranking, filtering and candidate-collection functions are added (rankDiscourseNodesByTitle at apps/obsidian/src/services/QueryEngine.ts:708) with only a hand-written markdown checklist instead of executable tests, so future changes can silently break search ordering.
Impact: Regressions in node search results would not be caught before release.

Repository rule requiring unit tests for new functionality

AGENTS.md (Testing) states "Write unit tests for new functionality" and CONTRIBUTING.md step 4 requires "Write tests that validate your change and/or fix." The PR instead adds apps/obsidian/src/services/__tests__/QueryEngine.search.test-cases.md, a non-executable spec, and explicitly states no test runner exists in apps/obsidian. The functions are pure and injectable via scorer (apps/obsidian/src/services/QueryEngine.ts:712-717), so they are directly unit-testable once a runner is configured.

Prompt for agents
AGENTS.md and CONTRIBUTING.md require unit tests for new functionality, but this PR adds only a markdown checklist. The pure functions createScorer, filterCandidatesByNodeTypeIds and rankDiscourseNodesByTitle in apps/obsidian/src/services/QueryEngine.ts are testable in Node when the obsidian module is aliased to a stub (prepareFuzzySearch is only needed when no scorer is injected). Consider adding a vitest setup to apps/obsidian mirroring apps/roam/vitest.config.mts with an alias for the obsidian package, and porting TC1-TC16 from the markdown spec into real test files.
Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

@trangdoan982
trangdoan982 force-pushed the eng-2108-add-ranked-search-to-queryengine-with-a-vault-iteration branch from 22188a8 to 44c784b Compare August 7, 2026 20:59
@trangdoan982
trangdoan982 force-pushed the eng-2108-add-ranked-search-to-queryengine-with-a-vault-iteration branch from 44c784b to 07bb663 Compare August 8, 2026 17:32
@trangdoan982
trangdoan982 force-pushed the eng-2108-add-ranked-search-to-queryengine-with-a-vault-iteration branch from 07bb663 to f89d51d Compare August 9, 2026 03:55
@trangdoan982
trangdoan982 force-pushed the eng-2108-add-ranked-search-to-queryengine-with-a-vault-iteration branch from f89d51d to 79938ba Compare August 9, 2026 17:22
…back

Add a Scorer seam over Obsidian's prepareFuzzySearch, plus the candidate fetch
and ranking functions the advanced node search panel needs.

Separate the candidate fetch from scoring so the vault scan runs once per
search-surface open rather than once per keystroke, and route it through
getFilesWithNodeTypeId, which already falls back to vault iteration when
Datacore is unavailable.

Score and render the same string (file.basename) so SearchResult.matches offsets
stay aligned with what renderResults re-slices. Filter by node type before
scoring, which leaves results identical but shrinks the number of scorer calls
on the per-keystroke path.

Rank on SearchResult.score alone; Array.prototype.sort is stable, so equal
scores keep candidate order without an explicit tie-break. An empty query
returns the full filtered set in title order rather than nothing, so a type
filter alone still narrows to a visible list.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@trangdoan982
trangdoan982 force-pushed the eng-2108-add-ranked-search-to-queryengine-with-a-vault-iteration branch from 79938ba to 9708e95 Compare August 9, 2026 17:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant