Skip to content

Apply the negation of '~or' in SQLite search - #238

Merged
stevevanhooser merged 1 commit into
mainfrom
claude/did-negated-or
Oct 24, 2026
Merged

stevevanhooser merged 1 commit into
mainfrom
claude/did-negated-or

Conversation

@stevevanhooser

@stevevanhooser stevevanhooser commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Closes #237.

What changes

did.database/search_doc_ids stripped the ~ from a ~or and searched it as a plain or, so ~or(a, b) returned the documents matching a or b. It now remembers the negation and returns:

  • the documents of the branch minus the OR's matches, or
  • inside an AND, where candidate_ids is a cell array (the restricted path), the candidates minus the OR's matches.

That is what did.datastructures.fieldsearch and DID-python's SQLiteDB._search_doc_ids already do.

The un-negated OR is searched by calling search_doc_ids again with operation = 'or', so it still takes the base.id fast path (baseIdLookupValues / search_base_ids) and the batched path from #234 (search_or_operands). A ~or nested inside an OR was already handed to search_doc_ids as a single operand by search_or_operands, so it now gets the negation too. The AND ranking (rank) is unchanged. query_struct_to_sql_str is not touched: a ~or never reaches it, and #236 is changing that function.

Tests

New tests/+did/+unittest/TestNegatedOr.m compares db.search with fieldsearch applied to every document, both with and without the search indexes (QueryLogSqlitedb.noSearchIndexes). It covers:

  • a top-level ~or, including one that matches nothing and one that matches every document;
  • a ~or nested in an OR, including a ~or inside a ~or;
  • a ~or inside an AND (restricted candidate path). The test also checks that this path never lists the whole branch;
  • a ~or of base.id lookups, at top level, nested in an OR and inside ANDs.

did.query can't build a ~or, so the structures are built by hand. They use field comparisons instead of isa, because fieldsearch only understands isa after did.query/to_searchstructure has expanded it.

TestBatchedOr/testMixedOr needs no change. Its batched and per-operand searches both send the ~or operand through search_doc_ids, so they still agree, and its other checks still hold under the corrected result. The extra branch-listing query isn't counted, because its SQL doesn't start with SELECT DISTINCT docs.doc_id.

I haven't run the MATLAB code locally (no MATLAB in this environment). CI will be the first run.

Companion PR

DID-python: VH-Lab/DID-python#107. It adds the same tests and moves the bridge hash to this branch's database.m commit 2c9d8fa7. Please merge this PR first, with a merge commit, so that hash stays on main.

🤖 Generated with Claude Code

https://claude.ai/code/session_012MroTijyNKsCMqFHpXFbpc

search_doc_ids stripped the '~' of a '~or' and searched it as a plain
'or'. A negated OR now returns the documents of the branch - or the
candidates, inside an AND restricted to them - minus the OR's matches,
as did.datastructures.fieldsearch and DID-python do. The OR itself is
still searched as a plain OR, so it keeps the base.id and batched paths.

TestNegatedOr compares db.search with fieldsearch for a top-level ~or,
a ~or nested in an OR, a ~or inside an AND, and a ~or of base.id
lookups, with and without the search indexes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012MroTijyNKsCMqFHpXFbpc
@github-actions

Copy link
Copy Markdown
Contributor

Test Results

515 tests  +4   515 ✅ +4   2m 45s ⏱️ +5s
 43 suites +1     0 💤 ±0 
  1 files   ±0     0 ❌ ±0 

Results for commit 2c9d8fa. ± Comparison against base commit 6c3224f.

@codecov

codecov Bot commented Oct 24, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 93.75000% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 67.33%. Comparing base (6c3224f) to head (2c9d8fa).

Files with missing lines Patch % Lines
src/did/+did/database.m 93.75% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #238      +/-   ##
==========================================
+ Coverage   67.24%   67.33%   +0.09%     
==========================================
  Files          67       67              
  Lines        4793     4807      +14     
==========================================
+ Hits         3223     3237      +14     
  Misses       1570     1570              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@stevevanhooser
stevevanhooser merged commit 6f9be24 into main Oct 24, 2026
6 checks passed
@stevevanhooser
stevevanhooser deleted the claude/did-negated-or branch October 24, 2026 20:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SQLite search ignores the negation of '~or' and searches it as a plain 'or'

3 participants