Repository navigation
Apply the negation of '~or' in SQLite search - #238
Merged
Merged
Conversation
search_doc_ids stripped the '~' of a '~or' and searched it as a plain 'or'. A negated OR now returns the documents of the branch - or the candidates, inside an AND restricted to them - minus the OR's matches, as did.datastructures.fieldsearch and DID-python do. The OR itself is still searched as a plain OR, so it keeps the base.id and batched paths. TestNegatedOr compares db.search with fieldsearch for a top-level ~or, a ~or nested in an OR, a ~or inside an AND, and a ~or of base.id lookups, with and without the search indexes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012MroTijyNKsCMqFHpXFbpc
Contributor
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #238 +/- ##
==========================================
+ Coverage 67.24% 67.33% +0.09%
==========================================
Files 67 67
Lines 4793 4807 +14
==========================================
+ Hits 3223 3237 +14
Misses 1570 1570 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
stevewds
approved these changes
Oct 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #237.
What changes
did.database/search_doc_idsstripped the~from a~orand searched it as a plainor, so~or(a, b)returned the documents matchingaorb. It now remembers the negation and returns:candidate_idsis a cell array (the restricted path), the candidates minus the OR's matches.That is what
did.datastructures.fieldsearchand DID-python'sSQLiteDB._search_doc_idsalready do.The un-negated OR is searched by calling
search_doc_idsagain withoperation = 'or', so it still takes thebase.idfast path (baseIdLookupValues/search_base_ids) and the batched path from #234 (search_or_operands). A~ornested inside an OR was already handed tosearch_doc_idsas a single operand bysearch_or_operands, so it now gets the negation too. The AND ranking (rank) is unchanged.query_struct_to_sql_stris not touched: a~ornever reaches it, and #236 is changing that function.Tests
New
tests/+did/+unittest/TestNegatedOr.mcomparesdb.searchwithfieldsearchapplied to every document, both with and without the search indexes (QueryLogSqlitedb.noSearchIndexes). It covers:~or, including one that matches nothing and one that matches every document;~ornested in an OR, including a~orinside a~or;~orinside an AND (restricted candidate path). The test also checks that this path never lists the whole branch;~orofbase.idlookups, at top level, nested in an OR and inside ANDs.did.querycan't build a~or, so the structures are built by hand. They use field comparisons instead ofisa, becausefieldsearchonly understandsisaafterdid.query/to_searchstructurehas expanded it.TestBatchedOr/testMixedOrneeds no change. Its batched and per-operand searches both send the~oroperand throughsearch_doc_ids, so they still agree, and its other checks still hold under the corrected result. The extra branch-listing query isn't counted, because its SQL doesn't start withSELECT DISTINCT docs.doc_id.I haven't run the MATLAB code locally (no MATLAB in this environment). CI will be the first run.
Companion PR
DID-python: VH-Lab/DID-python#107. It adds the same tests and moves the bridge hash to this branch's
database.mcommit2c9d8fa7. Please merge this PR first, with a merge commit, so that hash stays on main.🤖 Generated with Claude Code
https://claude.ai/code/session_012MroTijyNKsCMqFHpXFbpc