[stevevanhooser]

Closes #236.

This MATLAB code has not been run locally. There is no MATLAB in the environment it was written in. The tests run in this PR's CI.

Problem

1. SQL injection / wrong results. query_struct_to_sql_str put a text hasmember value into the SQL unquoted:
   • 'name17' → no such column
   • 'value' → the column doc_data.value, so every document matched
   • '1 OR 1=1' → injection, every document matched
   • a value containing " or ' → syntax error
2. SQL could not see list members. doc2sql stores an array or cell with more than one element only as a summary, e.g. '[1 3] double'. So in SQL, hasmember 2 missed [1 2 3], and ~hasmember 2 matched it.
3. Three different rules:
   • fieldsearch used ismember, which compares two texts character by character. It returns a logical array, and ~/AND then treat that array element by element.
   • the SQL search matched a comma-list item.
   • DID-python's field_search tested for a substring.

Fix

One rule (src/did/+did/+datastructures/fieldsearch.m)

hasmember now calls a new local function, hasMember:

• text in text: the member is the whole value or an item of its comma-separated list ((^|,\s*)M\s*(,|$), with M escaped)
• anything in a cell array: an element equals the member
• a number in a numeric or logical array: an element equals it
• text is never a member of a number, nor a number of text. A struct has no members.

DID-python's field_search gets the same rule (VH-Lab/DID-python#108).

Brute force, as DID-python (src/did/+did/database.m)

• A new protected static method, isBruteForceTerm, is true for every hasmember/~hasmember, and for the depends_on queries isBruteForceDependsOn already sent to brute force.
• search_doc_ids sends those terms to brute_force_search, which tests documents one by one with fieldsearch. They are still searched last in an AND, against the candidates the other terms leave.
• search_or_operands keeps them out of its batches.
• query_struct_to_sql_str's hasmember case is now an error, since no search reaches it. No hasmember value goes into SQL.

This is what DID-python's SQLiteDB already does: _query_struct_to_sql_str returns None for hasmember. The two languages now apply the same rule in the same way. Cost: a hasmember that isn't narrowed by an AND (to at most maxSearchCandidates) loads every document of the branch, as in Python.

main (#238, the #237 ~or fix) is merged in, with no conflict.

Tests

TestHasMemberQuoting (new)

• testRule: fieldsearch against a table of 19 value/member cases, each also negated. The same table is in DID-python's test_rule.
• db.search compared with fieldsearch for:
  • text values ('value', '1 OR 1=1', 'say "hi"', 'O''Brien', 'x, value', 'name17', 'zzz', '5', '')
  • numbers, including Inf, -Inf and NaN
  • each also as ~hasmember
• text and numbers don't mix (5 in the text '5').
• a struct field has no members.
• a field no document has: ~hasmember matches every document.
• array and cell fields ([1 2 3], {'alpha','beta'}), which SQL could not search.
• inside an AND: only the 3 candidates are loaded, and the branch is not listed.
• inside an OR: the results match, and no SQL search is run.

TestBatchedOr

hasmember moves out of the SQL-term list (allTerms) into a new testBruteForceHasMemberInAnOr, next to the brute-forced depends_on test.

Related

• DID-python counterpart: hasmember: one rule in field_search, as in DID-matlab (#236) DID-python#108. It moves the bridge hashes to 74ae15f1 (database) and f4ed409e (fieldsearch).
• Merge order: merge this PR first, with a merge commit (not squash), so the hashes DID-python's bridge records stay on main.

🤖 Generated with Claude Code

https://claude.ai/code/session_01C7XsYS2DKSy2ucFzEEWR3D

[github-actions]

Test Results

545 tests  +30   545 ✅ +30   2m 44s ⏱️ -8s
 44 suites + 1     0 💤 ± 0 
  1 files   ± 0     0 ❌ ± 0 

Results for commit 74ae15f. ± Comparison against base commit 6f9be24.

♻️ This comment has been updated with latest results.

[codecov]

Codecov Report

❌ Patch coverage is 96.15385% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 67.53%. Comparing base (6f9be24) to head (74ae15f).

Files with missing lines	Patch %	Lines
src/did/+did/database.m	87.50%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #239      +/-   ##
==========================================
+ Coverage   67.40%   67.53%   +0.13%     
==========================================
  Files          67       67              
  Lines        4807     4821      +14     
==========================================
+ Hits         3240     3256      +16     
+ Misses       1567     1565       -2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.