Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 33 additions & 4 deletions src/did/datastructures.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import json
import numbers
import re

import numpy as np
Expand Down Expand Up @@ -190,6 +191,37 @@ def or_operands(search_struct):
return operands


def _has_member(value, member):
"""Is *member* a member of *value*, for field_search's 'hasmember'?

Text is a member of text that is the whole value or an item of its
comma-separated list (white space after a comma, and before one, is
ignored); anything is a member of a list that holds an equal element; a
number is a member of an equal number. Text is never a member of a
number, nor a number of text. This is what DID-matlab's fieldsearch and
its SQL search match (DID-matlab#236).
"""

def is_number(x):
return isinstance(x, numbers.Real) and not isinstance(x, str)

if isinstance(value, np.ndarray):
value = value.tolist()
if isinstance(member, str):
if isinstance(value, str):
pattern = r"(^|,\s*)" + re.escape(member) + r"\s*(,|$)"
return re.search(pattern, value) is not None
if isinstance(value, (list, tuple)):
return any(isinstance(v, str) and v == member for v in value)
return False
if is_number(member):
if is_number(value):
return value == member
if isinstance(value, (list, tuple)):
return any(is_number(v) and v == member for v in value)
return False


def field_search(a, search_struct):
"""
Searches a dictionary to determine if it matches a search structure.
Expand Down Expand Up @@ -260,10 +292,7 @@ def field_search(a, search_struct):
b = eq_len(np.array(value).shape, param1)
elif op_lower == "hasmember":
if is_there:
try:
b = param1 in value
except TypeError:
pass
b = _has_member(value, param1)
elif op_lower == "hasfield":
b = is_there
elif op_lower == "partial_struct":
Expand Down
21 changes: 20 additions & 1 deletion src/did/did_matlab_python_bridge.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ classes:
- name: database
type: class
matlab_path: "+did/database.m"
matlab_last_sync_hash: "2c9d8fa7"
matlab_last_sync_hash: "74ae15f1"
status: ported
python_path: "did/database.py"
python_class: "Database"
Expand Down Expand Up @@ -124,6 +124,25 @@ classes:
did/database.py nor did/implementations/sqlitedb.py changes.
tests/test_negated_or.py pins the behaviour with the same cases as
MATLAB's did.unittest.TestNegatedOr.

DID-matlab c8adf741..74ae15f1 (DID-matlab#236, 2026-10-24):
query_struct_to_sql_str put a text 'hasmember' value into the SQL
unquoted (and as a double-quoted token, which SQLite reads as a column
when it names one), so 'value' matched every document and '1 OR 1=1'
was SQL injection; and the SQL could not see the members of an array
or cell, which doc2sql stores only as '[1 3] double'. MATLAB now
searches hasmember by brute force with fieldsearch, as Python always
has: the new protected static isBruteForceTerm (hasmember, and the
depends_on queries of isBruteForceDependsOn) sends a term to
brute_force_search, searched last in an AND and kept out of the OR
batches, and query_struct_to_sql_str's 'hasmember' case is an error.
That is what SQLiteDB already does (_query_struct_to_sql_str returns
None for hasmember; _SCAN_OPERATIONS ranks it last), so did/database.py
and did/implementations/sqlitedb.py do not change; the rule itself
changed in fieldsearch and field_search -- see the fieldsearch entry.
2c08472c merges DID-matlab main (2c9d8fa7, #237, above) into that
branch. tests/test_hasmember_quoting.py mirrors DID-matlab's
TestHasMemberQuoting.
sync_notes: >
Schema validation was ported on 2026-08-28, closing what had been the
largest capability gap between the two languages. Python previously had
Expand Down
16 changes: 15 additions & 1 deletion src/did/did_matlab_python_bridge_util.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -317,7 +317,7 @@ functions:
- name: fieldsearch
type: function
matlab_path: "+did/+datastructures/fieldsearch.m"
matlab_last_sync_hash: "051f535"
matlab_last_sync_hash: "f4ed409e"
status: ported
python_path: "did/datastructures.py"
python_name: "field_search"
Expand All @@ -330,6 +330,20 @@ functions:
holds a non-struct value (depends_on = [] read back from a database)
used to return early, skipping the negation; it now matches. Python's
field_search already negated it; nothing to port.
DID-matlab f4ed409e (DID-matlab#236, 2026-10-24): 'hasmember' had a
different rule in each place. MATLAB's fieldsearch used
ismember(param1, value), which compares two texts character by
character; its SQL search matched an item of a comma-separated list;
field_search used `in` (a substring test, and a TypeError -- no match
-- on a scalar number). All three now match one rule, MATLAB's in the
local function fieldsearch/hasMember and Python's in _has_member: text
is a member of text that is the whole value or an item of its
comma-separated list (pattern (^|,\s*)M\s*(,|$)); anything is a
member of a list (MATLAB cell or numeric array) that holds an equal
element, and a number of an equal number; text is never a member of a
number, nor a number of text; a dict (MATLAB struct) has no members.
tests/test_hasmember_quoting.py test_rule runs the same table of
cases as DID-matlab's TestHasMemberQuoting/testRule.

- name: orOperands
type: function
Expand Down
Loading
Loading