Skip to content

Commit acedfa6

Browse files
fix(eval): raise the Invalid filter error for filters without an equals sign (#826)
## Summary `parse_filters` looks for the `=` separator with `str.index`, which raises `ValueError: substring not found` when the separator is missing. The `-1` check and the `Invalid filter` message that follow it never run. This change switches to `str.find` so a filter without `=` raises `ValueError: Invalid filter <filter>`, matching the JavaScript SDK. ## Changes - Use `str.find("=")` in `parse_filters` so the existing `Invalid filter` error is reachable. - Add tests for the error message and for splitting on the first `=` only. ## Testing ``` cd py python -m pytest src/braintrust/test_framework.py -k "parse_filters or evaluate_filter" ``` Before the change: 1 failed, 9 passed. After the change: 10 passed.
1 parent ed5dba4 commit acedfa6

2 files changed

Lines changed: 10 additions & 1 deletion

File tree

‎py/src/braintrust/framework.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1157,7 +1157,7 @@ def deserialize_plain_string_as_json(s: str) -> Any:
11571157
def parse_filters(filters: list[str]) -> list[Filter]:
11581158
result = []
11591159
for f in filters:
1160-
equals_idx = f.index("=")
1160+
equals_idx = f.find("=")
11611161
if equals_idx == -1:
11621162
raise ValueError(f"Invalid filter {f}")
11631163
path, value = f[:equals_idx], f[equals_idx + 1 :]

‎py/src/braintrust/test_framework.py‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1462,6 +1462,15 @@ def test_evaluate_filter_nested_metadata(self):
14621462
f = Filter(path=["metadata", "priority"], pattern=re.compile("^P0$"))
14631463
assert evaluate_filter(datum, f) is True
14641464

1465+
def test_parse_filters_splits_on_first_equals(self):
1466+
filters = parse_filters(["metadata.name=a=b"])
1467+
assert filters[0].path == ["metadata", "name"]
1468+
assert filters[0].pattern.pattern == "a=b"
1469+
1470+
def test_parse_filters_rejects_filter_without_equals(self):
1471+
with pytest.raises(ValueError, match="Invalid filter metadata.name"):
1472+
parse_filters(["metadata.name"])
1473+
14651474
def test_evaluate_filter_input_field(self):
14661475
datum = EvalCase(input={"text": "hello world"}, metadata={"name": "foo"})
14671476
f = Filter(path=["input", "text"], pattern=re.compile("hello"))

0 commit comments

Comments
 (0)