Skip to content

Type check every argument of variadic functions (fixes #329)#361

Open
Sanjays2402 wants to merge 1 commit into
jmespath:developfrom
Sanjays2402:fix/typecheck-variadic-args
Open

Type check every argument of variadic functions (fixes #329)#361
Sanjays2402 wants to merge 1 commit into
jmespath:developfrom
Sanjays2402:fix/typecheck-variadic-args

Conversation

@Sanjays2402

Copy link
Copy Markdown

Fixes #329

The bug

Calling merge() (or any variadic function) with an invalid type in any argument after the first leaks a raw Python exception instead of the JMESPath-spec invalid-type error:

>>> import jmespath
>>> jmespath.search('merge(`{"a": 1}`, `null`)', {})
Traceback (most recent call last):
  ...
  File ".../jmespath/functions.py", line 266, in _func_merge
    merged.update(arg)
TypeError: 'NoneType' object is not iterable

The original report (#329) hits this via merge(a, b || \null`)where a branch resolves tonull. The same leak occurs for numbers and arrays, and it is a ValueError` for strings:

expression before after
merge(\{"a":1}`, `null`)` TypeError: 'NoneType' object is not iterable JMESPathTypeError (invalid-type)
merge(\{"a":1}`, `5`)` TypeError: 'int' object is not iterable JMESPathTypeError
merge(\{"a":1}`, `[1,2]`)` TypeError: cannot convert dictionary update sequence ... JMESPathTypeError
merge(\{"a":1}`, `"s"`)` ValueError: dictionary update sequence element #0 has length 1; 2 is required JMESPathTypeError
merge(\{"a":1}`, `{"b":2}`)` {"a":1,"b":2} {"a":1,"b":2} (unchanged)

Note the first argument was already validated correctly — merge(\null`)` raised the proper error. Only the trailing variadic arguments slipped through.

Root cause

Functions._type_check iterates over range(len(signature)):

def _type_check(self, actual, signature, function_name):
    for i in range(len(signature)):
        allowed_types = signature[i]['types']
        ...

For a variadic function the signature has a single entry (e.g. merge's is {'types': ['object'], 'variadic': True}), so the loop only ever checks actual[0]. Every argument at index ≥ 1 skips validation and reaches the raw function body, where the un-typed value raises whatever low-level Python error the implementation happens to hit.

_validate_arguments already guarantees the arity (len(actual) >= len(signature) for variadic), so the trailing arguments are known to exist — they were simply never checked.

The fix

Iterate over the actual arguments and, once the fixed signature entries are exhausted, keep validating against the final (variadic) entry:

def _type_check(self, actual, signature, function_name):
    for i in range(len(actual)):
        if i < len(signature):
            allowed_types = signature[i]['types']
        else:
            # Variadic function: every trailing argument is described
            # by the final signature entry, so keep validating against
            # it instead of leaving the extra args unchecked.
            allowed_types = signature[-1]['types']
        if allowed_types:
            self._type_check_single(actual[i], allowed_types, function_name)
  • Non-variadic functions are unchanged: _validate_arguments enforces len(actual) == len(signature), so i never exceeds the signature and the behaviour is identical to before.
  • not_null() is unaffected: its variadic entry is {'types': [], 'variadic': True}; with an empty types list the if allowed_types guard skips the check, exactly as it did previously.

Tests

Added four compliance cases to tests/compliance/functions.json exercising an invalid variadic argument at positions 2 and 3 of merge (a null literal, a null-valued key reference, a number literal, and a string key reference), each expecting invalid-type.

Proof they guard the regression (source fix stashed, tests kept):

# without the fix
FAILED tests/test_compliance.py::test_error_expression[...merge(`{"a": 1}`, `null`)-invalid-type...]
FAILED tests/test_compliance.py::test_error_expression[...merge(`{"a": 1}`, null_key)-invalid-type...]
FAILED tests/test_compliance.py::test_error_expression[...merge(`{"a": 1}`, `{"b": 2}`, `5`)-invalid-type...]
3 failed, 46 passed

# with the fix
4 passed, 152 deselected

Full suite: 995 passed, 1 skipped (baseline was 991 passed, 1 skipped — exactly the 4 new cases; no regressions). flake8 clean on the changed source.

Variadic functions such as merge() only validated their first
argument. _type_check iterated over range(len(signature)), and for a
variadic function the signature has a single entry, so any argument
after the first bypassed type validation entirely and reached the raw
function body.

As a result, merge(`{"a": 1}`, `null`) raised a bare Python
TypeError ("'NoneType' object is not iterable") instead of the
spec-mandated invalid-type error (JMESPathTypeError). Non-object
second arguments leaked TypeError/ValueError for numbers, arrays and
strings as well.

Iterate over the actual arguments instead, clamping the index to the
final signature entry once it is exhausted, so each trailing variadic
argument is checked against the variadic type. not_null() is
unaffected: its variadic entry allows any type, so the check is
skipped exactly as before.

Added compliance cases covering an invalid variadic argument at
positions 2 and 3 for merge().

Fixes jmespath#329
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Different behavior on merge function between the JS lib and the Python lib

1 participant