Skip to content

fix(local): reject negative array indices in json path - #1340

Open
shashvat-singham wants to merge 1 commit into
qdrant:devfrom
shashvat-singham:fix/local-negative-json-path-index
Open

fix(local): reject negative array indices in json path#1340
shashvat-singham wants to merge 1 commit into
qdrant:devfrom
shashvat-singham:fix/local-negative-json-path-index

Conversation

@shashvat-singham

Copy link
Copy Markdown

What & why

In local mode a json path like a[-1] is accepted and silently resolves to the last element of the array, and a negative index past the start leaks a raw IndexError out of the filter.

Qdrant core parses a bracket index with digit1 mapped to usize, so a negative index is a parse error there:

delimited(char('['), number, char(']')).map(JsonPathItem::Index),

fn number(input: &str) -> IResult<&str, usize> {
    map_res(recognize(digit1), str::parse).parse(input)
}

Local mode used int(), which additionally accepts a sign, underscore separators and surrounding whitespace. The downstream bounds checks are written as current_key.index < len(data), which assume a non-negative index, so a negative one passes straight through to Python's own wrap-around indexing.

Reproduction on dev:

from qdrant_client.local.json_path_parser import parse_json_path
from qdrant_client.local.payload_value_extractor import value_by_key
from qdrant_client.local.payload_value_setter import set_value_by_key

payload = {"a": [10, 20, 30]}
value_by_key(payload, "a[-1]")   # [30]   -- expected: invalid path
value_by_key(payload, "a[-4]")   # IndexError: list index out of range

payload = {"a": [{"x": 1}, {"x": 2}]}
set_value_by_key(payload, parse_json_path("a[-1]"), {"x": 99})
# {'a': [{'x': 1}, {'x': {'x': 99}}]}  -- writes to the last element

parse_json_path("a[+1]")    # index=1
parse_json_path("a[1_0]")   # index=10
parse_json_path("a[ 1 ]")   # index=1

So both the read path (value_by_key, and therefore filters and count) and the write path (set_value_by_key, and therefore set_payload) are affected.

The fix

Validate the bracket contents against the same grammar as core in _match_brackets, rather than deferring to int(). That is a single-point fix: with the index guaranteed non-negative, the existing index < len(data) checks become sound again, so no call site needed changing.

isascii() is part of the check because str.isdigit() is also true for characters like ² and ٣, which digit1 does not accept.

A note on the existing test

test_set_value_by_key already asserted this behaviour:

    try:
        ...
        key = "a[-1]"
        set_value_by_key(payload, parse_json_path(key), new_value)
        assert False, "Negative indexation is not supported"
    except Exception:
        assert True

It could never fail: assert False raises AssertionError, which its own except Exception then swallowed. Meanwhile the payload was being mutated. I switched that whole # region exceptions block to pytest.raises(ValueError) so the intent is actually enforced — happy to split that into its own PR if you would rather keep this one to the parser change.

Tests

Added regression coverage to test_parse_json_path, test_value_by_key and test_set_value_by_key. All three fail on dev without the parser change and pass with it.

$ python -m pytest qdrant_client/local/tests/ -q
72 passed in 7.53s

ruff-format --line-length=99 and mypy are clean on the changed files. tests/test_local_persistence.py has 4 failures on my machine both with and without this change (Windows PermissionError on tempfile cleanup), so they are unrelated.


All Submissions:

  • Contributions should target the dev branch. Did you create your branch from dev?
  • Have you followed the guidelines in our Contributing document?
  • Have you checked to ensure there aren't other open Pull Requests for the same update/change?

Changes to Core Features:

  • Have you added an explanation of what your changes do and why you'd like us to include them?
  • Have you written new tests for your core changes, as applicable?
  • Have you successfully ran tests with your changes locally?

qdrant core parses a bracket index with `digit1` mapped to `usize`, so
`a[-1]` is a parse error. Local mode used `int()`, which accepts a sign,
underscore separators and surrounding whitespace, and the downstream
`index < len(data)` bounds checks then let the negative index through to
Python's own wrap-around indexing.

As a result `value_by_key` and `set_value_by_key` silently addressed
elements from the end of the list, and a negative index past the start
leaked a raw IndexError out of the filter.

Validate the index against the same grammar as core instead.

The existing "Negative indexation is not supported" test could not catch
this: `assert False` raises AssertionError, which its own `except
Exception` swallowed. Switch that region to `pytest.raises`.
@netlify

netlify Bot commented Aug 15, 2026

Copy link
Copy Markdown

Deploy Preview for poetic-froyo-8baba7 ready!

Name Link
🔨 Latest commit db80927
🔍 Latest deploy log https://app.netlify.com/projects/poetic-froyo-8baba7/deploys/6a80230c318888000810f398
😎 Deploy Preview https://deploy-preview-1340--poetic-froyo-8baba7.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@coderabbitai

coderabbitai Bot commented Aug 15, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 703ce1a1-6ceb-4fdf-a73e-8cad2093a355

📥 Commits

Reviewing files that changed from the base of the PR and between f003e6c and db80927.

📒 Files selected for processing (2)
  • qdrant_client/local/json_path_parser.py
  • qdrant_client/local/tests/test_payload_utils.py

📝 Walkthrough

Walkthrough

The JSON path parser now accepts only unsigned ASCII digits for array indices. It rejects signs, whitespace, underscores, non-ASCII digits, and malformed brackets with ValueError. Tests cover parser validation, negative indices in value_by_key, and explicit error assertions for malformed paths.

Estimated code review effort: 2 (Simple) | ~10 minutes

Merge Risk: ⚪ Minimal · up to db809

This localized change rejects invalid negative and non-core array indices while preserving valid JSON paths, with regression coverage added; no actionable merge-blocking risk remains after normal checks and review.

Possibly related PRs

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely describes the main change: rejecting negative array indices in local JSON paths.
Description check ✅ Passed The description explains the bug, the core-compatible parser fix, affected paths, regression tests, and validation results.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant