Skip to content

Enforce maxStringLength when skipping strings - #1736

Open
Dongnyoung wants to merge 2 commits into
FasterXML:2.xfrom
Dongnyoung:fix-skipped-string-max-length-2x
Open

Dongnyoung wants to merge 2 commits into
FasterXML:2.xfrom
Dongnyoung:fix-skipped-string-max-length-2x

Conversation

@Dongnyoung

Copy link
Copy Markdown
Contributor

Summary

Fixes #1735

StreamReadConstraints.maxStringLength was not enforced when String values were skipped by synchronous parsers without being materialized.

This change tracks String length in _skipString() and applies the configured maxStringLength constraint for:

  • ReaderBasedJsonParser
  • UTF8StreamJsonParser
  • UTF8DataInputJsonParser

For UTF-8 input, 4-byte characters are counted as two UTF-16 code units, consistent with Java String length semantics.

A regression test verifies that skipping an oversized String value by advancing to the next token throws StreamConstraintsException.

@pjfanning

Copy link
Copy Markdown
Member

These constraints were never meant to be exact and they are really about protecting against inputs that are crafted to make the parser use more memory than you would expect.
If we are skipping over content in a streaming way and don't keep it in memory, do we really care about tracking this?
This is just my opinion so don't close this just based on these comments. It's useful to have other people review this too.

@Dongnyoung

Copy link
Copy Markdown
Contributor Author

@pjfanning Thanks, that’s a good point. I agree that skipping a String does not materialize it, so this path does not create the same memory concern as reading the String value.

My reason for opening this PR was behavioral consistency: the 3.x synchronous parsers currently enforce maxStringLength while skipping strings, whereas the 2.x parsers accept an oversized string if the caller advances without accessing its value. I was wondering whether the constraint should apply to the string token regardless of whether it is materialized.

If maxStringLength is intended only to limit memory used when materializing strings, then exempting skipped strings makes sense. I’ll leave this open for further review.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants