Enforce maxStringLength when skipping strings - #1736
Dongnyoung wants to merge 2 commits into
Conversation
|
These constraints were never meant to be exact and they are really about protecting against inputs that are crafted to make the parser use more memory than you would expect. |
|
@pjfanning Thanks, that’s a good point. I agree that skipping a String does not materialize it, so this path does not create the same memory concern as reading the String value. My reason for opening this PR was behavioral consistency: the 3.x synchronous parsers currently enforce maxStringLength while skipping strings, whereas the 2.x parsers accept an oversized string if the caller advances without accessing its value. I was wondering whether the constraint should apply to the string token regardless of whether it is materialized. If maxStringLength is intended only to limit memory used when materializing strings, then exempting skipped strings makes sense. I’ll leave this open for further review. |
Summary
Fixes #1735
StreamReadConstraints.maxStringLengthwas not enforced when String values were skipped by synchronous parsers without being materialized.This change tracks String length in
_skipString()and applies the configuredmaxStringLengthconstraint for:ReaderBasedJsonParserUTF8StreamJsonParserUTF8DataInputJsonParserFor UTF-8 input, 4-byte characters are counted as two UTF-16 code units, consistent with Java String length semantics.
A regression test verifies that skipping an oversized String value by advancing to the next token throws
StreamConstraintsException.