Skip to content

Bound what request validation and JSON reading cost (B03) - #18

Closed
AsyncAssassin wants to merge 2 commits into
mainfrom
fix/amount-length-first
Closed

AsyncAssassin wants to merge 2 commits into
mainfrom
fix/amount-length-first

Conversation

@AsyncAssassin

@AsyncAssassin AsyncAssassin commented Sep 23, 2026 •

Copy link
Copy Markdown
Owner

Summary

B03 from the merged review of v0.4.0; 0.4.1 is released once all fixes are merged. Validation of POST /api/v1/observed-events parsed amount into a BigDecimal even when the string broke its 80-character limit: Hibernate Validator checks every constraint, and the parse takes time quadratic in the length. A million-digit amount held a request thread for about ten seconds before the 400, and Jackson accepted strings of up to 20 million characters. Any OPERATOR could trigger it, and anyone in local.

  • Length first: the amount check leaves a value longer than 80 characters to @Size alone, and the parse itself refuses such a value, so a caller that skips validation cannot reach it either.
  • JSON string limit: the application's ObjectMapper accepts strings of at most 100 000 characters instead of Jackson's 20 million. Every field that reads a string already had a far lower limit, so no valid request or bridge page is refused. A hit on the limit says so, in the API detail (400 invalid-request) and in the bridge page error.

The regression review of the first version (/code-review) found the same cost next to it, fixed in the second commit:

  • externalRef and label: their non-blank pattern .*\S.* backtracked quadratically on a long value that ends in a line break, and validation ran it after @Size had failed. The pattern is now (?s).*\S.*, which is linear. A value with a line break is no longer reported as blank; control characters stay with B27.
  • YAML bodies: Spring MVC also read application/yaml, because springdoc brings the YAML data format, with none of the JSON limits. The converter is removed, so YAML gets 415 as docs/architecture.md already said.
  • Unknown fields: Jackson buffered them until the known fields were complete, at many times their size, and applied the string limit to them depending on key order. The request DTOs and the bridge page DTOs now skip unknown fields as they read them.
  • Unicode spaces: the getters that leave a blank value to @NotBlank used Kotlin's isBlank(), while @NotBlank uses Java's trim(), so an amount, direction, or status of U+00A0 passed validation and failed later as invalid-request without the errors list.

Other amount parsers need no change: the Alchemy mapper accepts at most 64 hex digits before it parses, and Jackson bounds the bridge's BigDecimal to 1000 characters, verified for both a JSON number and a string.

docs/api.md, the bridge page contract in docs/architecture.md, docs/failure-modes.md, and docs/testing.md describe the limits. CHANGELOG.md lists the changes under [Unreleased] → Security and Fixed.

Verification

  • ./gradlew clean check: 284 tests, 0 failures, 2 skipped (the env-gated Alchemy live smoke).

  • RequestDtoValidationTests validates the DTOs directly under preemptive one-second timeouts:

    • a million-digit amount gets one length violation;
    • mapping it without validation fails at once;
    • a 100 000-character externalRef or label ending in a line break gets one length violation;
    • Unicode spaces fail the field's own rule.
  • Integration tests:

    • a 100 000-digit amount gets only the length error;
    • a 100 001-character string gets invalid-request with the limit detail;
    • an unknown 200 000-character field placed first is ignored and the event is created;
    • a YAML body gets 415 and creates nothing.
  • The bridge test builds its mapper with the application's customizer: unknown 100 001-character fields in the page and in an event are skipped, and a 100 001-character nextCursor fails the page with the limit message.

  • Mutation check: removing each fix (YAML converter, (?s), ignoreUnknown on the request and on the page, the Unicode blank test, the parse guard) turns its test red.

  • Live local run over HTTP with a 256 MiB heap, the main jar against this branch:

    Request main This branch
    amount of 100 000 digits 400 after 0.27 s, errors amountValid and amount 400 after 0.10 s, error amount only
    amount of 1 000 000 digits 400 after 11.9 s 400 invalid-request after 0.005 s
    externalRef of 100 000 characters ending in \n 400 after 22.2 s, two errors 400 after 0.04 s, length error only
    YAML body to POST /api/v1/accounts 201, account created 415 unsupported-media-type
    20 MB body, unknown array before externalRef 500, OutOfMemoryError in the log 201 after 0.11 s
    amount of U+00A0 400 invalid-request, no errors list 400 validation-failed, amountValid

This PR and #15, #16, and #17 each add a section under [Unreleased] in CHANGELOG.md, so each later merge needs a one-file rebase. The other files merge cleanly with all three.

Validation of POST /api/v1/observed-events parsed amount into a
BigDecimal even when the string broke its 80-character limit: the
validator checks every constraint, and the parse takes time quadratic
in the length. A million-digit amount held a request thread for about
ten seconds before the 400, and Jackson accepted strings of up to
20 million characters.

The amount check now leaves an over-long value to @SiZe alone. The
application's ObjectMapper caps strings at 100 000 characters, far
above the longest legitimate one, a provider cursor of at most 4096
by default; a longer string fails a request with 400 invalid-request
and a bridge page as malformed JSON.
Review of the amount fix found the same cost next to it. The non-blank
pattern of externalRef and label backtracked quadratically on a long
value that ends in a line break, and validation ran it after @SiZe had
failed: at the new 100 000-character cap it held a request thread for
about ten seconds. It now lets the dot cross line breaks, which makes
it linear; a value with a line break is no longer reported as blank.

The string cap reached JSON only. Spring MVC also read application/yaml
bodies, because springdoc brings the YAML data format, with no limit;
that converter is removed, so YAML gets 415 as the docs always said.
Request and bridge page DTOs skip unknown fields instead of buffering
them until the known ones are complete, which held many times a body's
size in heap and made the cap depend on key order. A hit on the cap
now says so, in the API detail and in the bridge page error.

The amount parse refuses an over-long value itself, so a caller that
skips validation cannot reach the quadratic parse. The getters that
leave a blank value to @notblank use its definition of blank, so a
value of Unicode spaces gets validation-failed instead of passing
validation and failing later without the errors list.
@AsyncAssassin

Copy link
Copy Markdown
Owner Author

Combined into #23 together with the other batch-1 fixes, so they merge without a rebase per PR. This description keeps the detailed evidence for its fix.

@AsyncAssassin
AsyncAssassin deleted the fix/amount-length-first branch September 24, 2026 05:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant