Skip to content

Fix COMPRESS-731: document that the factory and the LZMA/XZ input streams have no memory limit by default - #841

Open
Inf1mum wants to merge 1 commit into
apache:masterfrom
Inf1mum:fix/COMPRESS-731-docs-memory-limits
Open

Inf1mum wants to merge 1 commit into
apache:masterfrom
Inf1mum:fix/COMPRESS-731-docs-memory-limits

Conversation

@Inf1mum

@Inf1mum Inf1mum commented Nov 16, 2026

Copy link
Copy Markdown

Fixes COMPRESS-731.

Problem

LZMA and XZ decompression may allocate up to about 2 GiB for the dictionary declared inside the stream (xz-java's DICT_SIZE_MAX, verified against xz-java 1.12: LZMAInputStream.DICT_SIZE_MAX = Integer.MAX_VALUE & ~15), but no documentation says the defaults are unbounded:

  • the Javadoc of new CompressorStreamFactory() and new CompressorStreamFactory(boolean) does not mention that these constructors run with memoryLimitInKb = -1, and the memoryLimitInKb parameter of CompressorStreamFactory(boolean, int) does not explain what -1 means;
  • the builders of LZMACompressorInputStream and XZCompressorInputStream default to -1, and the class-level examples show .setMemoryLimitKiB(-1) without saying what it does;
  • negative values other than -1 behave differently between the two formats in xz-java 1.12: LZMAInputStream checks memoryLimit != -1 && memoryNeeded > memoryLimit (so any other negative value rejects every stream), while the XZ BlockInputStream checks memoryLimit >= 0 (so any negative value disables the check);
  • src/site/xdoc/examples.xml says there is no way to detect the LZMA format, although LZMAUtils.matches() has been part of CompressorStreamFactory.detect() since 1.10, and its "Restricting Memory Usage" section describes the 1.14 state without recommending a limit for untrusted input.

Change

Documentation only:

  • CompressorStreamFactory: both existing-constructor Javadocs now state that no memory limit is imposed (LZMA/XZ may allocate up to about 2 GiB), and the memoryLimitInKb parameter documents -1 as no limit;
  • LZMACompressorInputStream.Builder.setMemoryLimitKiB and the deprecated two-arg constructor: default -1 = no limit; any other negative value makes the underlying decoder reject every stream;
  • XZCompressorInputStream: the class and builder examples explain .setMemoryLimitKiB(-1), and the setMemoryLimitKiB parameter doc states that any negative value (default -1) imposes no limit;
  • corrected copy-paste Javadoc found in the same files while documenting: XZCompressorInputStream.Builder linked LZMACompressorInputStream, both builder() methods linked LZMACompressorOutputStream, the LZMA builder example assigned to LZMACompressorOutputStream, and the two now-unused LZMACompressor* imports in XZCompressorInputStream are removed;
  • examples.xml: the compressor-detection note now says only Brotli cannot be detected (LZMA since 1.10), and "Restricting Memory Usage" documents the unbounded default, recommends a heap-fitting limit for untrusted input with a factory example, and points to the builders' setMemoryLimitKiB.

Verification

  • mvn test: 4781 tests, 0 failures, 0 errors (30 skipped, pre-existing);
  • mvn verify (includes apache-rat, checkstyle, japicmp): BUILD SUCCESS, no API differences reported;
  • javadoc:javadoc: BUILD SUCCESS;
  • factual claims checked against xz-java 1.12 sources (LZMAInputStream, XZInputStream, SingleXZInputStream, BlockInputStream).

What: documentation only. State in the Javadoc of the CompressorStreamFactory
no-arg and single-arg constructors, and of the LZMA and XZ builders'
setMemoryLimitKiB, that the default (-1) imposes no memory limit; fix the
outdated LZMA auto-detection note and extend the "Restricting Memory Usage"
section of the site documentation; correct the copy-paste Javadoc links and
examples in LZMACompressorInputStream and XZCompressorInputStream.

Why: COMPRESS-731. LZMA and XZ decompression may allocate up to about 2 GiB
for the dictionary declared inside the stream (xz-java DICT_SIZE_MAX), but
nothing says the factory and builder defaults are unbounded, and
examples.xml claims LZMA cannot be auto-detected although
LZMAUtils.matches() has been part of CompressorStreamFactory.detect() since
1.10. The "Restricting Memory Usage" section also predates the current
APIs and did not recommend setting a limit when decompressing untrusted
input.

How: Javadoc edits in CompressorStreamFactory, LZMACompressorInputStream
and XZCompressorInputStream, plus src/site/xdoc/examples.xml. The memory
section now recommends a heap-fitting limit for untrusted input with an
example, and documents that the builders expose the same limit. No
behavior change: japicmp reports no API differences.

JIRA: COMPRESS-731

Signed-off-by: Inf1mum <vadimlazuk5@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant