Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -504,6 +504,11 @@ private static String youNeed(final String name, final String url) {

/**
* Constructs an instance with the decompress Concatenated option set to false.
* <p>
* No memory limit is imposed: the LZMA and XZ decompressors will allocate as much memory as the stream requires - up to about 2 GiB.
* </p>
*
* @see #CompressorStreamFactory(boolean, int)
*/
public CompressorStreamFactory() {
this.decompressUntilEof = null;
Expand All @@ -512,9 +517,13 @@ public CompressorStreamFactory() {

/**
* Constructs an instance with the provided decompress Concatenated option.
* <p>
* No memory limit is imposed: the LZMA and XZ decompressors will allocate as much memory as the stream requires - up to about 2 GiB.
* </p>
*
* @param decompressUntilEOF if true, decompress until the end of the input; if false, stop after the first stream and leave the input position to point to
* the next byte after the stream. This setting applies to the gzip, bzip2 and XZ formats only.
* @see #CompressorStreamFactory(boolean, int)
* @since 1.10
*/
public CompressorStreamFactory(final boolean decompressUntilEOF) {
Expand All @@ -527,7 +536,8 @@ public CompressorStreamFactory(final boolean decompressUntilEOF) {
* @param decompressUntilEOF if true, decompress until the end of the input; if false, stop after the first stream and leave the input position to point to
* the next byte after the stream. This setting applies to the gzip, bzip2 and XZ formats only.
* @param memoryLimitInKb Some streams require allocation of potentially significant byte arrays/tables, and they can offer checks to prevent OOMs on
* corrupt files. Set the maximum allowed memory allocation in KBs.
* corrupt files. Set the maximum allowed memory allocation in KBs, or {@code -1} to impose no memory limit - the behavior of
* the {@link #CompressorStreamFactory()} and {@link #CompressorStreamFactory(boolean)} constructors.
*
* @since 1.14
*/
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ public class LZMACompressorInputStream extends CompressorInputStream implements
* For example:
* </p>
* <pre>{@code
* LZMACompressorOutputStream s = LZMACompressorInputStream.builder()
* LZMACompressorInputStream s = LZMACompressorInputStream.builder()
* .setPath(path)
* .get();
* }
Expand Down Expand Up @@ -74,9 +74,13 @@ public LZMACompressorInputStream get() throws IOException {

/**
* Sets a working memory threshold in kibibytes (KiB).
* <p>
* The default is {@code -1}: no memory limit is imposed and the stream may allocate as much memory as the LZMA dictionary size requires - up to about
* 2 GiB. Any negative value other than {@code -1} makes the underlying LZMA decoder reject every stream with a {@link MemoryLimitException}.
* </p>
*
* @param memoryLimitKiB Sets a working memory threshold in kibibytes (KiB). Processing throws MemoryLimitException if memory use is above this.
* threshold.
* @param memoryLimitKiB Sets a working memory threshold in kibibytes (KiB), or {@code -1} to impose no memory limit. Processing throws
* MemoryLimitException if memory use is above this threshold.
* @return {@code this} instance.
*/
public Builder setMemoryLimitKiB(final int memoryLimitKiB) {
Expand All @@ -86,9 +90,9 @@ public Builder setMemoryLimitKiB(final int memoryLimitKiB) {
}

/**
* Constructs a new builder of {@link LZMACompressorOutputStream}.
* Constructs a new builder of {@link LZMACompressorInputStream}.
*
* @return A new builder of {@link LZMACompressorOutputStream}.
* @return A new builder of {@link LZMACompressorInputStream}.
* @since 1.28.0
*/
public static Builder builder() {
Expand Down Expand Up @@ -136,7 +140,8 @@ public LZMACompressorInputStream(final InputStream inputStream) throws IOExcepti
* Creates a new input stream that decompresses LZMA-compressed data from the specified input stream.
*
* @param inputStream where to read the compressed data.
* @param memoryLimitKiB Sets a working memory threshold in kibibytes (KiB). Processing throws MemoryLimitException if memory use is above this threshold.
* @param memoryLimitKiB Sets a working memory threshold in kibibytes (KiB), or {@code -1} to impose no memory limit. Processing throws MemoryLimitException
* if memory use is above this threshold.
* @throws IOException if the input is not in the .lzma format, the input is corrupt or truncated, the .lzma headers specify sizes that are not supported by
* this implementation, or the underlying {@code inputStream} throws an exception.
*
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,6 @@

import org.apache.commons.compress.MemoryLimitException;
import org.apache.commons.compress.compressors.CompressorInputStream;
import org.apache.commons.compress.compressors.lzma.LZMACompressorInputStream;
import org.apache.commons.compress.compressors.lzma.LZMACompressorOutputStream;
import org.apache.commons.compress.utils.InputStreamStatistics;
import org.apache.commons.io.IOUtils;
import org.apache.commons.io.build.AbstractStreamBuilder;
Expand All @@ -44,7 +42,7 @@
* XZCompressorInputStream s = XZCompressorInputStream.builder()
* .setPath(path)
* .setDecompressConcatenated(false)
* .setMemoryLimitKiB(-1)
* .setMemoryLimitKiB(-1) // The default: no memory usage limit
* .get();
* }
* </pre>
Expand All @@ -56,7 +54,7 @@ public class XZCompressorInputStream extends CompressorInputStream implements In

// @formatter:off
/**
* Builds a new {@link LZMACompressorInputStream}.
* Builds a new {@link XZCompressorInputStream}.
*
* <p>
* For example:
Expand All @@ -65,7 +63,7 @@ public class XZCompressorInputStream extends CompressorInputStream implements In
* XZCompressorInputStream s = XZCompressorInputStream.builder()
* .setPath(path)
* .setDecompressConcatenated(false)
* .setMemoryLimitKiB(-1)
* .setMemoryLimitKiB(-1) // The default: no memory usage limit
* .get();
* }
* </pre>
Expand Down Expand Up @@ -107,8 +105,9 @@ public Builder setDecompressConcatenated(final boolean decompressConcatenated) {
/**
* Sets a working memory threshold in kibibytes (KiB).
*
* @param memoryLimitKiB The memory limit used when reading blocks. The memory usage limit is expressed in kibibytes (KiB) or {@code -1} to impose no
* memory usage limit. If the estimated memory limit is exceeded on {@link #read()}, a {@link MemoryLimitException} is thrown.
* @param memoryLimitKiB The memory limit used when reading blocks. The memory usage limit is expressed in kibibytes (KiB); any negative value (the
* default is {@code -1}) imposes no memory usage limit. If the estimated memory limit is exceeded on {@link #read()}, a
* {@link MemoryLimitException} is thrown.
* @return {@code this} instance.
*/
public Builder setMemoryLimitKiB(final int memoryLimitKiB) {
Expand All @@ -118,9 +117,9 @@ public Builder setMemoryLimitKiB(final int memoryLimitKiB) {
}

/**
* Constructs a new builder of {@link LZMACompressorOutputStream}.
* Constructs a new builder of {@link XZCompressorInputStream}.
*
* @return A new builder of {@link LZMACompressorOutputStream}.
* @return A new builder of {@link XZCompressorInputStream}.
* @since 1.28.0
*/
public static Builder builder() {
Expand Down
23 changes: 19 additions & 4 deletions src/site/xdoc/examples.xml
Original file line number Diff line number Diff line change
Expand Up @@ -84,9 +84,10 @@ CompressorInputStream input = new CompressorStreamFactory()
.createCompressorInputStream(originalInput);
]]></source>

<p>Note that there is no way to detect the LZMA or Brotli formats so only
the two-arg version of
<code>createCompressorInputStream</code> can be used. Prior
<p>Note that the Brotli format cannot be detected so only the two-arg
version of
<code>createCompressorInputStream</code> can be used for it. LZMA
has been auto-detected since Compress 1.10. Prior
to Compress 1.9 the .Z format hasn't been auto-detected
either.</p>

Expand All @@ -98,7 +99,21 @@ CompressorInputStream input = new CompressorStreamFactory()
constructor argument that can be used to set an upper limit of
memory that may be used while decompressing or compressing a
stream. As of 1.14 this setting only affects decompressing Z,
XZ and LZMA compressed streams.</p>
XZ and LZMA compressed streams. By default no limit is set,
which means the LZMA and XZ decompressors may allocate up to
about 2 GiB of memory for the dictionary declared inside the
stream. When decompressing untrusted input you should set a
limit that fits the memory available to your application:</p>
<source><![CDATA[
CompressorInputStream input = new CompressorStreamFactory(true, 64 * 1024)
.createCompressorInputStream(originalInput);
]]></source>
<p>If decompressing requires more memory than the limit allows,
a <code>MemoryLimitException</code> is thrown before the memory
is allocated. The builders of
<code>LZMACompressorInputStream</code> and
<code>XZCompressorInputStream</code> offer the same limit via
their <code>setMemoryLimitKiB</code> method.</p>
<p>Since Compress 1.19 <code>SevenZFile</code> also has an
optional constructor to pass an upper memory limit which is supported
in LZMA compressed streams. Since Compress 1.21 this setting
Expand Down