Skip to content

cldotdev/archive-sift

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

4 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

archive-sift

Extract suspicious files from large archives for malware analysis.

Game ISOs and other archives often exceed upload limits for analysis services. This tool extracts only the files that matter for malware analysis (executables, scripts, installers) and writes them to a ZIP archive or directory. Both modes generate a manifest with SHA-256 hashes, so you can look up known files before uploading.

All processing runs inside an isolated Docker container with no network access.

The extracted output is live malware. Files in the output ZIP or directory may be actual malicious executables, scripts, or installers. Do not open, execute, or double-click anything in the output on a normal workstation. Transfer the output to an analysis environment (VM snapshot, isolated host, malware sandbox) before inspecting it.

Prerequisites

Usage

bin/archive-sift --input <file-or-directory> --output <path> [--pattern <regex>]

If --output ends with .zip, the tool produces a ZIP archive. Otherwise, it writes extracted files to a directory.

Both modes generate a manifest listing each file's SHA-256 hash, size, detected MIME type, and path. For ZIP output, the manifest is written alongside the ZIP as <stem>.manifest.txt. For directory output, it is written as manifest.txt inside the directory.

Options

Option Required Description
--input Yes Path to an archive file or a directory containing archives
--output Yes Path for the output (ZIP file or directory)
--pattern No Override the default suspicious file extension regex

Examples

Extract suspicious files from a single ISO into a ZIP:

bin/archive-sift --input /downloads/game.iso --output suspicious.zip

Extract to a directory (no compression):

bin/archive-sift --input /downloads/game.iso --output /tmp/extracted

Process all archives in a directory:

bin/archive-sift --input /downloads/games/ --output suspicious.zip

Extract only executables:

bin/archive-sift --input game.7z --output out.zip --pattern '\.(exe|dll)$'

Upload the output to a malware analysis service (VirusTotal, Hybrid Analysis, MalwareBazaar, or an internal sandbox) or transfer it to an isolated VM for manual inspection.

What Gets Extracted

By default, the tool extracts files matching these extensions:

Category Extensions
PE executables .exe .dll .sys .scr .com .drv .ocx .cpl
Installers .msi .msix
Scripts .bat .cmd .ps1 .vbs .vbe .wsf .hta
Shortcuts and configuration .lnk .inf .reg
Cross-platform .jar

Extension matching is position-independent: a file named Setup.exe.txt matches because .exe appears as a valid extension within the name. This catches double-extension disguise tricks where a suspicious extension is hidden before a benign trailing extension.

Use --pattern to override with a custom regex.

Supported Archive Formats

Any format supported by 7-Zip: ZIP, ISO, 7z, RAR, CAB, WIM, TAR, GZ, BZ2, XZ, and more.

When --input is a directory, the tool scans for files with these extensions: .zip, .iso, .7z, .rar, .cab, .wim, .tar, .gz, .bz2, .xz.

Resource Limits

The tool enforces caps to prevent resource exhaustion from crafted archives. All limits are configurable via environment variables:

Variable Default Description
ARCHIVE_SIFT_MAX_FILES 10000 Maximum number of matching files per archive
ARCHIVE_SIFT_MAX_FILE_BYTES 1073741824 (1 GiB) Maximum uncompressed size of any single file (matching files and nested archives)
ARCHIVE_SIFT_MAX_TOTAL_BYTES 4294967296 (4 GiB) Maximum total uncompressed size of all matching files and nested archives
ARCHIVE_SIFT_TIMEOUT 1800 (30 minutes) Wall-clock timeout in seconds for extraction (per-archive cap)
ARCHIVE_SIFT_MAX_NEST_DEPTH 3 Maximum nesting depth for recursive archive extraction

When a cap is exceeded, the archive is skipped with an error message naming the cap and the observed value. File count and total size caps are enforced cumulatively across all recursion levels for a single top-level archive.

Example:

ARCHIVE_SIFT_MAX_FILES=500 bin/archive-sift --input huge.iso --output out.zip

Security Model

The Docker container runs with strict isolation:

  • --network=none: no network access
  • --read-only: read-only root filesystem
  • --cap-drop=ALL: all Linux capabilities removed
  • --security-opt=no-new-privileges: no privilege escalation
  • Custom seccomp profile (docker/seccomp.json): restricts available syscalls beyond Docker's default (no ptrace, bpf, chroot, clone3, mount, unshare, and others)
  • Input mounted read-only
  • Temporary files use only in-container tmpfs
  • Memory limited to 4 GB, process count limited to 256

Seccomp Override

To use a custom seccomp profile:

ARCHIVE_SIFT_SECCOMP_PATH=/path/to/profile.json bin/archive-sift --input archive.zip --output out.zip

To disable seccomp filtering (not recommended):

ARCHIVE_SIFT_SECCOMP_PATH=unconfined bin/archive-sift --input archive.zip --output out.zip

Output Safety

The tool never deletes or modifies a pre-existing path at --output when no suspicious files are found. Output is materialized via a staging directory and moved to --output only on success.

How It Works

  1. The host-side script (bin/archive-sift) builds the Docker image if needed
  2. It launches an isolated container with the input mounted read-only
  3. Inside the container, 7z lists archive contents in structured format
  4. Each entry's path is validated: entries with path traversal (..), absolute paths, backslashes, control characters, Unicode bidirectional overrides, zero-width characters, trailing whitespace, or excessively long names are rejected and logged
  5. Matching files are checked against resource caps (file count, per-file size, total size)
  6. Matching files are extracted to tmpfs with a wall-clock timeout (preserving directory structure)
  7. Execute permissions are removed from all extracted files
  8. Non-regular files (symlinks, FIFOs) are removed
  9. A manifest is generated with each file's SHA-256 hash, size, detected MIME type, and path
  10. Files are either packed into a ZIP or copied to a directory, depending on --output
  11. When the input is a directory, files from each archive are prefixed with the archive name to avoid collisions

Nested Archives

When an archive contains entries with archive extensions (.zip, .iso, .7z, .rar, .cab, .wim, .tar, .gz, .bz2, .xz), the tool recursively extracts and scans them for suspicious files. Files from nested archives appear in the output under a path prefixed with the nested archive's name (e.g., Crack.rar/keygen.exe).

Recursion depth is bounded by ARCHIVE_SIFT_MAX_NEST_DEPTH (default 3). Archives exceeding the depth limit are logged as skipped. If a nested archive's name itself matches the suspicious-extension pattern (e.g., .jar), it is kept in the output alongside its extracted contents.

Manifest Format

The manifest is a tab-separated file with four columns per line:

<sha256>\t<size>\t<detected_type>\t<path>

The detected_type column contains the MIME type detected by libmagic (e.g., application/x-dosexec for PE executables, text/plain for text files). This allows the analyst to spot extension/content mismatches, such as a file named payload.dll that is actually plain text.

License

MIT

About

Extract suspicious files from large archives for malware analysis.

Topics

Resources

License

Stars

0 stars

Watchers

0 watching

Forks

Packages

 
 
 

Contributors