Pipeline for Nix

[chinyeungli]

Issues

• Closes: nixpkgs-clarity: Design and implement Scancode pipeline for single nixpkg #1938

Changes

This PR creates a dedicated pipeline for Nix packages.

Requirements

• This pipeline only works on packages that found in the official nixpkgs.
• It takes a single Nix PURL as an input (i.e. pkg:nix/nixpkgs/...)
• A 'version' or a 'commit' qualifier is required in the input PURL as this is needed to fetch the correct source and binary for deployment to development mapping scan.
• It also requires the input PURL to have a "system" qualifier to resolve system-specific binaries.
• Docker is required. The code use docker to build sources by applying patches/configs that's defined in the upstream .nix files.
• If the PURL has no output qualifier, the pipeline tries debug first (dwarf symbols are needed for D2D mapping) and falls back to out. If the PURL specifies an output, only that one is used.

Summary

• The code will get metadata from https://search.devbox.sh and find and fetch the binary package from https://cache.nixos.org/
• If a Nix package cannot be found in https://cache.nixos.org/ and a commit hash is provided, the code will try to fetch and build the source from https://github.com/NixOS/nixpkgs and patch the sources using the exact configuration defined in the upstream .nix files.
• The pipeline will check if there is any license mismatch between the codebase and the declared license from the detected packages

Notes for deployment to development mapping

As Nix is not tied to a single programming language ecosystem, this pipeline executes all currently supported D2D steps across multiple languages.

Handling Missing Debug Symbols:

For pkg:nix/nixpkgs/SDL_mixer@1.2.12?system=x86_64-linux , as seen in https://search.devbox.sh/v2/pkg?name=SDL_mixer , there is no debug build found under the "outputs" section. Therefore, even the ELF binary is not stripped:

there is no dwarf symbols collected and no development to deployment mapping can be performed

Template File Mapping (.in files):

For pkg:nix/nixpkgs/openssl@3.6.0?out=debug&system=x86_64-linux, following are some of the dwarf symbols collected:

include/openssl/asn1.h
include/openssl/asn1t.h

However, these exact header files do not exist in the source codebase; they are generated from GNU Autoconf template files. To complete the mapping, the code has been updated to map to the corresponding .in template files:

Source-Only Scans:
For pkg:nix/nixpkgs/haskellPackages.aasam@0.2.0.0?system=x86_64-linux&commit=a3ae4cdd64f675cf2580affecdee01c401b43638 , this package does not exist on https://search.devbox.sh/, so no binary can be fetched, and the local Nix build fails because the package declares dependencies incompatible with the GHC version in this nixpkgs commit. The pipeline still fetches the patched source tree and runs the license scan; only the D2D mapping is skipped.:

Design notes

• Custom Nix image: Built on first use with zstd, xz, bzip2, and gzip installed. Without this, nix-shell -p fetches the decompressors on every extraction, which hits the CDN's rate limit.
• Local nixpkgs cache: A bare git clone of nixpkgs lives on the host. Each scan shallow-fetches its commit and creates a temporary worktree, bind-mounted into the container read-only. Avoids GitHub rate limits.
• Concurrent scans: RQ runs each scan in its own worker process. File locks serialize image builds, git operations, and GC.
• Nix store GC: The nix-eval-cache volume is pruned with nix-collect-garbage --delete-older-than 12h, at most once every 3 hours, so disk usage stays bounded.

Notes

Because the pipeline relies on a Linux-based Docker container (nixos/nix) to build sources, providing a PURL for a non-Linux system (e.g., darwin) triggers a system barrier warning.
When this happens, the pipeline safely falls back to evaluating the source using the container's native Linux environment. The extracted source tree will contain Linux-specific patches rather than macOS patches. The impact on the d2d mapping is expected to be minimal, though a small number of files may remain unmapped due to missing OS-specific structural patches.

Checklist

• I have read the contributing guidelines
• I have linked an existing issue above
• I have added unit tests covering the new code
• I have reviewed and understood every line of this PR

[Shriprasad-P]

Review

PR: Pipeline for Nix

Touched: Dockerfile, docker-compose.yml, docs/built-in-pipelines.rst, pyproject.toml, scanpipe/pipelines/__init__.py, scanpipe/pipelines/scan_nix_package.py

• Addresses a concrete correctness/reliability issue based on the title and diff.
• Diff is fairly large (+1973/-22); a short summary of risk areas from the author would help reviewers.

Commenting as a drive-by reviewer after reading the diff. Happy to look again if maintainers want a deeper pass on a specific file.