cargo metadata tries to collect dev dependencies

[alexanderkjall]

I modified the Debian cargo buildsystem to use auditable (finally) But ran into an issue that is maybe a bit philosophical in how it should be solved.

This is the error message from a test run trying to compile lsd:

thread 'main' panicked at 'cargo metadata failure: error: no matching package named `assert_cmd` found
location searched: registry `crates-io`
required by package `lsd v0.23.1 (/<<PKGBUILDDIR>>)`
', src/collect_audit_data.rs:77:9
stack backtrace:
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
error: could not compile `lsd`

assert_cmd is a dev-dependencies to lsd, and the Debian build system doesn't inject those in the environment when running the cargo install command.

That is hard to change, as that would introduce a large number of dependency loops.

Is it important to collect dev-dependencies, or could cargo metadata be instructed to not look for those maybe (This I haven't researched yet)?

[Shnatsel]

I am not aware of any such mechanism. Perhaps there is some nightly config flag to keep going on failures? It's worth taking a look at, but I doubt there is one.

Since this is an issue in cargo metadata and not cargo auditable itself, unfortunately it is not something I can directly fix.

[Shnatsel]

We can finally support integration into Debian by using Cargo's native SBOM precursor feature in the latest git. A build command like this should work and not require any dev-dependencies to be present:

CARGO_BUILD_SBOM=true cargo +nightly auditable build -Z sbom --release

rust-lang/cargo#15695 may be a bit of a problem but should be easy enough to work around with a touch src/* if it ever comes up.

[Shnatsel]

The native SBOM support has now shipped in v0.7.0

[petrak-at-canonical]

Hello, are there any plans to make this work on non-Nightly cargo? Or any known workarounds?

[Shnatsel]

For now the only workaround to make this work on stable builds of Cargo is to make it think it's a nightly build:

__CARGO_TEST_CHANNEL_OVERRIDE_DO_NOT_USE_THIS=nightly CARGO_BUILD_SBOM=true cargo auditable build -Z sbom --release

This is similar to the RUSTC_BOOTSTRAP=1 environment variable that lets you use nightly Rust features on stable.

This is needed because cargo-auditable relies on Cargo's SBOM precursor to circumvent the limitations of cargo-metadata. The SBOM functionality in Cargo is nightly-only for now.

I cannot provide a timeframe for the stabilization of the SBOM functionality in Cargo. I would like to work on it and I'm currently seeking funding to move it forward, but the timeline ultimately depends on the Cargo team.

Please note that while the SBOM feature in Cargo is not stabilized, the format is unstable as well and may change between Cargo releases. But if you're working on something like a Linux distribution and can upgrade Cargo and cargo-auditable in lockstep, that shouldn't be a problem.

(For completeness, implementing this would also work and not depend on the Cargo SBOM precursor functionality, but it would also require nightly, so that's doesn't help you).

[Shnatsel]

Another workaround is generating dummy Cargo.toml files for dev-dependencies.

When the dev-dependencies aren't used in the build, all that cargo metadata needs is a Cargo.toml file in the right place. So if the package you're trying to build has something like this in its Cargo.toml:

[package]
name = "the_package_you_are_building"
version = "1.0"
# some fields omitted

[dev-dependencies]
some_dev_dependency = "0.2"

then you can generate a Cargo.toml file with these contents:

[package]
name = "some_dev_dependency"
version = "0.2"

put it in the same place where you've vendored the rest of the dependencies and that should be enough to satisfy cargo metadata.

If there's anything that's not clear, I'm happy to discuss this in more depth. It might be easier over something like Slack or Matrix.

[Shnatsel]

Or, an even simpler workaround: remove the [dev-dependencies] section of the Cargo.toml of the package you're building before the build. If you're not providing the dev-dependencies in the build environment, this is the neatest approach.

[petrak-at-canonical]

I'll go from here for now. If I do want to contact you, where can I get a hold of you? (Not sure which Matrix/Slack server to join.)

Thanks for the help!

[Shnatsel]

You can message me directly at @shnatsel:matrix.org

[Shnatsel]

After thinking about it some more, the best recipe for Debian and Ubuntu is:

RUSTC_BOOTSTRAP=1 CARGO_BUILD_SBOM=true cargo auditable build -Z sbom --release

Debian already sets RUSTC_BOOTSTRAP=1 which activates nightly features on stable to make -Z avoid-dev-deps work, and that's the only contentious part. In fact, this issue only exists due to the use of RUSTC_BOOTSTRAP=1 in the first place!