From 22326a189d9eedf9f703fd2f7262c9822e1b038e Mon Sep 17 00:00:00 2001 From: "randomizedcoder dave.seddon.ca@gmail.com" Date: Mon, 31 Aug 2026 09:48:11 -0700 Subject: [PATCH 1/2] Add modular Nix flake: dev shell, OCI images, static analysis Add a thin flake.nix orchestrator that delegates to a modular nix/ tree: flake-utils + a plain mkShell, one file per concern, and a single nix/versions.nix as the source of truth for tool versions. Provides: - nix develop: Python 3.12 dev shell with cmax on PATH (a shell function shadows it interactively to run the working tree) plus ruff, mypy, bandit, shellcheck, and helper commands (cmax-test, cmax-lint, ...). - nix build .#cmax: buildPythonApplication of the cmax CLI, with an install check for `cmax --version` and the bundled cmax.yaml / run.sh resources. - nix build .#oci-cmax: native per-system OCI image (amd64 + aarch64) via dockerTools.buildLayeredImage. - Report-only static analysis: analysis-ruff, -ruff-format, -mypy, -bandit, -shellcheck, and a combined `analysis` summary. These never gate. - nix flake check: sandbox-safe gates only (package build + nixfmt). - nix run .#test: pytest in the host environment (the suite's command stubs hard-code /bin/bash and /bin/cat, absent in the hermetic sandbox). pyproject.toml gains [tool.ruff], [tool.mypy], and [tool.bandit] config so the analysers run the same way inside and outside Nix. Docs: nix/README.md plus a "Develop with Nix" quickstart in README.md. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01TiU7ttVHcNojp2T1TEutgX --- README.md | 27 ++++++ flake.lock | 61 +++++++++++++ flake.nix | 71 +++++++++++++++ nix/README.md | 170 +++++++++++++++++++++++++++++++++++ nix/analysis/bandit.nix | 15 ++++ nix/analysis/default.nix | 47 ++++++++++ nix/analysis/mk-report.nix | 43 +++++++++ nix/analysis/mypy.nix | 18 ++++ nix/analysis/ruff-format.nix | 15 ++++ nix/analysis/ruff.nix | 15 ++++ nix/analysis/shellcheck.nix | 17 ++++ nix/checks/default.nix | 25 ++++++ nix/checks/nixfmt.nix | 17 ++++ nix/clustermax.nix | 62 +++++++++++++ nix/containers/default.nix | 20 +++++ nix/default.nix | 101 +++++++++++++++++++++ nix/devshell.nix | 55 ++++++++++++ nix/lib/mkOciImage.nix | 41 +++++++++ nix/overlays.nix | 8 ++ nix/packages.nix | 31 +++++++ nix/versions.nix | 34 +++++++ pyproject.toml | 24 +++++ 22 files changed, 917 insertions(+) create mode 100644 flake.lock create mode 100644 flake.nix create mode 100644 nix/README.md create mode 100644 nix/analysis/bandit.nix create mode 100644 nix/analysis/default.nix create mode 100644 nix/analysis/mk-report.nix create mode 100644 nix/analysis/mypy.nix create mode 100644 nix/analysis/ruff-format.nix create mode 100644 nix/analysis/ruff.nix create mode 100644 nix/analysis/shellcheck.nix create mode 100644 nix/checks/default.nix create mode 100644 nix/checks/nixfmt.nix create mode 100644 nix/clustermax.nix create mode 100644 nix/containers/default.nix create mode 100644 nix/default.nix create mode 100644 nix/devshell.nix create mode 100644 nix/lib/mkOciImage.nix create mode 100644 nix/overlays.nix create mode 100644 nix/packages.nix create mode 100644 nix/versions.nix diff --git a/README.md b/README.md index 5199aac..cd91f0f 100644 --- a/README.md +++ b/README.md @@ -316,6 +316,32 @@ evidence. Use `-vvv` to include recommended remediation and labeled reference URLs. Use `--command raw` to print the saved collector log. Use `--no-interactive` to print the review and exit. +## Develop with Nix + +The repository ships a [Nix](https://nixos.org) flake for a reproducible +development environment, a hermetic build of `cmax`, OCI container images, and +report-only static analysers. It is optional: it does not change the pip install +above. If you are new to Nix, [`nix/README.md`](nix/README.md) has a short +introduction, install steps, and video walkthroughs. + +Quickstart, from the repo root (flakes must be enabled — see +[`nix/README.md`](nix/README.md)): + +``` +nix develop # dev shell (Python, ruff, mypy, bandit, shellcheck); type 'cmax-help' +nix build .#cmax # build the CLI -> ./result/bin/cmax +nix run .#test # run the pytest suite +nix build .#analysis # run all static analysers; cat result/summary.txt +nix build .#oci-cmax # OCI image for the host arch; docker load < result +nix flake check # package build + CLI smoke check + nix formatting +``` + +The full target list is in the header comment of `flake.nix` and in +[`nix/README.md`](nix/README.md). + +> Flakes only see git-tracked files. After adding or editing files under `nix/`, +> `git add` them before `nix build` / `nix develop`. + ## Repository contents | Path | Contents | @@ -323,6 +349,7 @@ URLs. Use `--command raw` to print the saved collector log. Use | `cmax/` | This directory contains the command code and `cmax.yaml` configuration. | | `cmax/scripts/1-audit/` | This directory contains the scripts that run an audit. | | `tests/audit/` | This directory contains all audit tests, fixtures, and test helpers. | +| `flake.nix`, `nix/` | The Nix flake and its modules. See [`nix/README.md`](nix/README.md). | This release excludes provider results, internal notes, the private dashboard, benchmark implementations, bundled data, and database code. diff --git a/flake.lock b/flake.lock new file mode 100644 index 0000000..5998912 --- /dev/null +++ b/flake.lock @@ -0,0 +1,61 @@ +{ + "nodes": { + "flake-utils": { + "inputs": { + "systems": "systems" + }, + "locked": { + "lastModified": 1731533236, + "narHash": "sha256-l0KFg5HjrsfsO/JpG+r7fRrqm12kzFHyUHqHCVpMMbI=", + "owner": "numtide", + "repo": "flake-utils", + "rev": "11707dc2f618dd54ca8739b309ec4fc024de578b", + "type": "github" + }, + "original": { + "owner": "numtide", + "repo": "flake-utils", + "type": "github" + } + }, + "nixpkgs": { + "locked": { + "lastModified": 1788039129, + "narHash": "sha256-pa4Q0qErvCvzCaaUph7Sm37RhR4xvPrYI8Lgz6k85+A=", + "owner": "NixOS", + "repo": "nixpkgs", + "rev": "d2f67949798825fe853f7c5d0492b8bf016d3f88", + "type": "github" + }, + "original": { + "owner": "NixOS", + "ref": "nixos-unstable", + "repo": "nixpkgs", + "type": "github" + } + }, + "root": { + "inputs": { + "flake-utils": "flake-utils", + "nixpkgs": "nixpkgs" + } + }, + "systems": { + "locked": { + "lastModified": 1681028828, + "narHash": "sha256-Vy1rq5AaRuLzOxct8nz4T6wlgyUR7zLU309k9mBC768=", + "owner": "nix-systems", + "repo": "default", + "rev": "da67096a3b9bf56a91d16901293e51ba5b49a27e", + "type": "github" + }, + "original": { + "owner": "nix-systems", + "repo": "default", + "type": "github" + } + } + }, + "root": "root", + "version": 7 +} diff --git a/flake.nix b/flake.nix new file mode 100644 index 0000000..30e993d --- /dev/null +++ b/flake.nix @@ -0,0 +1,71 @@ +# +# flake.nix — ClusterMAX +# +# Thin orchestrator. Every concern lives under ./nix/ and is wired up here. +# See ./nix/default.nix for the per-system aggregator. +# +# Targets: +# nix develop # dev shell (ruff, mypy, bandit, shellcheck, pytest) +# nix build .#cmax # build the cmax CLI +# nix run .#cmax -- --help # run the CLI +# nix build .#oci-cmax # OCI image for the host arch (amd64 or aarch64) +# nix build .#analysis # run ALL static-analysis reports (report-only) +# nix build .#analysis-ruff # ruff lint report +# nix build .#analysis-ruff-format # ruff format --check report +# nix build .#analysis-mypy # mypy type report +# nix build .#analysis-bandit # bandit security report +# nix build .#analysis-shellcheck # shellcheck report for the audit .sh scripts +# nix run .#test # run the pytest suite (uses the host toolchain) +# nix flake check # package build + CLI smoke + nix formatting (gates) +# nix fmt # format the .nix files +# +# Static analysis is report-only: the analysis-* targets always succeed and write +# their findings to $out/report.txt. +# +# The pytest suite runs via `nix run .#test`, not `nix flake check`: its command +# stubs hard-code /bin/bash and /bin/cat, which do not exist in the hermetic Nix +# build sandbox. The gates that DO run under `nix flake check` are the package +# build (its installCheck smoke-tests the CLI + resources) and nix formatting. +# +# Containers are native per-system: build .#oci-cmax on an x86_64 host for the +# amd64 image, and on an aarch64 host (or through binfmt/qemu) for the arm64 image. +# +{ + description = "ClusterMAX — GPU cluster audit and security CLI"; + + inputs = { + nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable"; + flake-utils.url = "github:numtide/flake-utils"; + }; + + outputs = + { + self, + nixpkgs, + flake-utils, + }: + flake-utils.lib.eachSystem [ "x86_64-linux" "aarch64-linux" ] ( + system: + let + pkgs = import nixpkgs { inherit system; }; + lib = nixpkgs.lib; + + aggregator = import ./nix { + inherit pkgs lib; + src = ./.; + }; + in + { + inherit (aggregator) + packages + devShells + checks + apps + formatter + ; + } + ) + // { + overlays.default = import ./nix/overlays.nix { inherit self; }; + }; +} diff --git a/nix/README.md b/nix/README.md new file mode 100644 index 0000000..7b244c9 --- /dev/null +++ b/nix/README.md @@ -0,0 +1,170 @@ +# Nix flake: dev shell, package, containers, and analysis + +This flake gives you a reproducible development environment, a hermetic build +of the `cmax` CLI, OCI container images, and a set of report-only static +analysers — pinned so you get the same tools and results on any machine. The +design is modular: a slim top-level `flake.nix` wires together small +single-purpose modules under `nix/`. + +## New to Nix? + +[Nix](https://nixos.org) is a package manager that builds software in isolation +from pinned inputs. In practice that means `nix develop` drops you into a shell +with the exact Python and tools this project needs — nothing installed +system-wide, nothing to conflict with your distro — and `nix build` produces the +same result on any machine. + +**Install Nix** (multi-user, recommended): + +``` +sh <(curl -L https://nixos.org/nix/install) --daemon +``` + +Single-user (no root, e.g. in a container): use `--no-daemon` instead. Full +instructions: . + +**Enable flakes** (once). Either add this line to `/etc/nix/nix.conf` (or +`~/.config/nix/nix.conf`): + +``` +experimental-features = nix-command flakes +``` + +…or prefix each command with +`--extra-experimental-features 'nix-command flakes'`. + +**Two commands to get started**, from the repo root: + +``` +nix develop # enter the dev shell; type 'cmax-help' +nix build .#cmax # build the CLI -> ./result/bin/cmax +``` + +Video walkthroughs of the install: +[Ubuntu](https://youtu.be/cb7BBZLhuUY) · +[Fedora](https://youtu.be/RvaTxMa4IiY). Handy references: the +[flakes wiki](https://nixos.wiki/wiki/flakes) and +[search.nixos.org](https://search.nixos.org) to find any package. + +> Flakes only see git-tracked files. After adding or editing files under +> `nix/`, `git add` them before `nix build` / `nix develop`. + +## Dev shell + +``` +nix develop # dev shell (Python 3.12, ruff, mypy, bandit, shellcheck) +``` + +Type `cmax-help` for the menu. The shell defines helper functions that call the +same tool binaries the flake uses, so the shell and CI cannot drift: + +| Command | Action | +|---|---| +| `cmax [args]` | Run the CLI from the working tree (your edits apply at once). | +| `cmax-test [paths]` | Run pytest (default: `tests`). | +| `cmax-lint` | `ruff check cmax` | +| `cmax-fmt` | `ruff format cmax` (rewrites files) | +| `cmax-fmt-check` | `ruff format --check cmax` | +| `cmax-types` | `mypy cmax` | +| `cmax-sec` | `bandit -r cmax` | +| `cmax-shellcheck` | shellcheck the audit `.sh` scripts | +| `cmax-analysis` | run every analyser (report-only) | + +`cmax` is also on `PATH` as the built program, so `nix develop -c cmax ...` works +too; the shell function shadows it interactively to run your working tree. + +## Build & run + +``` +nix build .#cmax # -> ./result/bin/cmax +./result/bin/cmax --version +nix run .#cmax -- --help # build and run in one step +``` + +The build runs an install check: it confirms `cmax --version` works and that the +bundled resources (`cmax.yaml`, `scripts/1-audit/run.sh`) shipped in the wheel. + +## Static analysis (report-only) + +Each analyser is a build target. The targets **always succeed** and write their +findings to `$out/report.txt` — they generate reports, they do not gate. + +``` +nix build .#analysis # run all analysers; writes result/summary.txt +cat result/summary.txt + +nix build .#analysis-ruff # ruff lint +nix build .#analysis-ruff-format # ruff format --check +nix build .#analysis-mypy # mypy (non-strict) +nix build .#analysis-bandit # bandit security scan +nix build .#analysis-shellcheck # shellcheck for the audit .sh scripts +``` + +Each per-tool result holds `report.txt` (the findings), `exit-code.txt` (the +tool's real exit status), and `count.txt` (a rough finding-line count). Tool +configuration lives in the repo's `pyproject.toml` (`[tool.ruff]`, +`[tool.mypy]`, `[tool.bandit]`), so the same rules apply inside and outside Nix. + +## Tests + +``` +nix run .#test # run the pytest suite in the host environment +nix run .#test -- tests/audit/test_security.py # a subset +``` + +The suite runs through `nix run .#test`, **not** `nix flake check`. Its command +stubs write helper scripts that hard-code `/bin/bash` and `/bin/cat`; those paths +exist on a real host but not inside Nix's hermetic build sandbox, so the tests +must run in the host. Inside the dev shell, `cmax-test` does the same thing. + +## Container images (OCI) + +``` +nix build .#oci-cmax # OCI image for the host architecture +docker load < result # -> loads cmax:latest +docker run --rm cmax:latest --version +``` + +The image is built with `dockerTools.buildLayeredImage` over the `cmax` package +plus `bash` and `coreutils` (the audit scripts are shell scripts that call +`python3`; cluster tools such as `kubectl`/`srun` are provided by the +environment, not baked in). The entrypoint is `/bin/cmax`, and the image +timestamp is fixed for reproducibility. + +**amd64 and aarch64.** Images are native per-system: build `.#oci-cmax` on an +x86_64 host for the amd64 image and on an aarch64 host for the arm64 image. On a +single host you can build the other architecture through binfmt/QEMU emulation. +The package and image derivations evaluate for both `x86_64-linux` and +`aarch64-linux`. + +## Gates: `nix flake check` + +``` +nix flake check # package build + CLI smoke check + nix formatting +nix fmt # format the .nix files +``` + +Only sandbox-safe gates run here: the package build (whose install check +smoke-tests the CLI and the bundled resources) and a `nixfmt` formatting check. +Static analysis is report-only (above) and the test suite runs in the host +(above), so neither gates this command. + +## Module layout + +| Path | Role | +|---|---| +| `flake.nix` | Thin orchestrator; the target list lives in its header comment. | +| `nix/default.nix` | Per-system aggregator: assembles packages, devShells, checks, apps. | +| `nix/versions.nix` | Single source of truth for the Python interpreter and tool versions. | +| `nix/packages.nix` | Dependency lists fed to the dev shell. | +| `nix/clustermax.nix` | The `cmax` package (`buildPythonApplication`). | +| `nix/devshell.nix` | The `nix develop` environment and its helper functions. | +| `nix/lib/mkOciImage.nix` | OCI image factory. | +| `nix/containers/` | The `oci-cmax` image for the current system. | +| `nix/analysis/` | Report-only analysers (`ruff`, `mypy`, `bandit`, `shellcheck`) + combined report. | +| `nix/checks/` | The `nix flake check` gates (package build + nixfmt). | +| `nix/overlays.nix` | Exposes `cmax` and `cmax-oci` to downstream flakes. | + +Every module is a function that takes an explicit attribute set and returns a +derivation (or a set of them). Tool versions come from `nix/versions.nix` alone, +so there is one place to change them. diff --git a/nix/analysis/bandit.nix b/nix/analysis/bandit.nix new file mode 100644 index 0000000..49a2b9b --- /dev/null +++ b/nix/analysis/bandit.nix @@ -0,0 +1,15 @@ +# +# nix/analysis/bandit.nix — bandit security report (report-only). +# +{ + pkgs, + versions, + mkReport, +}: +mkReport { + name = "bandit"; + nativeBuildInputs = [ versions.bandit ]; + text = '' + bandit -r cmax + ''; +} diff --git a/nix/analysis/default.nix b/nix/analysis/default.nix new file mode 100644 index 0000000..6daafff --- /dev/null +++ b/nix/analysis/default.nix @@ -0,0 +1,47 @@ +# +# nix/analysis/default.nix — report-only static-analysis aggregator. +# +# Report-only model: analyzers are exposed as *packages*, never as +# `nix flake check` gates. Each per-tool package writes $out/report.txt; the +# combined `analysis` package gathers them under one tree with a summary.txt. +# +{ + pkgs, + lib, + versions, + src, +}: +let + mkReport = import ./mk-report.nix { inherit pkgs lib src; }; + + tools = { + "analysis-ruff" = import ./ruff.nix { inherit pkgs versions mkReport; }; + "analysis-ruff-format" = import ./ruff-format.nix { inherit pkgs versions mkReport; }; + "analysis-mypy" = import ./mypy.nix { inherit pkgs versions mkReport; }; + "analysis-bandit" = import ./bandit.nix { inherit pkgs versions mkReport; }; + "analysis-shellcheck" = import ./shellcheck.nix { inherit pkgs versions mkReport; }; + }; + + # Combined report: one subdirectory per tool plus a top-level summary. + combined = pkgs.runCommand "clustermax-analysis" { } ( + '' + mkdir -p "$out" + summary="$out/summary.txt" + echo "ClusterMAX static-analysis summary (report-only)" >"$summary" + echo "" >>"$summary" + '' + + lib.concatStringsSep "\n" ( + lib.mapAttrsToList (name: drv: '' + cp -r ${drv} "$out/${name}" + printf '%-22s exit=%s lines=%s\n' \ + "${name}" "$(cat ${drv}/exit-code.txt)" "$(cat ${drv}/count.txt)" >>"$summary" + '') tools + ) + + '' + + echo "" >>"$summary" + cat "$summary" + '' + ); +in +tools // { analysis = combined; } diff --git a/nix/analysis/mk-report.nix b/nix/analysis/mk-report.nix new file mode 100644 index 0000000..3b741cc --- /dev/null +++ b/nix/analysis/mk-report.nix @@ -0,0 +1,43 @@ +# +# nix/analysis/mk-report.nix — uniform report-only analysis runner. +# +# Copies the source into a writable tree, runs one tool, and tees everything to +# $out/report.txt. It NEVER fails the build: static analysis here produces +# reports, it does not gate. $out/exit-code.txt records the tool's real exit +# status and $out/count.txt a crude finding-line count. +# +{ + pkgs, + lib, + src, +}: +{ + name, + nativeBuildInputs ? [ ], + # Shell snippet that runs the tool. cwd is a writable copy of the repo. + text, +}: +pkgs.runCommand "clustermax-analysis-${name}" + { + inherit nativeBuildInputs; + passthru.reportName = name; + } + '' + cp -r ${src} ./source + chmod -R +w ./source + cd ./source + + mkdir -p "$out" + report="$out/report.txt" + + set +e + { + ${text} + } >"$report" 2>&1 + status=$? + set -e + + echo "$status" >"$out/exit-code.txt" + grep -c . "$report" >"$out/count.txt" 2>/dev/null || echo 0 >"$out/count.txt" + echo "clustermax analysis ${name}: tool exit=$status, $(cat "$out/count.txt") report lines" + '' diff --git a/nix/analysis/mypy.nix b/nix/analysis/mypy.nix new file mode 100644 index 0000000..90da39c --- /dev/null +++ b/nix/analysis/mypy.nix @@ -0,0 +1,18 @@ +# +# nix/analysis/mypy.nix — mypy type report (report-only, non-strict). +# +# The codebase has no type annotations yet; --ignore-missing-imports keeps the +# report focused on real errors rather than un-annotated third-party stubs. +# +{ + pkgs, + versions, + mkReport, +}: +mkReport { + name = "mypy"; + nativeBuildInputs = [ versions.mypy ]; + text = '' + mypy --ignore-missing-imports --no-error-summary cmax + ''; +} diff --git a/nix/analysis/ruff-format.nix b/nix/analysis/ruff-format.nix new file mode 100644 index 0000000..c9441bc --- /dev/null +++ b/nix/analysis/ruff-format.nix @@ -0,0 +1,15 @@ +# +# nix/analysis/ruff-format.nix — ruff format --check report (report-only). +# +{ + pkgs, + versions, + mkReport, +}: +mkReport { + name = "ruff-format"; + nativeBuildInputs = [ versions.ruff ]; + text = '' + ruff format --check --diff cmax + ''; +} diff --git a/nix/analysis/ruff.nix b/nix/analysis/ruff.nix new file mode 100644 index 0000000..233341c --- /dev/null +++ b/nix/analysis/ruff.nix @@ -0,0 +1,15 @@ +# +# nix/analysis/ruff.nix — ruff lint report (report-only). +# +{ + pkgs, + versions, + mkReport, +}: +mkReport { + name = "ruff"; + nativeBuildInputs = [ versions.ruff ]; + text = '' + ruff check --output-format=full cmax + ''; +} diff --git a/nix/analysis/shellcheck.nix b/nix/analysis/shellcheck.nix new file mode 100644 index 0000000..5624c17 --- /dev/null +++ b/nix/analysis/shellcheck.nix @@ -0,0 +1,17 @@ +# +# nix/analysis/shellcheck.nix — shellcheck report for the audit .sh scripts. +# +# -x follows `source`d files (e.g. audit-common.sh). Report-only. +# +{ + pkgs, + versions, + mkReport, +}: +mkReport { + name = "shellcheck"; + nativeBuildInputs = [ versions.shellcheck ]; + text = '' + find cmax/scripts -name '*.sh' -print0 | sort -z | xargs -0 shellcheck -x + ''; +} diff --git a/nix/checks/default.nix b/nix/checks/default.nix new file mode 100644 index 0000000..227b022 --- /dev/null +++ b/nix/checks/default.nix @@ -0,0 +1,25 @@ +# +# nix/checks/default.nix — the `nix flake check` gates. +# +# These must run inside Nix's hermetic build sandbox, so only sandbox-safe gates +# live here: nix formatting, plus the package build (its installCheck smoke-tests +# the CLI and asserts the bundled resources shipped — see ../clustermax.nix). +# +# The pytest suite is NOT a check: it stubs external tools with scripts that +# hard-code /bin/bash and /bin/cat, which do not exist in the sandbox. It runs in +# the host environment instead, via `nix run .#test` or `cmax-test` in the shell. +# +{ + pkgs, + lib, + versions, + cmax, + src, +}: +{ + # Building the package runs its installCheckPhase: `cmax --version` plus the + # cmax.yaml / scripts/1-audit/run.sh resource assertions. + cmax = cmax; + + nixfmt = import ./nixfmt.nix { inherit pkgs versions src; }; +} diff --git a/nix/checks/nixfmt.nix b/nix/checks/nixfmt.nix new file mode 100644 index 0000000..bcab136 --- /dev/null +++ b/nix/checks/nixfmt.nix @@ -0,0 +1,17 @@ +# +# nix/checks/nixfmt.nix — assert every .nix file is nixfmt-formatted (a gate). +# +{ + pkgs, + versions, + src, +}: +pkgs.runCommand "clustermax-nixfmt-check" + { + nativeBuildInputs = [ versions.nixfmt ]; + } + '' + cd ${src} + find . -name '*.nix' -print0 | xargs -0 nixfmt --check + touch "$out" + '' diff --git a/nix/clustermax.nix b/nix/clustermax.nix new file mode 100644 index 0000000..56ac51b --- /dev/null +++ b/nix/clustermax.nix @@ -0,0 +1,62 @@ +# +# nix/clustermax.nix — the cmax CLI package. +# +# buildPythonApplication over the repo's pyproject (setuptools backend). The +# dynamic version resolves through setup.py; with no CLUSTERMAX_BUILD_VERSION set +# it falls back to cmax/_version.py (0.2.1), which setup.py rewrites into the wheel. +# +{ + pkgs, + lib, + versions, + src, +}: +let + inherit (versions) python; +in +python.pkgs.buildPythonApplication { + pname = "clustermax"; + version = "0.2.1"; + pyproject = true; + + inherit src; + + build-system = [ versions.setuptools ]; + + # pyproject pins `requires = ["setuptools==84.0.0"]`; nixpkgs ships a nearby + # setuptools that the backend builds fine with. Skip the frontend's exact-pin + # check (build isolation is already off) rather than chase the pinned version. + pypaBuildFlags = [ "--skip-dependency-check" ]; + + dependencies = versions.runtimeDeps python.pkgs; + + # Tests run as a dedicated flake check (see nix/checks/pytest.nix), not here. + doCheck = false; + + pythonImportsCheck = [ "cmax" ]; + + # Assert the CLI runs and that the bundled resources shipped in the wheel — + # the same invariants the CI wheel smoke-test guards. + doInstallCheck = true; + installCheckPhase = '' + runHook preInstallCheck + + echo "checking cmax --version" + "$out/bin/cmax" --version + + siteDir=$(echo "$out/lib/"python*"/site-packages") + test -f "$siteDir/cmax/cmax.yaml" \ + || (echo "missing cmax/cmax.yaml in $siteDir" && exit 1) + test -f "$siteDir/cmax/scripts/1-audit/run.sh" \ + || (echo "missing scripts/1-audit/run.sh in $siteDir" && exit 1) + + runHook postInstallCheck + ''; + + meta = { + description = "ClusterMAX GPU cluster audit and security CLI"; + homepage = "https://clustermax.semianalysis.com/"; + license = lib.licenses.asl20; + mainProgram = "cmax"; + }; +} diff --git a/nix/containers/default.nix b/nix/containers/default.nix new file mode 100644 index 0000000..d7bc624 --- /dev/null +++ b/nix/containers/default.nix @@ -0,0 +1,20 @@ +# +# nix/containers/default.nix — OCI images for the current system. +# +# Native per-system: `nix build .#oci-cmax` produces the amd64 image on an +# x86_64 host and the arm64 image on an aarch64 host. +# +{ + pkgs, + lib, + cmax, +}: +let + mkOciImage = import ../lib/mkOciImage.nix { inherit pkgs lib; }; +in +{ + oci = mkOciImage { + name = "cmax"; + inherit cmax; + }; +} diff --git a/nix/default.nix b/nix/default.nix new file mode 100644 index 0000000..61e2c6e --- /dev/null +++ b/nix/default.nix @@ -0,0 +1,101 @@ +# +# nix/default.nix — per-system aggregator. +# +# Imports every sub-module and assembles the flake outputs for one system: +# packages, devShells, checks, apps, and the formatter. flake.nix re-exports +# these under flake-utils.lib.eachSystem. +# +{ + pkgs, + lib, + src, +}: +let + versions = import ./versions.nix { inherit pkgs; }; + packages = import ./packages.nix { inherit pkgs versions; }; + + cmax = import ./clustermax.nix { + inherit + pkgs + lib + versions + src + ; + }; + containers = import ./containers { inherit pkgs lib cmax; }; + analysis = import ./analysis { + inherit + pkgs + lib + versions + src + ; + }; + + devshell = import ./devshell.nix { + inherit + pkgs + lib + packages + cmax + ; + }; + checks = import ./checks { + inherit + pkgs + lib + versions + cmax + src + ; + }; + + # The test suite runs in the host environment (not the hermetic sandbox), + # because its command stubs hard-code /bin/bash and /bin/cat. + testApp = pkgs.writeShellApplication { + name = "clustermax-test"; + runtimeInputs = [ versions.pythonEnv ]; + text = '' + python -m pytest -q "''${@:-tests}" + ''; + }; +in +{ + packages = { + default = cmax; + inherit cmax; + oci-cmax = containers.oci; + } + // analysis; # analysis-ruff, analysis-mypy, ..., and combined `analysis` + + devShells.default = devshell; + + checks = checks; + + apps = { + cmax = { + type = "app"; + program = "${lib.getExe cmax}"; + meta.description = "Run the cmax CLI"; + }; + analysis = { + type = "app"; + program = "${lib.getExe ( + pkgs.writeShellApplication { + name = "clustermax-analysis"; + text = '' + cat ${analysis.analysis}/summary.txt + ''; + } + )}"; + meta.description = "Print the combined static-analysis summary"; + }; + test = { + type = "app"; + program = "${lib.getExe testApp}"; + meta.description = "Run the pytest suite in the host environment"; + }; + }; + + formatter = versions.nixfmt; +} diff --git a/nix/devshell.nix b/nix/devshell.nix new file mode 100644 index 0000000..952bc86 --- /dev/null +++ b/nix/devshell.nix @@ -0,0 +1,55 @@ +# +# nix/devshell.nix — the `nix develop` environment. +# +# Plain pkgs.mkShell. The shellHook prints a menu and defines helper functions +# that call the SAME tool binaries the flake checks and analysis reports use, so +# the shell and CI cannot drift. +# +{ + pkgs, + lib, + packages, + cmax, +}: +pkgs.mkShell { + name = "clustermax-dev"; + # The built cmax is on PATH so `cmax` works in any invocation (incl. + # `nix develop -c cmax ...`); the shell function below shadows it + # interactively to run the working tree instead. + packages = packages.allDevPackages ++ [ cmax ]; + + shellHook = '' + # Interactively, run the working-tree source so `cmax` reflects local edits. + cmax() { python3 -m cmax.cli "$@" ; } + cmax-test() { python3 -m pytest -q "''${@:-tests}" ; } + cmax-lint() { ruff check cmax ; } + cmax-fmt() { ruff format cmax ; } + cmax-fmt-check() { ruff format --check cmax ; } + cmax-types() { mypy cmax ; } + cmax-sec() { bandit -r cmax ; } + cmax-shellcheck() { find cmax/scripts -name '*.sh' -print0 | xargs -0 shellcheck ; } + cmax-analysis() { + echo "== ruff ==" ; cmax-lint || true + echo "== ruff format ==" ; cmax-fmt-check || true + echo "== mypy ==" ; cmax-types || true + echo "== bandit ==" ; cmax-sec || true + echo "== shellcheck ==" ; cmax-shellcheck || true + } + cmax-help() { + cat <<'EOF' + ClusterMAX dev shell + cmax [args] run the CLI from the working tree (live edits) + cmax-test [paths] run pytest (default: tests) + cmax-lint ruff check cmax + cmax-fmt ruff format cmax (rewrites files) + cmax-fmt-check ruff format --check cmax + cmax-types mypy cmax + cmax-sec bandit -r cmax + cmax-shellcheck shellcheck the audit .sh scripts + cmax-analysis run every analyzer (report-only) + EOF + } + + cmax-help + ''; +} diff --git a/nix/lib/mkOciImage.nix b/nix/lib/mkOciImage.nix new file mode 100644 index 0000000..6221a37 --- /dev/null +++ b/nix/lib/mkOciImage.nix @@ -0,0 +1,41 @@ +# +# nix/lib/mkOciImage.nix — OCI image factory. +# +# Wraps dockerTools.buildLayeredImage over the cmax package plus bash/coreutils +# (the audit scripts are .sh files that shell out to python3; external cluster +# tools like kubectl/slurm are environment-provided, not baked in). +# +# The image architecture follows pkgs.stdenv.hostPlatform, so building on an +# x86_64 host yields the amd64 image and building on aarch64 yields the arm64 +# image — native per-system, no cross plumbing. +# +{ pkgs, lib }: +{ + name, + cmax, + tag ? "latest", +}: +pkgs.dockerTools.buildLayeredImage { + inherit name tag; + + # Reproducible: a fixed epoch instead of "now". + created = "1970-01-01T00:00:00Z"; + + contents = [ + cmax + pkgs.bashInteractive + pkgs.coreutils + pkgs.dockerTools.caCertificates + ]; + + config = { + Entrypoint = [ "/bin/cmax" ]; + Env = [ "SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt" ]; + Labels = { + "org.opencontainers.image.title" = "clustermax"; + "org.opencontainers.image.description" = "ClusterMAX GPU cluster audit and security CLI"; + "org.opencontainers.image.licenses" = "Apache-2.0"; + "org.opencontainers.image.source" = "https://github.com/SemiAnalysisAI/ClusterMAX"; + }; + }; +} diff --git a/nix/overlays.nix b/nix/overlays.nix new file mode 100644 index 0000000..9396b11 --- /dev/null +++ b/nix/overlays.nix @@ -0,0 +1,8 @@ +# +# nix/overlays.nix — expose the package and image to downstream flakes. +# +{ self }: +final: _prev: { + cmax = self.packages.${final.system}.cmax or null; + cmax-oci = self.packages.${final.system}.oci-cmax or null; +} diff --git a/nix/packages.nix b/nix/packages.nix new file mode 100644 index 0000000..b23c3d2 --- /dev/null +++ b/nix/packages.nix @@ -0,0 +1,31 @@ +# +# nix/packages.nix — dependency lists fed to the dev shell. +# +# Splits packages by role so the dev shell can pull them all in one place while +# each concern (checks, containers) imports only what it needs from versions.nix. +# +{ pkgs, versions }: +let + # The Python interpreter plus runtime deps and pytest, importable as `cmax`. + inherit (versions) pythonEnv; + + # Static-analysis tools exposed both as report packages and in the shell. + analysisTools = [ + versions.ruff + versions.mypy + versions.bandit + versions.shellcheck + ]; + + # General development conveniences. + devTools = [ + versions.nixfmt + pkgs.git + ]; +in +{ + inherit analysisTools devTools; + + # Everything the `nix develop` shell should put on PATH. + allDevPackages = [ pythonEnv ] ++ analysisTools ++ devTools; +} diff --git a/nix/versions.nix b/nix/versions.nix new file mode 100644 index 0000000..9a83703 --- /dev/null +++ b/nix/versions.nix @@ -0,0 +1,34 @@ +# +# nix/versions.nix — single source of truth for tool and dependency versions. +# +# Every other module imports this so shell, checks, package, and container all +# agree on the same Python interpreter and tool set — no drift. +# +{ pkgs }: +let + # CI runs on Python 3.12; pin to match (pyproject requires >= 3.10). + python = pkgs.python312; + + # Runtime dependencies of the cmax package (pyproject `dependencies`). + runtimeDeps = ps: [ + ps.prompt-toolkit + ps.pyyaml + ]; + + # A Python environment that can import cmax and run the test suite. + # Used by the pytest check and the dev shell. + pythonEnv = python.withPackages (ps: (runtimeDeps ps) ++ [ ps.pytest ]); +in +{ + inherit python runtimeDeps pythonEnv; + + # Build-time tools for producing the wheel / package. + setuptools = python.pkgs.setuptools; + + # Static-analysis tools (report-only) and the Nix formatter. + ruff = pkgs.ruff; + mypy = pkgs.mypy; + bandit = pkgs.bandit; + shellcheck = pkgs.shellcheck; + nixfmt = pkgs.nixfmt; +} diff --git a/pyproject.toml b/pyproject.toml index 3dee9fd..47a343c 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -43,3 +43,27 @@ cmax = [ [tool.check-wheel-contents] # The audit runs these Python files as package resources through their paths. ignore = ["W004"] + +# --- Static analysis (report-only; wired into the Nix flake analysis-* targets) --- +# These configs make the analyzers usable both inside `nix build .#analysis` and +# directly (ruff/mypy/bandit) from a plain checkout. + +[tool.ruff] +target-version = "py310" +line-length = 100 +extend-exclude = ["tests/audit/fixtures"] + +[tool.ruff.lint] +# A sensible default rule set. Reports only — nothing gates on these. +select = ["E", "F", "W", "I", "UP", "B", "C4", "SIM"] + +[tool.mypy] +python_version = "3.10" +ignore_missing_imports = true +# Non-strict: the codebase is not annotated yet, so surface only real errors. +warn_unused_ignores = true +warn_redundant_casts = true + +[tool.bandit] +# Test fixtures and the suite are not shipped code. +exclude_dirs = ["tests"] From d946ce23adada89bb58673884676607be78291f4 Mon Sep 17 00:00:00 2001 From: "randomizedcoder dave.seddon.ca@gmail.com" Date: Mon, 31 Aug 2026 10:00:16 -0700 Subject: [PATCH 2/2] Add prioritized static-analysis review Run the flake's report-only analysers (ruff 0.16.4, mypy 2.1.0, bandit 1.9.4, shellcheck 0.11.0) over the codebase and triage the results. - STATIC_ANALYSIS.md: findings grouped by priority (P1 security -> P2 correctness -> P3 maintainability -> P4 style, plus shell), with counts, representative file:line examples, and a recommended fix order. Notably: 3 bandit findings to act on (2 XML parsing + 1 urlopen scheme); the ~28 subprocess findings and 5 hardcoded-password false positives are documented as accepted; 9 ruff bug-class + 75 mypy type findings for correctness; 157 E501 + 11-file ruff-format diff as mechanical style work. - reports/static-analysis/: the verbatim tool outputs for reference. - .gitattributes: exempt the captured reports from whitespace checks. No code under cmax/ is changed; this is a review, not a fix. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01TiU7ttVHcNojp2T1TEutgX --- .gitattributes | 3 + STATIC_ANALYSIS.md | 185 ++ reports/static-analysis/bandit.txt | 543 ++++ reports/static-analysis/mypy.txt | 81 + reports/static-analysis/ruff-format.txt | 3955 +++++++++++++++++++++++ reports/static-analysis/ruff.txt | 2739 ++++++++++++++++ reports/static-analysis/shellcheck.txt | 1435 ++++++++ reports/static-analysis/summary.txt | 8 + 8 files changed, 8949 insertions(+) create mode 100644 .gitattributes create mode 100644 STATIC_ANALYSIS.md create mode 100644 reports/static-analysis/bandit.txt create mode 100644 reports/static-analysis/mypy.txt create mode 100644 reports/static-analysis/ruff-format.txt create mode 100644 reports/static-analysis/ruff.txt create mode 100644 reports/static-analysis/shellcheck.txt create mode 100644 reports/static-analysis/summary.txt diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..8f5bf35 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,3 @@ +# Captured static-analysis reports are verbatim tool output. Keep them exactly +# as produced (aligned columns, diff blank lines) — exempt from whitespace checks. +reports/static-analysis/*.txt -whitespace diff --git a/STATIC_ANALYSIS.md b/STATIC_ANALYSIS.md new file mode 100644 index 0000000..baf8516 --- /dev/null +++ b/STATIC_ANALYSIS.md @@ -0,0 +1,185 @@ +# ClusterMAX static-analysis review + +First pass over the `cmax` codebase with the flake's report-only analysers. +This document triages the findings by priority so a maintainer can act on the +signal and skip the noise. **No code is changed by this review** — it is a +catalogue and a recommended order of work. + +- **Generated by**: `nix build .#analysis` (branch `static-analysis-review`, off `nix`). +- **Date**: 2026-08-31. +- **Tool versions** (pinned by the flake's `nixpkgs`): ruff 0.16.4, mypy 2.1.0, + bandit 1.9.4, shellcheck 0.11.0. Python target 3.10 (interpreter 3.12.14). +- **Raw reports**: [`reports/static-analysis/`](reports/static-analysis/) — one + `*.txt` per tool plus `summary.txt`. +- **Tooling docs**: [`nix/README.md`](nix/README.md). + +Analysis is report-only: the analysers always succeed and never gate a build. +See "Reproduce & fix" at the end. + +## Summary + +| Tool | Findings | P1 security | P2 correctness | P3 maintainability | P4 style | +|---|---:|---:|---:|---:|---:| +| bandit | 41 | 3 | – | – | – (38 accepted) | +| ruff (lint) | 224 | – | 9 | 58 | 157 | +| mypy | 75 | – | 75 | – | – | +| ruff-format | 11 files | – | – | – | 11 files | +| shellcheck | 279 | – | ~24 | – | ~255 | + +Recommended order: **P1 → P2 → P3 → P4**. P4 (line length + formatting) is the +largest bucket but is mechanical and auto-fixable; do it last, or first as a +one-shot so later diffs stay clean. + +--- + +## P1 — Security (bandit) + +41 bandit findings: **2 Medium**, 39 Low. Only **3** need action; the rest are +false positives or accepted by design (documented below so they are not +re-triaged every run). + +### Act on these + +| Test | Sev | Location | Issue | Recommended fix | +|---|---|---|---|---| +| `B314` | Medium | `cmax/scripts/1-audit/checks/platform_config.py:864` | `xml.etree.ElementTree.fromstring` on tool output | Parse with `defusedxml`, or call `defusedxml.defuse_stdlib()` once at startup. | +| `B310` | Medium | `cmax/minimum_sync.py:214` | `urllib` `urlopen` allows `file:`/custom schemes | Validate the URL scheme is `https` before opening. | +| `B405` | Low | `cmax/scripts/1-audit/checks/platform_config.py:66` | `import xml.etree.ElementTree` | Same as `B314` — switch to `defusedxml`. | + +The two XML findings are the same root cause: XML from cluster tools is parsed +with the stdlib parser, which is vulnerable to entity-expansion attacks. Fixing +the import and the parse call clears all three. + +### Accepted — no action + +| Test | Count | Why accepted | +|---|---:|---| +| `B603` subprocess-without-shell / `B404` import-subprocess / `B607` partial-path | 28 | By design: `cmax` is an audit tool that shells out to cluster commands (`kubectl`, `srun`, `nvidia-smi`, …). Calls use argument lists, not `shell=True`. | +| `B105`/`B106` hardcoded-password | 5 | All false positives: the flagged strings are status/message values, not secrets — e.g. `'pass'`, `'0'`, `'kubelet CPU Manager policy check passed'` (`cmax/security.py:21`, `cmax/audit_report.py:23`, `cmax/scripts/1-audit/security_version_audit.py:814,827`). | +| `B101` assert-used | 4 | Non-critical asserts in `cmax/progress.py`. Harmless; convert to explicit checks only if these paths run under `python -O`. | +| `B110` try-except-pass | 1 | `cmax/scripts/1-audit/checks/system/hbm_memory_exposure.py:711` — a best-effort probe. Fine; add a comment if you want the intent recorded. | + +--- + +## P2 — Correctness (ruff `F`/`B` + mypy) + +Real defects or latent bugs — code that is wrong or can fail at runtime. + +### ruff bug-class rules (9) + +| Rule | Count | Meaning | Example | +|---|---:|---|---| +| `F401` | 5 | Unused import | `cmax/banner.py:43`, `cmax/security.py:7,12` | +| `B023` | 1 | Function uses a loop variable not bound in the loop (classic closure bug) | `cmax/progress.py:155` | +| `B905` | 1 | `zip()` without `strict=` — silently drops items on length mismatch | — | +| `B007` | 1 | Loop control variable not used in the body | — | +| `F841` | 1 | Local variable assigned but never used | — | + +`B023` and `B905` are the two worth reading closely — both can produce silently +wrong results. `F401`/`F841` are safe deletions (`ruff check --fix` handles them). + +### mypy (75 errors) + +Non-strict, `ignore_missing_imports`, so these are genuine local type problems — +mostly attribute/index access on values typed `Any | dict | None`, i.e. **missing +`None` handling** that can raise at runtime. + +| Category | Count | Category | Count | +|---|---:|---|---:| +| `assignment` | 25 | `attr-defined` | 4 | +| `arg-type` | 18 | `var-annotated` | 2 | +| `index` | 10 | `return-value` | 2 | +| `union-attr` | 6 | other | 3 | +| `operator` | 5 | | | + +Most affected: `cmax/minimum_refresh.py` (13), `cmax/progress.py` (11), +`cmax/security.py` (4). Representative: `cmax/scripts/1-audit/gpu_profiles.py:74` +— `Item "None" of "... | None" has no attribute "get"` (a `.get()` on a value +that can be `None`). Fixing the `union-attr`/`index` cases first removes the +runtime-crash risk; `assignment`/`arg-type` are mostly annotations to tighten. + +--- + +## P3 — Maintainability (ruff `SIM`/`UP`/`I001`/`E741`/`E402`/`C4`) + +58 findings. Not bugs — clarity, modern syntax, and import hygiene. Many are +auto-fixable. + +| Group | Count | What | Auto-fix | +|---|---:|---|---| +| `I001` | 13 | Unsorted / unformatted imports | `ruff check --fix` | +| `UP035`/`UP045`/`UP037`/`UP022` | 21 | Deprecated typing imports and old-style syntax (pyupgrade) | mostly `--fix` | +| `SIM105`/`SIM115`/`SIM114`/… | 17 | Simplifiable code (`contextlib.suppress`, context managers, merged branches) | some `--fix` | +| `E741` | 3 | Ambiguous names (`l`, `I`, `O`) | manual | +| `E402` | 2 | Module import not at top of file | manual | +| `C408`/`C420` | 2 | Unnecessary `dict()` / dict-comprehension rewrite | `--fix` | + +`SIM115` (open a file without a context manager) is worth a manual look — it can +leak file handles. The rest are cosmetic-to-minor. + +--- + +## P4 — Style (ruff `E501` + ruff-format) + +Largest bucket, lowest priority, fully mechanical. + +- **`E501` line-too-long — 157**. All are length-only. Either raise the limit or + reflow. Concentrated in `cmax/security.py` (28) and `cmax/progress.py` (7). +- **ruff-format — 11 files** would be reformatted (3955-line diff): `audit_report.py`, + `audit_review.py`, `banner.py`, `cli.py`, `criteria_links.py`, `minimum_refresh.py`, + `minimum_sync.py`, `progress.py`, `report_style.py`, `security.py`, + `target_selection.py`. + +One command fixes both: `ruff format cmax`. Doing this **once, in its own commit** +keeps every later review diff readable. `[tool.ruff] line-length = 100` in +`pyproject.toml` sets the width. + +--- + +## Shell scripts (shellcheck) + +279 findings across 10 audit scripts: **1 error, 206 warning, 67 info, 5 style**. +Concentrated in `cluster-audit-standalone.sh` (133), `cluster-audit-slurm.sh` (89), +`host-check.sh` (23). + +### Act on these + +| Code | Count | Meaning | Action | +|---|---:|---|---| +| `SC2148` | 1 (error) | No shebang / shell directive | `cmax/scripts/1-audit/audit-common.sh` is `source`d, so it has no shebang; add `# shellcheck shell=bash` at the top to set the dialect. | +| `SC2086` | 15 | Unquoted expansion — word-splitting / globbing | Quote the variables (real correctness risk with spaces/globs). | +| `SC2015` | 35 | `A && B || C` is not if/then/else | Review each; `C` runs when `B` fails, which is often not intended. | +| `SC2153`/`SC2010`/`SC2012` | ~13 | Possible misspelled var / parsing `ls` | Check the var names; prefer globs over parsing `ls`. | + +### Mostly noise + +- **`SC2034` unused-variable — 200** (72% of all findings). Many are collected + keys or documentation values referenced indirectly. Triage per script; silence + intentional ones with `# shellcheck disable=SC2034` rather than deleting. + +Fixing the single error and the `SC2086` quoting issues first gives the best +return; `SC2034` can be swept later, script by script. + +--- + +## Reproduce & fix + +``` +# Regenerate every report (writes result/summary.txt + per-tool report.txt): +nix build .#analysis && cat result/summary.txt + +# A single tool: +nix build .#analysis-bandit # or -ruff, -ruff-format, -mypy, -shellcheck + +# In the dev shell, iterate and apply the safe auto-fixes: +nix develop +cmax-lint # ruff check cmax +cmax-fmt # ruff format cmax (fixes all of P4) +cmax-types # mypy cmax +cmax-sec # bandit -r cmax +cmax-shellcheck # shellcheck the audit .sh scripts +``` + +Suggested sequence: land the three P1 security fixes, then the P2 `ruff --fix` +deletions and the `B023`/`B905`/`SIM115` reads, then a single `ruff format` +commit for P4, and finally sweep P3 and the shell warnings per file. diff --git a/reports/static-analysis/bandit.txt b/reports/static-analysis/bandit.txt new file mode 100644 index 0000000..85d3796 --- /dev/null +++ b/reports/static-analysis/bandit.txt @@ -0,0 +1,543 @@ +[main] INFO profile include tests: None +[main] INFO profile exclude tests: None +[main] INFO cli include tests: None +[main] INFO cli exclude tests: None +[main] INFO running on Python 3.14.7 +Run started:2026-08-31 16:55:06.056940+00:00 + +Test results: +>> Issue: [B105:hardcoded_password_string] Possible hardcoded password: 'pass' + Severity: Low Confidence: Medium + CWE: CWE-259 (https://cwe.mitre.org/data/definitions/259.html) + More Info: https://bandit.readthedocs.io/en/1.9.4/plugins/b105_hardcoded_password_string.html + Location: cmax/audit_report.py:23:7 +22 +23 PASS = "pass" +24 WARNING = "warning" + +-------------------------------------------------- +>> Issue: [B310:blacklist] Audit url open for permitted schemes. Allowing use of file:/ or custom schemes is often unexpected. + Severity: Medium Confidence: High + CWE: CWE-22 (https://cwe.mitre.org/data/definitions/22.html) + More Info: https://bandit.readthedocs.io/en/1.9.4/blacklists/blacklist_calls.html#b310-urllib-urlopen + Location: cmax/minimum_sync.py:214:13 +213 try: +214 with urllib.request.urlopen(request, timeout=timeout) as response: +215 # urllib follows a redirect across schemes, so an origin or CDN rule + +-------------------------------------------------- +>> Issue: [B404:blacklist] Consider possible security implications associated with the subprocess module. + Severity: Low Confidence: High + CWE: CWE-78 (https://cwe.mitre.org/data/definitions/78.html) + More Info: https://bandit.readthedocs.io/en/1.9.4/blacklists/blacklist_imports.html#b404-import-subprocess + Location: cmax/progress.py:31:0 +30 import signal +31 import subprocess +32 import sys + +-------------------------------------------------- +>> Issue: [B101:assert_used] Use of assert detected. The enclosed code will be removed when compiling to optimised byte code. + Severity: Low Confidence: High + CWE: CWE-703 (https://cwe.mitre.org/data/definitions/703.html) + More Info: https://bandit.readthedocs.io/en/1.9.4/plugins/b101_assert_used.html + Location: cmax/progress.py:1572:8 +1571 def _run_tui(self) -> None: +1572 assert self._tui_app is not None +1573 + +-------------------------------------------------- +>> Issue: [B101:assert_used] Use of assert detected. The enclosed code will be removed when compiling to optimised byte code. + Severity: Low Confidence: High + CWE: CWE-703 (https://cwe.mitre.org/data/definitions/703.html) + More Info: https://bandit.readthedocs.io/en/1.9.4/plugins/b101_assert_used.html + Location: cmax/progress.py:1593:8 +1592 return +1593 assert termios is not None and tty is not None +1594 input_fd = sys.stdin.fileno() + +-------------------------------------------------- +>> Issue: [B603:subprocess_without_shell_equals_true] subprocess call - check for execution of untrusted input. + Severity: Low Confidence: High + CWE: CWE-78 (https://cwe.mitre.org/data/definitions/78.html) + More Info: https://bandit.readthedocs.io/en/1.9.4/plugins/b603_subprocess_without_shell_equals_true.html + Location: cmax/progress.py:1923:11 +1922 chunks: list[str] = [] +1923 proc = subprocess.Popen( +1924 list(command), +1925 cwd=str(cwd) if cwd is not None else None, +1926 env=env, +1927 stdout=subprocess.PIPE, +1928 stderr=subprocess.STDOUT, +1929 text=True, +1930 # The collector prints dmesg excerpts and raw vendor tool output, which +1931 # can carry bytes that are not valid UTF-8. Strict decoding would raise +1932 # inside the read loop and kill a healthy collector mid-run. +1933 errors="replace", +1934 bufsize=1, +1935 ) +1936 assert proc.stdout is not None + +-------------------------------------------------- +>> Issue: [B101:assert_used] Use of assert detected. The enclosed code will be removed when compiling to optimised byte code. + Severity: Low Confidence: High + CWE: CWE-703 (https://cwe.mitre.org/data/definitions/703.html) + More Info: https://bandit.readthedocs.io/en/1.9.4/plugins/b101_assert_used.html + Location: cmax/progress.py:1936:4 +1935 ) +1936 assert proc.stdout is not None +1937 display.start() + +-------------------------------------------------- +>> Issue: [B603:subprocess_without_shell_equals_true] subprocess call - check for execution of untrusted input. + Severity: Low Confidence: High + CWE: CWE-78 (https://cwe.mitre.org/data/definitions/78.html) + More Info: https://bandit.readthedocs.io/en/1.9.4/plugins/b603_subprocess_without_shell_equals_true.html + Location: cmax/progress.py:1961:11 +1960 chunks: list[str] = [] +1961 proc = subprocess.Popen( +1962 list(command), +1963 cwd=str(cwd) if cwd is not None else None, +1964 env=env, +1965 stdout=subprocess.PIPE, +1966 stderr=subprocess.STDOUT, +1967 text=True, +1968 errors="replace", +1969 bufsize=1, +1970 ) +1971 assert proc.stdout is not None + +-------------------------------------------------- +>> Issue: [B101:assert_used] Use of assert detected. The enclosed code will be removed when compiling to optimised byte code. + Severity: Low Confidence: High + CWE: CWE-703 (https://cwe.mitre.org/data/definitions/703.html) + More Info: https://bandit.readthedocs.io/en/1.9.4/plugins/b101_assert_used.html + Location: cmax/progress.py:1971:4 +1970 ) +1971 assert proc.stdout is not None +1972 try: + +-------------------------------------------------- +>> Issue: [B404:blacklist] Consider possible security implications associated with the subprocess module. + Severity: Low Confidence: High + CWE: CWE-78 (https://cwe.mitre.org/data/definitions/78.html) + More Info: https://bandit.readthedocs.io/en/1.9.4/blacklists/blacklist_imports.html#b404-import-subprocess + Location: cmax/scripts/1-audit/checks/fabric/nic-topology-check.py:75:0 +74 import socket +75 import subprocess +76 import sys + +-------------------------------------------------- +>> Issue: [B404:blacklist] Consider possible security implications associated with the subprocess module. + Severity: Low Confidence: High + CWE: CWE-78 (https://cwe.mitre.org/data/definitions/78.html) + More Info: https://bandit.readthedocs.io/en/1.9.4/blacklists/blacklist_imports.html#b404-import-subprocess + Location: cmax/scripts/1-audit/checks/fabric/virtio-net-check.py:134:0 +133 import socket +134 import subprocess +135 import sys + +-------------------------------------------------- +>> Issue: [B404:blacklist] Consider possible security implications associated with the subprocess module. + Severity: Low Confidence: High + CWE: CWE-78 (https://cwe.mitre.org/data/definitions/78.html) + More Info: https://bandit.readthedocs.io/en/1.9.4/blacklists/blacklist_imports.html#b404-import-subprocess + Location: cmax/scripts/1-audit/checks/gpu/vboost.py:11:0 +10 import socket +11 import subprocess +12 import sys + +-------------------------------------------------- +>> Issue: [B404:blacklist] Consider possible security implications associated with the subprocess module. + Severity: Low Confidence: High + CWE: CWE-78 (https://cwe.mitre.org/data/definitions/78.html) + More Info: https://bandit.readthedocs.io/en/1.9.4/blacklists/blacklist_imports.html#b404-import-subprocess + Location: cmax/scripts/1-audit/checks/platform_config.py:63:0 +62 import socket +63 import subprocess +64 import sys + +-------------------------------------------------- +>> Issue: [B405:blacklist] Using xml.etree.ElementTree to parse untrusted XML data is known to be vulnerable to XML attacks. Replace xml.etree.ElementTree with the equivalent defusedxml package, or make sure defusedxml.defuse_stdlib() is called. + Severity: Low Confidence: High + CWE: CWE-20 (https://cwe.mitre.org/data/definitions/20.html) + More Info: https://bandit.readthedocs.io/en/1.9.4/blacklists/blacklist_imports.html#b405-import-xml-etree + Location: cmax/scripts/1-audit/checks/platform_config.py:66:0 +65 import uuid +66 import xml.etree.ElementTree as ElementTree +67 from pathlib import Path + +-------------------------------------------------- +>> Issue: [B603:subprocess_without_shell_equals_true] subprocess call - check for execution of untrusted input. + Severity: Low Confidence: High + CWE: CWE-78 (https://cwe.mitre.org/data/definitions/78.html) + More Info: https://bandit.readthedocs.io/en/1.9.4/plugins/b603_subprocess_without_shell_equals_true.html + Location: cmax/scripts/1-audit/checks/platform_config.py:252:11 +251 def run_command(command: list[str], *, timeout: int = 30, input_text: str | None = None): +252 return subprocess.run( +253 command, +254 input=input_text, +255 stdout=subprocess.PIPE, +256 stderr=subprocess.PIPE, +257 text=True, +258 timeout=timeout, +259 ) +260 + +-------------------------------------------------- +>> Issue: [B314:blacklist] Using xml.etree.ElementTree.fromstring to parse untrusted XML data is known to be vulnerable to XML attacks. Replace xml.etree.ElementTree.fromstring with its defusedxml equivalent function or make sure defusedxml.defuse_stdlib() is called + Severity: Medium Confidence: High + CWE: CWE-20 (https://cwe.mitre.org/data/definitions/20.html) + More Info: https://bandit.readthedocs.io/en/1.9.4/blacklists/blacklist_calls.html#b313-b320-xml-bad-elementtree + Location: cmax/scripts/1-audit/checks/platform_config.py:864:8 +863 try: +864 ElementTree.fromstring(data) +865 xml_ok = True + +-------------------------------------------------- +>> Issue: [B404:blacklist] Consider possible security implications associated with the subprocess module. + Severity: Low Confidence: High + CWE: CWE-78 (https://cwe.mitre.org/data/definitions/78.html) + More Info: https://bandit.readthedocs.io/en/1.9.4/blacklists/blacklist_imports.html#b404-import-subprocess + Location: cmax/scripts/1-audit/checks/system/hbm_memory_exposure.py:28:0 +27 import socket +28 import subprocess +29 import sys + +-------------------------------------------------- +>> Issue: [B603:subprocess_without_shell_equals_true] subprocess call - check for execution of untrusted input. + Severity: Low Confidence: High + CWE: CWE-78 (https://cwe.mitre.org/data/definitions/78.html) + More Info: https://bandit.readthedocs.io/en/1.9.4/plugins/b603_subprocess_without_shell_equals_true.html + Location: cmax/scripts/1-audit/checks/system/hbm_memory_exposure.py:146:11 +145 def run_command(command: list[str], *, timeout: int = 30, input_text: str | None = None) -> subprocess.CompletedProcess[str]: +146 return subprocess.run( +147 command, +148 input=input_text, +149 stdout=subprocess.PIPE, +150 stderr=subprocess.PIPE, +151 text=True, +152 timeout=timeout, +153 ) +154 + +-------------------------------------------------- +>> Issue: [B106:hardcoded_password_funcarg] Possible hardcoded password: 'kubelet CPU Manager policy check passed' + Severity: Low Confidence: Medium + CWE: CWE-259 (https://cwe.mitre.org/data/definitions/259.html) + More Info: https://bandit.readthedocs.io/en/1.9.4/plugins/b106_hardcoded_password_funcarg.html + Location: cmax/scripts/1-audit/checks/system/hbm_memory_exposure.py:560:11 +559 summary_key="kubelet_cpu_manager_policy", +560 pass_message="kubelet CPU Manager policy check passed", +561 not_applicable_message="no Kubernetes GPU hosts were checked", +562 ) +563 +564 +565 def run_slurm_check(harness: str) -> tuple[list[dict[str, Any]], list[str]]: +566 if not os.environ.get("SLURM_JOB_ID"): +567 report = collect_host(root=Path("/"), harness=harness) + +-------------------------------------------------- +>> Issue: [B110:try_except_pass] Try, Except, Pass detected. + Severity: Low Confidence: High + CWE: CWE-703 (https://cwe.mitre.org/data/definitions/703.html) + More Info: https://bandit.readthedocs.io/en/1.9.4/plugins/b110_try_except_pass.html + Location: cmax/scripts/1-audit/checks/system/hbm_memory_exposure.py:711:8 +710 kubectl(["delete", "pod", pod_name, "-n", namespace, "--ignore-not-found=true", "--wait=false"], timeout=20) +711 except Exception: +712 pass +713 + +-------------------------------------------------- +>> Issue: [B404:blacklist] Consider possible security implications associated with the subprocess module. + Severity: Low Confidence: High + CWE: CWE-78 (https://cwe.mitre.org/data/definitions/78.html) + More Info: https://bandit.readthedocs.io/en/1.9.4/blacklists/blacklist_imports.html#b404-import-subprocess + Location: cmax/scripts/1-audit/plan_audit.py:10:0 +9 import shutil +10 import subprocess +11 import sys + +-------------------------------------------------- +>> Issue: [B603:subprocess_without_shell_equals_true] subprocess call - check for execution of untrusted input. + Severity: Low Confidence: High + CWE: CWE-78 (https://cwe.mitre.org/data/definitions/78.html) + More Info: https://bandit.readthedocs.io/en/1.9.4/plugins/b603_subprocess_without_shell_equals_true.html + Location: cmax/scripts/1-audit/plan_audit.py:19:11 +18 def command_ok(command: list[str]) -> bool: +19 return subprocess.run( +20 command, +21 stdout=subprocess.DEVNULL, +22 stderr=subprocess.DEVNULL, +23 ).returncode == 0 +24 + +-------------------------------------------------- +>> Issue: [B607:start_process_with_partial_path] Starting a process with a partial executable path + Severity: Low Confidence: High + CWE: CWE-78 (https://cwe.mitre.org/data/definitions/78.html) + More Info: https://bandit.readthedocs.io/en/1.9.4/plugins/b607_start_process_with_partial_path.html + Location: cmax/scripts/1-audit/plan_audit.py:46:13 +45 def hostname_label() -> str: +46 result = subprocess.run( +47 ["hostname", "-s"], +48 stdout=subprocess.PIPE, +49 stderr=subprocess.DEVNULL, +50 text=True, +51 ) +52 label = result.stdout.strip() if result.returncode == 0 else "" + +-------------------------------------------------- +>> Issue: [B603:subprocess_without_shell_equals_true] subprocess call - check for execution of untrusted input. + Severity: Low Confidence: High + CWE: CWE-78 (https://cwe.mitre.org/data/definitions/78.html) + More Info: https://bandit.readthedocs.io/en/1.9.4/plugins/b603_subprocess_without_shell_equals_true.html + Location: cmax/scripts/1-audit/plan_audit.py:46:13 +45 def hostname_label() -> str: +46 result = subprocess.run( +47 ["hostname", "-s"], +48 stdout=subprocess.PIPE, +49 stderr=subprocess.DEVNULL, +50 text=True, +51 ) +52 label = result.stdout.strip() if result.returncode == 0 else "" + +-------------------------------------------------- +>> Issue: [B607:start_process_with_partial_path] Starting a process with a partial executable path + Severity: Low Confidence: High + CWE: CWE-78 (https://cwe.mitre.org/data/definitions/78.html) + More Info: https://bandit.readthedocs.io/en/1.9.4/plugins/b607_start_process_with_partial_path.html + Location: cmax/scripts/1-audit/plan_audit.py:54:17 +53 if not label: +54 result = subprocess.run( +55 ["hostname"], +56 stdout=subprocess.PIPE, +57 stderr=subprocess.DEVNULL, +58 text=True, +59 ) +60 label = result.stdout.strip() if result.returncode == 0 else "cluster" + +-------------------------------------------------- +>> Issue: [B603:subprocess_without_shell_equals_true] subprocess call - check for execution of untrusted input. + Severity: Low Confidence: High + CWE: CWE-78 (https://cwe.mitre.org/data/definitions/78.html) + More Info: https://bandit.readthedocs.io/en/1.9.4/plugins/b603_subprocess_without_shell_equals_true.html + Location: cmax/scripts/1-audit/plan_audit.py:54:17 +53 if not label: +54 result = subprocess.run( +55 ["hostname"], +56 stdout=subprocess.PIPE, +57 stderr=subprocess.DEVNULL, +58 text=True, +59 ) +60 label = result.stdout.strip() if result.returncode == 0 else "cluster" + +-------------------------------------------------- +>> Issue: [B404:blacklist] Consider possible security implications associated with the subprocess module. + Severity: Low Confidence: High + CWE: CWE-78 (https://cwe.mitre.org/data/definitions/78.html) + More Info: https://bandit.readthedocs.io/en/1.9.4/blacklists/blacklist_imports.html#b404-import-subprocess + Location: cmax/scripts/1-audit/run_checks.py:9:0 +8 import re +9 import subprocess +10 import sys + +-------------------------------------------------- +>> Issue: [B607:start_process_with_partial_path] Starting a process with a partial executable path + Severity: Low Confidence: High + CWE: CWE-78 (https://cwe.mitre.org/data/definitions/78.html) + More Info: https://bandit.readthedocs.io/en/1.9.4/plugins/b607_start_process_with_partial_path.html + Location: cmax/scripts/1-audit/run_checks.py:130:15 +129 try: +130 proc = subprocess.run( +131 ["kubectl", "get", "namespaces", "-o", "json"], +132 stdout=subprocess.PIPE, +133 stderr=subprocess.DEVNULL, +134 text=True, +135 timeout=30, +136 ) +137 except (OSError, subprocess.TimeoutExpired): + +-------------------------------------------------- +>> Issue: [B603:subprocess_without_shell_equals_true] subprocess call - check for execution of untrusted input. + Severity: Low Confidence: High + CWE: CWE-78 (https://cwe.mitre.org/data/definitions/78.html) + More Info: https://bandit.readthedocs.io/en/1.9.4/plugins/b603_subprocess_without_shell_equals_true.html + Location: cmax/scripts/1-audit/run_checks.py:130:15 +129 try: +130 proc = subprocess.run( +131 ["kubectl", "get", "namespaces", "-o", "json"], +132 stdout=subprocess.PIPE, +133 stderr=subprocess.DEVNULL, +134 text=True, +135 timeout=30, +136 ) +137 except (OSError, subprocess.TimeoutExpired): + +-------------------------------------------------- +>> Issue: [B603:subprocess_without_shell_equals_true] subprocess call - check for execution of untrusted input. + Severity: Low Confidence: High + CWE: CWE-78 (https://cwe.mitre.org/data/definitions/78.html) + More Info: https://bandit.readthedocs.io/en/1.9.4/plugins/b603_subprocess_without_shell_equals_true.html + Location: cmax/scripts/1-audit/run_checks.py:230:15 +229 ) +230 proc = subprocess.run( +231 argv, +232 stdout=subprocess.PIPE, +233 stderr=subprocess.PIPE, +234 text=True, +235 env=env, +236 ) +237 if proc.stderr: + +-------------------------------------------------- +>> Issue: [B404:blacklist] Consider possible security implications associated with the subprocess module. + Severity: Low Confidence: High + CWE: CWE-78 (https://cwe.mitre.org/data/definitions/78.html) + More Info: https://bandit.readthedocs.io/en/1.9.4/blacklists/blacklist_imports.html#b404-import-subprocess + Location: cmax/scripts/1-audit/run_legacy_audit.py:7:0 +6 import os +7 import subprocess +8 import sys + +-------------------------------------------------- +>> Issue: [B607:start_process_with_partial_path] Starting a process with a partial executable path + Severity: Low Confidence: High + CWE: CWE-78 (https://cwe.mitre.org/data/definitions/78.html) + More Info: https://bandit.readthedocs.io/en/1.9.4/plugins/b607_start_process_with_partial_path.html + Location: cmax/scripts/1-audit/run_legacy_audit.py:55:13 +54 +55 result = subprocess.run( +56 ["bash", str(audit_script), "--name", slug, "--output-dir", str(tmpdir)], +57 cwd=legacy_cwd, +58 ) +59 if result.returncode != 0: + +-------------------------------------------------- +>> Issue: [B603:subprocess_without_shell_equals_true] subprocess call - check for execution of untrusted input. + Severity: Low Confidence: High + CWE: CWE-78 (https://cwe.mitre.org/data/definitions/78.html) + More Info: https://bandit.readthedocs.io/en/1.9.4/plugins/b603_subprocess_without_shell_equals_true.html + Location: cmax/scripts/1-audit/run_legacy_audit.py:55:13 +54 +55 result = subprocess.run( +56 ["bash", str(audit_script), "--name", slug, "--output-dir", str(tmpdir)], +57 cwd=legacy_cwd, +58 ) +59 if result.returncode != 0: + +-------------------------------------------------- +>> Issue: [B105:hardcoded_password_string] Possible hardcoded password: '0' + Severity: Low Confidence: Medium + CWE: CWE-259 (https://cwe.mitre.org/data/definitions/259.html) + More Info: https://bandit.readthedocs.io/en/1.9.4/plugins/b105_hardcoded_password_string.html + Location: cmax/scripts/1-audit/security_version_audit.py:814:66 +813 # proven failure outranks an unresolved host, which outranks a clean one. +814 _VIRTIO_SEVERITY = {"fail": 3, "unknown": 2, "not_applicable": 1, "pass": 0} +815 # Tie-break among readings that already agree on the status, so the reading that + +-------------------------------------------------- +>> Issue: [B105:hardcoded_password_string] Possible hardcoded password: '0' + Severity: Low Confidence: Medium + CWE: CWE-259 (https://cwe.mitre.org/data/definitions/259.html) + More Info: https://bandit.readthedocs.io/en/1.9.4/plugins/b105_hardcoded_password_string.html + Location: cmax/scripts/1-audit/security_version_audit.py:827:53 +826 # loses a proven finding rather than a sentence of detail. +827 _VIRTIO_MINIMUM_SEVERITY = {"fail": 2, "unknown": 1, "pass": 0} +828 VERSION_REASON_NOT_OBSERVED = "not-observed" + +-------------------------------------------------- +>> Issue: [B404:blacklist] Consider possible security implications associated with the subprocess module. + Severity: Low Confidence: High + CWE: CWE-78 (https://cwe.mitre.org/data/definitions/78.html) + More Info: https://bandit.readthedocs.io/en/1.9.4/blacklists/blacklist_imports.html#b404-import-subprocess + Location: cmax/security.py:9:0 +8 import shutil +9 import subprocess +10 import sys + +-------------------------------------------------- +>> Issue: [B105:hardcoded_password_string] Possible hardcoded password: 'pass' + Severity: Low Confidence: Medium + CWE: CWE-259 (https://cwe.mitre.org/data/definitions/259.html) + More Info: https://bandit.readthedocs.io/en/1.9.4/plugins/b105_hardcoded_password_string.html + Location: cmax/security.py:21:7 +20 +21 PASS = "pass" +22 WARNING = "warning" + +-------------------------------------------------- +>> Issue: [B603:subprocess_without_shell_equals_true] subprocess call - check for execution of untrusted input. + Severity: Low Confidence: High + CWE: CWE-78 (https://cwe.mitre.org/data/definitions/78.html) + More Info: https://bandit.readthedocs.io/en/1.9.4/plugins/b603_subprocess_without_shell_equals_true.html + Location: cmax/security.py:90:12 +89 return ( +90 subprocess.run( +91 command, +92 stdout=subprocess.DEVNULL, +93 stderr=subprocess.DEVNULL, +94 timeout=timeout, +95 ).returncode +96 == 0 + +-------------------------------------------------- +>> Issue: [B603:subprocess_without_shell_equals_true] subprocess call - check for execution of untrusted input. + Severity: Low Confidence: High + CWE: CWE-78 (https://cwe.mitre.org/data/definitions/78.html) + More Info: https://bandit.readthedocs.io/en/1.9.4/plugins/b603_subprocess_without_shell_equals_true.html + Location: cmax/security.py:123:21 +122 try: +123 returncode = subprocess.run( +124 [command, "--quiet", "--vm"], +125 stdout=subprocess.DEVNULL, +126 stderr=subprocess.DEVNULL, +127 timeout=5.0, +128 ).returncode +129 except (OSError, subprocess.TimeoutExpired): + +-------------------------------------------------- +>> Issue: [B404:blacklist] Consider possible security implications associated with the subprocess module. + Severity: Low Confidence: High + CWE: CWE-78 (https://cwe.mitre.org/data/definitions/78.html) + More Info: https://bandit.readthedocs.io/en/1.9.4/blacklists/blacklist_imports.html#b404-import-subprocess + Location: cmax/target_selection.py:7:0 +6 import socket +7 import subprocess +8 import sys + +-------------------------------------------------- +>> Issue: [B603:subprocess_without_shell_equals_true] subprocess call - check for execution of untrusted input. + Severity: Low Confidence: High + CWE: CWE-78 (https://cwe.mitre.org/data/definitions/78.html) + More Info: https://bandit.readthedocs.io/en/1.9.4/plugins/b603_subprocess_without_shell_equals_true.html + Location: cmax/target_selection.py:51:15 +50 try: +51 return subprocess.run( +52 command, +53 capture_output=True, +54 text=True, +55 check=False, +56 timeout=timeout, +57 env=dict(environ), +58 ) +59 except (OSError, subprocess.SubprocessError) as exc: + +-------------------------------------------------- + +Code scanned: + Total lines of code: 18898 + Total lines skipped (#nosec): 0 + Total potential issues skipped due to specifically being disabled (e.g., #nosec BXXX): 0 + +Run metrics: + Total issues (by severity): + Undefined: 0 + Low: 39 + Medium: 2 + High: 0 + Total issues (by confidence): + Undefined: 0 + Low: 0 + Medium: 5 + High: 36 +Files skipped (0): diff --git a/reports/static-analysis/mypy.txt b/reports/static-analysis/mypy.txt new file mode 100644 index 0000000..f379e01 --- /dev/null +++ b/reports/static-analysis/mypy.txt @@ -0,0 +1,81 @@ +cmax/scripts/1-audit/gpu_profiles.py:74: error: Item "None" of "Any | dict[Any, Any] | None" has no attribute "get" [union-attr] +cmax/scripts/1-audit/gpu_profiles.py:75: error: Item "None" of "Any | dict[Any, Any] | None" has no attribute "get" [union-attr] +cmax/scripts/1-audit/gpu_profiles.py:79: error: Item "None" of "Any | dict[Any, Any] | None" has no attribute "get" [union-attr] +cmax/scripts/1-audit/gpu_profiles.py:80: error: Argument 1 to "_model" has incompatible type "Any | dict[Any, Any] | None"; expected "dict[str, Any]" [arg-type] +cmax/scripts/1-audit/gpu_profiles.py:82: error: Item "None" of "Any | dict[Any, Any] | None" has no attribute "get" [union-attr] +cmax/scripts/1-audit/gpu_profiles.py:83: error: Item "None" of "Any | dict[Any, Any] | None" has no attribute "get" [union-attr] +cmax/scripts/1-audit/gpu_profiles.py:84: error: Item "None" of "Any | dict[Any, Any] | None" has no attribute "get" [union-attr] +cmax/scripts/1-audit/gpu_profiles.py:85: error: Argument 1 to "_gpu_memory" has incompatible type "Any | dict[Any, Any] | None"; expected "dict[str, Any]" [arg-type] +cmax/scripts/1-audit/security_version_audit.py:605: error: Value of type "tuple[int, ...] | None" is not indexable [index] +cmax/scripts/1-audit/security_version_audit.py:615: error: Value of type "tuple[int, ...] | None" is not indexable [index] +cmax/scripts/1-audit/security_version_audit.py:624: error: Value of type "tuple[int, ...] | None" is not indexable [index] +cmax/scripts/1-audit/security_version_audit.py:681: error: Unsupported operand types for >= ("tuple[int, ...]" and "None") [operator] +cmax/scripts/1-audit/security_version_audit.py:681: note: Right operand is of type "tuple[int, ...] | None" +cmax/scripts/1-audit/security_version_audit.py:693: error: Value of type "tuple[int, ...] | None" is not indexable [index] +cmax/scripts/1-audit/security_version_audit.py:695: error: Value of type variable "SupportsRichComparisonT" of "max" cannot be "tuple[int, ...] | None" [type-var] +cmax/scripts/1-audit/security_version_audit.py:696: error: Unsupported operand types for <= ("tuple[int, ...]" and "None") [operator] +cmax/scripts/1-audit/security_version_audit.py:696: note: Right operand is of type "tuple[int, ...] | None" +cmax/scripts/1-audit/security_version_audit.py:702: error: Value of type "tuple[int, ...] | None" is not indexable [index] +cmax/scripts/1-audit/security_version_audit.py:709: error: Value of type "tuple[int, ...] | None" is not indexable [index] +cmax/scripts/1-audit/security_version_audit.py:719: error: Value of type "tuple[int, ...] | None" is not indexable [index] +cmax/scripts/1-audit/security_version_audit.py:726: error: Value of type "tuple[int, ...] | None" is not indexable [index] +cmax/scripts/1-audit/security_version_audit.py:739: error: Argument "key" to "max" has incompatible type "Callable[[str], tuple[int, ...] | None]"; expected "Callable[[str], SupportsDunderLT[Any] | SupportsDunderGT[Any]]" [arg-type] +cmax/scripts/1-audit/security_version_audit.py:739: error: Incompatible return value type (got "tuple[int, ...] | None", expected "SupportsDunderLT[Any] | SupportsDunderGT[Any]") [return-value] +cmax/scripts/1-audit/security_version_audit.py:740: error: Argument "key" to "min" has incompatible type "Callable[[str], tuple[int, ...] | None]"; expected "Callable[[str], SupportsDunderLT[Any] | SupportsDunderGT[Any]]" [arg-type] +cmax/scripts/1-audit/security_version_audit.py:740: error: Incompatible return value type (got "tuple[int, ...] | None", expected "SupportsDunderLT[Any] | SupportsDunderGT[Any]") [return-value] +cmax/scripts/1-audit/security_version_audit.py:741: error: Unsupported operand types for >= ("tuple[int, ...]" and "None") [operator] +cmax/scripts/1-audit/security_version_audit.py:741: note: Right operand is of type "tuple[int, ...] | None" +cmax/scripts/1-audit/security_version_audit.py:749: error: Unsupported operand types for < ("tuple[int, ...]" and "None") [operator] +cmax/scripts/1-audit/security_version_audit.py:749: note: Right operand is of type "tuple[int, ...] | None" +cmax/scripts/1-audit/security_version_audit.py:1208: error: Incompatible types in assignment (expression has type "str | None", variable has type "str") [assignment] +cmax/scripts/1-audit/security_version_audit.py:1219: error: Incompatible types in assignment (expression has type "str | None", variable has type "str") [assignment] +cmax/scripts/1-audit/security_version_audit.py:1231: error: Incompatible types in assignment (expression has type "str | None", variable has type "str") [assignment] +cmax/progress.py:55: error: Incompatible types in assignment (expression has type "None", variable has type Module) [assignment] +cmax/progress.py:56: error: Incompatible types in assignment (expression has type "None", variable has type Module) [assignment] +cmax/progress.py:672: error: Unsupported operand types for - ("float" and "None") [operator] +cmax/progress.py:672: note: Error code "operator" not covered by "type: ignore[type-var]" comment +cmax/progress.py:672: note: Right operand is of type "float | None" +cmax/progress.py:890: error: Incompatible types in assignment (expression has type "tuple[str, str]", variable has type "tuple[str] | tuple[()]") [assignment] +cmax/progress.py:1531: error: "object" has no attribute "data" [attr-defined] +cmax/progress.py:1550: error: "None" has no attribute "__enter__" [attr-defined] +cmax/progress.py:1595: error: Incompatible types in assignment (expression has type "list[Any]", variable has type "None") [assignment] +cmax/progress.py:1624: error: "object" has no attribute "key" [attr-defined] +cmax/progress.py:1626: error: "object" has no attribute "data" [attr-defined] +cmax/minimum_refresh.py:685: error: Argument 1 to "append" of "list" has incompatible type "str | None"; expected "str" [arg-type] +cmax/minimum_refresh.py:687: error: Argument 1 to "append" of "list" has incompatible type "str | None"; expected "str" [arg-type] +cmax/minimum_refresh.py:852: error: Argument 1 to "append" of "list" has incompatible type "str | None"; expected "str" [arg-type] +cmax/minimum_refresh.py:854: error: Argument 1 to "append" of "list" has incompatible type "str | None"; expected "str" [arg-type] +cmax/minimum_refresh.py:856: error: Argument 1 to "append" of "list" has incompatible type "str | None"; expected "str" [arg-type] +cmax/minimum_refresh.py:1074: error: Value of type "dict[str, Any] | None" is not indexable [index] +cmax/minimum_refresh.py:1075: error: Value of type "dict[str, Any] | None" is not indexable [index] +cmax/minimum_refresh.py:1103: error: List comprehension has incompatible type List[dict[str, Any]]; expected List[str] [misc] +cmax/minimum_refresh.py:2030: error: Argument 4 to "_minimum_downgrades" has incompatible type "Any | None"; expected "dict[Any, Any]" [arg-type] +cmax/minimum_refresh.py:2035: error: Argument 4 to "_minimum_downgrades" has incompatible type "Any | None"; expected "dict[Any, Any]" [arg-type] +cmax/minimum_refresh.py:2040: error: Argument 4 to "_minimum_downgrades" has incompatible type "Any | None"; expected "dict[Any, Any]" [arg-type] +cmax/minimum_refresh.py:2045: error: Argument 4 to "_minimum_downgrades" has incompatible type "Any | None"; expected "dict[Any, Any]" [arg-type] +cmax/minimum_refresh.py:2314: error: Argument 1 to "amd_bulletin_page" has incompatible type "int | str | None"; expected "str" [arg-type] +cmax/security.py:1499: error: Argument 1 has incompatible type "Any | dict[str, Any] | None"; expected "dict[str, Any]" [arg-type] +cmax/security.py:1508: error: Argument 1 to "_get" has incompatible type "Any | dict[str, Any] | None"; expected "dict[str, Any]" [arg-type] +cmax/security.py:1521: error: Argument 1 to "_get" has incompatible type "Any | dict[str, Any] | None"; expected "dict[str, Any]" [arg-type] +cmax/security.py:1588: error: Need type annotation for "details" (hint: "details: list[] = ...") [var-annotated] +cmax/audit_report.py:929: error: Need type annotation for "detect_findings" [var-annotated] +cmax/audit_report.py:961: error: Argument 1 to "_harness_not_applicable" has incompatible type "str | None"; expected "str" [arg-type] +cmax/scripts/1-audit/checks/gpu/vboost.py:159: error: Incompatible default for parameter "env" (default has type "_Environ[str]", parameter has type "dict[str, str]") [assignment] +cmax/scripts/1-audit/checks/gpu/vboost.py:309: error: Incompatible default for parameter "env" (default has type "_Environ[str]", parameter has type "dict[str, str]") [assignment] +cmax/scripts/1-audit/checks/gpu/vboost.py:364: error: Incompatible default for parameter "env" (default has type "_Environ[str]", parameter has type "dict[str, str]") [assignment] +cmax/scripts/1-audit/checks/fabric/virtio-net-check.py:329: error: Unused "type: ignore" comment [unused-ignore] +cmax/scripts/1-audit/checks/fabric/virtio-net-check.py:335: error: Incompatible default for parameter "env" (default has type "_Environ[str]", parameter has type "dict[str, str]") [assignment] +cmax/scripts/1-audit/checks/fabric/virtio-net-check.py:411: error: Incompatible default for parameter "env" (default has type "_Environ[str]", parameter has type "dict[str, str]") [assignment] +cmax/scripts/1-audit/checks/fabric/virtio-net-check.py:482: error: Incompatible default for parameter "env" (default has type "_Environ[str]", parameter has type "dict[str, str]") [assignment] +cmax/scripts/1-audit/checks/fabric/virtio-net-check.py:627: error: Incompatible default for parameter "env" (default has type "_Environ[str]", parameter has type "dict[str, str]") [assignment] +cmax/scripts/1-audit/checks/fabric/virtio-net-check.py:817: error: Incompatible default for parameter "env" (default has type "_Environ[str]", parameter has type "dict[str, str]") [assignment] +cmax/scripts/1-audit/checks/fabric/virtio-net-check.py:956: error: Incompatible default for parameter "env" (default has type "_Environ[str]", parameter has type "dict[str, str]") [assignment] +cmax/scripts/1-audit/checks/fabric/virtio-net-check.py:1361: error: Incompatible default for parameter "env" (default has type "_Environ[str]", parameter has type "dict[str, str]") [assignment] +cmax/scripts/1-audit/checks/fabric/virtio-net-check.py:1621: error: Incompatible default for parameter "env" (default has type "_Environ[str]", parameter has type "dict[str, str]") [assignment] +cmax/scripts/1-audit/checks/fabric/virtio-net-check.py:1652: error: Incompatible default for parameter "env" (default has type "_Environ[str]", parameter has type "dict[str, str]") [assignment] +cmax/scripts/1-audit/checks/fabric/virtio-net-check.py:1745: error: Incompatible default for parameter "env" (default has type "_Environ[str]", parameter has type "dict[str, str]") [assignment] +cmax/scripts/1-audit/checks/fabric/nic-topology-check.py:337: error: Incompatible types in assignment (expression has type "int", variable has type "str") [assignment] +cmax/scripts/1-audit/checks/fabric/nic-topology-check.py:339: error: Incompatible types in assignment (expression has type "int", variable has type "str") [assignment] +cmax/scripts/1-audit/checks/fabric/nic-topology-check.py:393: error: Incompatible default for parameter "env" (default has type "_Environ[str]", parameter has type "dict[str, str]") [assignment] +cmax/scripts/1-audit/checks/fabric/nic-topology-check.py:616: error: Incompatible default for parameter "env" (default has type "_Environ[str]", parameter has type "dict[str, str]") [assignment] +cmax/scripts/1-audit/checks/fabric/nic-topology-check.py:649: error: Incompatible default for parameter "env" (default has type "_Environ[str]", parameter has type "dict[str, str]") [assignment] diff --git a/reports/static-analysis/ruff-format.txt b/reports/static-analysis/ruff-format.txt new file mode 100644 index 0000000..648a3c0 --- /dev/null +++ b/reports/static-analysis/ruff-format.txt @@ -0,0 +1,3955 @@ +--- cmax/audit_report.py ++++ cmax/audit_report.py +@@ -64,18 +64,14 @@ + "securityVersions.cudaToolkit.status": ( + "The CUDA Toolkit meets the published security minimum." + ), +- "securityVersions.runc.status": ( +- "The runc version meets the published security minimum." +- ), ++ "securityVersions.runc.status": ("The runc version meets the published security minimum."), + "securityVersions.docker.status": ( + "The Docker Engine version meets the published security minimum." + ), + "securityVersions.connectxFirmware.status": ( + "The ConnectX firmware meets the published security minimum." +- ), +- "securityVersions.dcgm.status": ( +- "The DCGM version meets the published security minimum." + ), ++ "securityVersions.dcgm.status": ("The DCGM version meets the published security minimum."), + "securityVersions.dcgmExporter.status": ( + "The DCGM Exporter version meets the published security minimum." + ), +@@ -91,9 +87,7 @@ + "security.fragnesia.status": ( + "The running kernel does not match the affected Fragnesia or Dirty Frag ranges." + ), +- "containers.nvidiaContainerToolkit": ( +- "The worker has NVIDIA Container Toolkit support." +- ), ++ "containers.nvidiaContainerToolkit": ("The worker has NVIDIA Container Toolkit support."), + "containers.workerCheckOk": ( + "The worker container check completed, so its runtime observations are valid." + ), +@@ -106,9 +100,7 @@ + "containers.singularity": "Singularity or Apptainer is installed on the worker.", + "software.nccl.installed": "NCCL is installed.", + "software.perf.installed": "perf (Linux performance counters) is installed.", +- "software.perf.perfEventParanoid": ( +- "perf_event_paranoid permits unprivileged perf profiling." +- ), ++ "software.perf.perfEventParanoid": ("perf_event_paranoid permits unprivileged perf profiling."), + "software.perf.kptrRestrict": "Kernel symbols are visible to perf.", + "gpus.pcieAcs.enabled": ( + "PCIe ACS is disabled on the GPU-NIC path, so peer-to-peer traffic is not redirected." +@@ -117,19 +109,13 @@ + "NCCL relies on auto-configuration without an overriding /etc/nccl.conf." + ), + "healthChecks.dcgmInstalled": "DCGM is installed.", +- "healthChecks.dcgmSlurm": ( +- "DCGM health checks are wired into the Slurm HealthCheckProgram." +- ), ++ "healthChecks.dcgmSlurm": ("DCGM health checks are wired into the Slurm HealthCheckProgram."), + "access.sudoAvailable": "Passwordless sudo is available to the audited user.", + "access.userManagement": ( + "User and group management commands (useradd / groupadd) are usable." +- ), +- "access.sshToComputeNodes": ( +- "Compute nodes are reachable over passwordless SSH." +- ), +- "access.externalIdp.detected": ( +- "An external identity provider integration was detected." + ), ++ "access.sshToComputeNodes": ("Compute nodes are reachable over passwordless SSH."), ++ "access.externalIdp.detected": ("An external identity provider integration was detected."), + "access.slurmCommandsOk": ( + "Core Slurm commands (sinfo / squeue / scontrol / sbatch / srun) are functional." + ), +@@ -161,9 +147,7 @@ + "kubelet_cpu_manager_policy.status": ( + "Kubelet uses the static CPU Manager policy for GPU workload isolation." + ), +- "vm_iommu.status": ( +- "GPU and RDMA devices use the expected IOMMU passthrough configuration." +- ), ++ "vm_iommu.status": ("GPU and RDMA devices use the expected IOMMU passthrough configuration."), + "arm_smmu_virtualization.status": ( + "The Arm virtual machine exposes the required virtualization for the SMMU command queue." + ), +@@ -173,9 +157,7 @@ + "nccl_ib_qps.status": ( + "The NCCL InfiniBand queue-pair setting is suitable for the detected fabric." + ), +- "networking.topologyConfigured": ( +- "The scheduler has a topology configuration." +- ), ++ "networking.topologyConfigured": ("The scheduler has a topology configuration."), + "networking.hcaNamingValid": ( + "The HCA and NIC device names use the expected naming convention." + ), +@@ -228,8 +210,7 @@ + for part in parts: + words.append(re.sub(r"(?<=[a-z0-9])(?=[A-Z])", " ", part).replace("_", " ")) + title = " / ".join( +- " ".join(token[:1].upper() + token[1:] for token in word.split()) +- for word in words ++ " ".join(token[:1].upper() + token[1:] for token in word.split()) for word in words + ) + for source, replacement in { + "Hbm": "HBM", +@@ -269,12 +250,10 @@ + + _REPRODUCTION_COMMANDS = { + "securityVersions.nvidiaDriver.status": ( +- "on the audited worker: `nvidia-smi --query-gpu=driver_version " +- "--format=csv,noheader`" ++ "on the audited worker: `nvidia-smi --query-gpu=driver_version --format=csv,noheader`" + ), + "securityVersions.nvidiaContainerToolkit.status": ( +- "on the audited worker: `nvidia-container-toolkit --version` or " +- "`nvidia-ctk --version`" ++ "on the audited worker: `nvidia-container-toolkit --version` or `nvidia-ctk --version`" + ), + "securityVersions.cudaToolkit.status": ( + "on the audited worker: `nvcc --version` (installed toolkit); " +@@ -293,7 +272,7 @@ + ), + "securityVersions.connectxFirmware.status": ( + "on every GPU worker: `for device in /sys/class/infiniband/*; do " +- "printf '%s ' \"$(basename \"$device\")\"; cat \"$device/fw_ver\"; done`" ++ 'printf \'%s \' "$(basename "$device")"; cat "$device/fw_ver"; done`' + ), + "securityVersions.dcgmExporter.status": ( + "from the Kubernetes control plane: `kubectl get pods -A -o " +@@ -320,8 +299,7 @@ + "`lsmod | grep '^gdrdrv'` and `test -c /dev/gdrdrv`" + ), + "kubelet_cpu_manager_policy.status": ( +- "on every GPU worker: `sudo jq -r .policyName " +- "/var/lib/kubelet/cpu_manager_state`" ++ "on every GPU worker: `sudo jq -r .policyName /var/lib/kubelet/cpu_manager_state`" + ), + "networking.topologyConfigured": ( + "from the Kubernetes control plane: inspect `kubectl get nodes " +@@ -330,7 +308,7 @@ + ), + "software.cudaVisibleDevicesStatus": ( + "in a standard CUDA pod with one GPU: `printf '%s\\n' " +- "\"$CUDA_VISIBLE_DEVICES\" \"$NVIDIA_VISIBLE_DEVICES\"; nvidia-smi`" ++ '"$CUDA_VISIBLE_DEVICES" "$NVIDIA_VISIBLE_DEVICES"; nvidia-smi`' + ), + "software.ncu.profilingEnabled": ( + "in a CUDA development pod with one GPU: run `ncu --query-metrics`, " +@@ -339,9 +317,7 @@ + "software.perf.perfEventParanoid": ( + "on the audited worker: `sysctl kernel.perf_event_paranoid`" + ), +- "software.perf.kptrRestrict": ( +- "on the audited worker: `sysctl kernel.kptr_restrict`" +- ), ++ "software.perf.kptrRestrict": ("on the audited worker: `sysctl kernel.kptr_restrict`"), + "ufm-profile": ( + "provider-side verification: export the UFM security configuration and " + "verify randomized management keys and rate limits for the tenant fabric" +@@ -387,6 +363,7 @@ + } + ) + ++ + def _nested(data: dict[str, Any], path: str) -> Any: + current: Any = data + for part in path.split("."): +@@ -411,9 +388,7 @@ + } + mode = controller.get("platformMode") if isinstance(controller, dict) else None + bluefield_present = bluefield_present or str(mode).strip().lower() in {"nic", "dpu"} +- graded_version = ( +- controller.get("gradedVersion") if isinstance(controller, dict) else None +- ) ++ graded_version = controller.get("gradedVersion") if isinstance(controller, dict) else None + bluefield_present = bluefield_present or str(graded_version).strip().lower() not in { + "", + "none", +@@ -470,12 +445,12 @@ + else: + names.append(str(device)) + return ( +- f"valid={str(value).lower()}; " +- f"devices={', '.join(names) if names else 'none reported'}" ++ f"valid={str(value).lower()}; devices={', '.join(names) if names else 'none reported'}" + ) + if key == "security.januscape.status": + januscape = _nested(audit, "security.januscape") + if isinstance(januscape, dict): ++ + def display(field: str) -> str: + observed = januscape.get(field, "unknown") + if isinstance(observed, bool): +@@ -503,11 +478,7 @@ + fragnesia = _nested(audit, "security.fragnesia") + kernel = _nested(audit, "security.guestKernel") + if isinstance(fragnesia, dict): +- running = ( +- kernel.get("running", "unknown") +- if isinstance(kernel, dict) +- else "unknown" +- ) ++ running = kernel.get("running", "unknown") if isinstance(kernel, dict) else "unknown" + return ( + f"observed version {running}; minimum version " + f"{fragnesia.get('ubuntuNoblePackageMinimum', 'unknown')}" +@@ -536,9 +507,7 @@ + return observation + + source_path = _VERSION_SOURCE_KEYS.get(key) +- if source_path is None and key.startswith("securityVersions.") and key.endswith( +- ".status" +- ): ++ if source_path is None and key.startswith("securityVersions.") and key.endswith(".status"): + source_path = key.rsplit(".", 1)[0] + if source_path is None: + return value +@@ -567,8 +536,7 @@ + for entry in devices + if isinstance(entry, dict) + and not ( +- _field(entry, "device") == "unknown" +- and _field(entry, "version") == "unknown" ++ _field(entry, "device") == "unknown" and _field(entry, "version") == "unknown" + ) + ] + if readings: +@@ -608,14 +576,8 @@ + if minimum_url: + references.append((minimum_links.REFERENCE_LABEL, minimum_url)) + if finding is not None: +- references.extend( +- (cve, f"https://nvd.nist.gov/vuln/detail/{cve}") +- for cve in finding.cves +- ) +- references.extend( +- findings._advisory_link(advisory) +- for advisory in finding.advisories +- ) ++ references.extend((cve, f"https://nvd.nist.gov/vuln/detail/{cve}") for cve in finding.cves) ++ references.extend(findings._advisory_link(advisory) for advisory in finding.advisories) + references = [reference for reference in references if reference[1]] + return _with_criteria_reference(key, tuple(dict.fromkeys(references))) + +@@ -626,10 +588,7 @@ + # Not-applicable values never reach this helper: `evaluate` skips them + # through `_is_not_applicable` before it classifies a value as passing. + if normalized == "unknown": +- return ( +- "The collector could not verify this check. Treat this value as " +- "unverified." +- ) ++ return "The collector could not verify this check. Treat this value as unverified." + if key == "security.januscape.status": + if normalized == "not_exposed": + return "The audit did not find Januscape exposure on this target." +@@ -639,10 +598,7 @@ + ) + if key == "securityVersions.virtioNetBluefield.exposure": + if normalized == "none": +- return ( +- "The audit did not find an active BlueField VIRTIO-Net controller " +- "exposure." +- ) ++ return "The audit did not find an active BlueField VIRTIO-Net controller exposure." + if normalized == "live": + return ( + "The BlueField VIRTIO-Net controller is active. Its separate " +@@ -660,9 +616,7 @@ + paths.append("the NVIDIA open kernel modules") + if paths: + return ( +- "The audit found a modern GPUDirect RDMA path through " +- + " and ".join(paths) +- + "." ++ "The audit found a modern GPUDirect RDMA path through " + " and ".join(paths) + "." + ) + return _PASS_ASSESSMENTS.get( + key, +@@ -684,6 +638,7 @@ + normalized = normalized.replace(separator, "_") + return normalized + ++ + # Counter-access results that mean the NCU check never reached a verdict, as + # opposed to "ncu ran (or was looked for) and is not there". Mirrors + # audit_findings._ncu_install_verdict_available, which suppresses the +@@ -745,9 +700,7 @@ + "software.ncu.profilingEnabled": _DependentCheck( + "software.ncu.installed", _ncu_absence_is_conclusive + ), +- "software.perf.perfEventParanoid": _DependentCheck( +- "software.perf.installed" +- ), ++ "software.perf.perfEventParanoid": _DependentCheck("software.perf.installed"), + "software.perf.kptrRestrict": _DependentCheck("software.perf.installed"), + "healthChecks.dcgmSlurm": _DependentCheck( + "healthChecks.dcgmInstalled", +@@ -810,11 +763,7 @@ + if check.id not in _SECURITY_EXTENSION_IDS: + continue + harnesses = _SECURITY_EXTENSION_HARNESSES.get(check.id) +- if ( +- harness is not None +- and harnesses is not None +- and harness not in harnesses +- ): ++ if harness is not None and harnesses is not None and harness not in harnesses: + checks.append( + AuditCheck( + check.id, +@@ -838,10 +787,7 @@ + "passthrough isolation is controlled by the physical host.", + references=_with_criteria_reference( + check.id, +- tuple( +- (reference.label, reference.url) +- for reference in check.references +- ), ++ tuple((reference.label, reference.url) for reference in check.references), + ), + ) + ) +@@ -857,10 +803,7 @@ + check.remediation, + _with_criteria_reference( + check.id, +- tuple( +- (reference.label, reference.url) +- for reference in check.references +- ), ++ tuple((reference.label, reference.url) for reference in check.references), + ), + _reproduction(check.id), + ) +@@ -1018,11 +961,7 @@ + continue + unverified = str(effective).strip().lower() == "unknown" + matched = [rule for rule in rules if rule.failing(effective)] +- failing = [ +- rule +- for rule in matched +- if rule.guard is None or rule.guard(audit) +- ] ++ failing = [rule for rule in matched if rule.guard is None or rule.guard(audit)] + if matched and not failing: + # The finding rules suppress a matched failing value for one of + # three reasons, and the report must not present them alike: the +@@ -1036,9 +975,7 @@ + # check either: an operator comparing profiles must see the same + # check set. + classify = getattr(findings, "classify_suppression", None) +- kind, reason = ( +- classify(key, audit) if classify is not None else ("unverified", "") +- ) ++ kind, reason = classify(key, audit) if classify is not None else ("unverified", "") + if kind == getattr(findings, "VERIFIED_KIND", "verified_ok"): + checks.append( + AuditCheck( +@@ -1053,8 +990,10 @@ + ) + continue + if kind == getattr(findings, "NOT_APPLICABLE_KIND", "not_applicable"): +- assessment = f"Not applicable: {reason}" if reason else ( +- "Not applicable to this hardware or configuration." ++ assessment = ( ++ f"Not applicable: {reason}" ++ if reason ++ else ("Not applicable to this hardware or configuration.") + ) + else: + assessment = reason or ( +@@ -1138,11 +1077,7 @@ + for status in (PASS, WARNING, FAIL, SKIPPED) + } + command = command or (f"cmax audit {category}" if category else "cmax audit") +- title = ( +- f"# ClusterMAX {category} audit report" +- if category +- else "# ClusterMAX audit report" +- ) ++ title = f"# ClusterMAX {category} audit report" if category else "# ClusterMAX audit report" + lines = [report_style.paint(title, "bold", color=color)] + visible = checks + if visible: +@@ -1172,17 +1107,13 @@ + status=check.status, + assessment=assessment, + details=details, +- recommendation=( +- check.recommendation if verbosity >= 3 else "" +- ), ++ recommendation=(check.recommendation if verbosity >= 3 else ""), + references=( + minimum_links.canonical_references(check.references) + if verbosity >= 2 + else tuple( + reference +- for reference in minimum_links.canonical_references( +- check.references +- ) ++ for reference in minimum_links.canonical_references(check.references) + if reference[0] == minimum_links.REFERENCE_LABEL + ) + ), +@@ -1197,15 +1128,12 @@ + ( + report_style.count(f"{counts[FAIL]} failed", FAIL, color=color), + report_style.count( +- f"{counts[WARNING]} warning" +- f"{'' if counts[WARNING] == 1 else 's'}", ++ f"{counts[WARNING]} warning{'' if counts[WARNING] == 1 else 's'}", + WARNING, + color=color, + ), + report_style.count(f"{counts[PASS]} passed", PASS, color=color), +- report_style.count( +- f"{counts[SKIPPED]} skipped", SKIPPED, color=color +- ), ++ report_style.count(f"{counts[SKIPPED]} skipped", SKIPPED, color=color), + ) + ) + lines.extend(("", summary)) + +--- cmax/audit_review.py ++++ cmax/audit_review.py +@@ -50,9 +50,7 @@ + if path.is_file() and path.suffix.lower() != ".json": + candidates.append(path) + if values is not None: +- candidates.extend( +- (values.with_name("audit.out"), values.parent / "logs" / "audit.out") +- ) ++ candidates.extend((values.with_name("audit.out"), values.parent / "logs" / "audit.out")) + if path.is_dir(): + candidates.extend( + ( +@@ -71,13 +69,9 @@ + if not source.exists(): + raise AuditReviewError(f"audit source does not exist: {source}") + values = next((item for item in _values_candidates(source) if item.is_file()), None) +- raw = next( +- (item for item in _raw_candidates(source, values) if item.is_file()), None +- ) ++ raw = next((item for item in _raw_candidates(source, values) if item.is_file()), None) + if values is None and raw is None: +- raise AuditReviewError( +- f"no audit.values.json or audit.out found at {source}" +- ) ++ raise AuditReviewError(f"no audit.values.json or audit.out found at {source}") + return AuditArtifacts(values=values, raw=raw) + + +@@ -112,8 +106,7 @@ + files = [path.resolve() for path in candidates if path.is_file()] + if not files: + raise AuditReviewError( +- "no saved audit found under CLUSTERMAX_RUNS_ROOT, runs/, or " +- "~/.clustermax/audit" ++ "no saved audit found under CLUSTERMAX_RUNS_ROOT, runs/, or ~/.clustermax/audit" + ) + # ClusterMAX timestamp directory names sort in chronological order. + return resolve_source(max(files, key=_timestamp_key)) +@@ -145,10 +138,7 @@ + def _show_check(check, number: int) -> str: + observed = json.dumps(check.observed, indent=2, sort_keys=True, default=str) + return ( +- f"{number}. {check.title}\n" +- f"status: {check.status}\n" +- f"key: {check.key}\n" +- f"observed: {observed}" ++ f"{number}. {check.title}\nstatus: {check.status}\nkey: {check.key}\nobserved: {observed}" + ) + + +@@ -186,9 +176,7 @@ + if name == "summary": + if artifacts.values is None: + return "Structured audit values are unavailable. Use 'raw'.", False +- return audit_report.format_report( +- checks, color=color, command="cmax audit review" +- ), False ++ return audit_report.format_report(checks, color=color, command="cmax audit review"), False + if name == "all": + return _check_list(checks), False + if name == "passes": +@@ -196,8 +184,7 @@ + return _check_list(checks, selected), False + if name == "paths": + return ( +- f"values: {artifacts.values or 'not found'}\n" +- f"raw: {artifacts.raw or 'not found'}" ++ f"values: {artifacts.values or 'not found'}\nraw: {artifacts.raw or 'not found'}" + ), False + if name == "show": + if not args: +@@ -211,8 +198,7 @@ + matches = [ + i + for i, check in enumerate(checks) +- if query.lower() in check.key.lower() +- or query.lower() in check.title.lower() ++ if query.lower() in check.key.lower() or query.lower() in check.title.lower() + ] + if len(matches) == 1: + index = matches[0] +@@ -258,10 +244,7 @@ + ) -> int: + """Render one saved audit and optionally start the review prompt.""" + output = output or sys.stdout +- color = bool( +- getattr(output, "isatty", lambda: False)() +- and "NO_COLOR" not in os.environ +- ) ++ color = bool(getattr(output, "isatty", lambda: False)() and "NO_COLOR" not in os.environ) + checks = load_checks(artifacts, rules_root) + print(f"Audit review: {artifacts.source}", file=output) + if artifacts.values is not None: +@@ -283,9 +266,7 @@ + raw_text, _ = execute_command("raw", artifacts, checks, color=color) + print(raw_text, file=output) + for command in commands or []: +- rendered, should_exit = execute_command( +- command, artifacts, checks, color=color +- ) ++ rendered, should_exit = execute_command(command, artifacts, checks, color=color) + if rendered: + print(rendered, file=output) + if should_exit: +@@ -306,9 +287,7 @@ + except (EOFError, KeyboardInterrupt): + print(file=output) + return 0 +- rendered, should_exit = execute_command( +- command, artifacts, checks, color=color +- ) ++ rendered, should_exit = execute_command(command, artifacts, checks, color=color) + if rendered: + print(rendered, file=output) + if should_exit: + +--- cmax/banner.py ++++ cmax/banner.py +@@ -109,10 +109,7 @@ + found: list[str] = [] + for letter, rows in sorted(self.glyphs.items()): + if len(rows) != self.height: +- found.append( +- f"{self.name} {letter}: has {len(rows)} rows, " +- f"needs {self.height}" +- ) ++ found.append(f"{self.name} {letter}: has {len(rows)} rows, needs {self.height}") + for index, row in enumerate(rows): + if len(row) != self.width: + found.append( +@@ -134,10 +131,7 @@ + if missing: + raise KeyError(f"{self.name} has no glyph for {missing!r}") + pad = " " * self.gap +- return [ +- pad.join(self.glyphs[letter][row] for letter in word) +- for row in range(self.height) +- ] ++ return [pad.join(self.glyphs[letter][row] for letter in word) for row in range(self.height)] + + + # Solid block letterforms, six rows tall. A vertical stroke is two columns and a +@@ -326,9 +320,7 @@ + FONTS = (BLOCK_FONT, ASCII_FONT) + + +-def select_font( +- *, width: int, unicode_ok: bool, word: str = WORD +-) -> Font | None: ++def select_font(*, width: int, unicode_ok: bool, word: str = WORD) -> Font | None: + """The widest font this terminal can both encode and fit, or None. + + None means every font is wider than the terminal, and the caller falls back +@@ -354,10 +346,7 @@ + """ + found = logo.defects() + if logo.HEIGHT != BLOCK_FONT.height: +- found.append( +- f"logo is {logo.HEIGHT} rows and the block font is " +- f"{BLOCK_FONT.height}" +- ) ++ found.append(f"logo is {logo.HEIGHT} rows and the block font is {BLOCK_FONT.height}") + return found + + +@@ -413,20 +402,14 @@ + if font is None: + return [resolved.paint(word, *GOLD.codes(shade))] + +- with_logo = ( +- font is BLOCK_FONT +- and resolved.unicode +- and width >= full_width(word) +- ) ++ with_logo = font is BLOCK_FONT and resolved.unicode and width >= full_width(word) + art_width = full_width(word) if with_logo else font.measure(word) + margin = " " * ((width - art_width) // 2) + + lines = [] + for index, row in enumerate(font.compose(word)): + trimmed = row.rstrip() +- painted = ( +- resolved.paint(trimmed, *GOLD.codes(shade)) if trimmed else "" +- ) ++ painted = resolved.paint(trimmed, *GOLD.codes(shade)) if trimmed else "" + if with_logo: + mark = logo.paint_row(logo.ROWS[index], resolved, shade) + line = (margin + mark + " " * LOGO_GAP + painted).rstrip() + +--- cmax/cli.py ++++ cmax/cli.py +@@ -73,9 +73,7 @@ + ) + + +-def _add_full_audit_profile_options( +- parser: argparse.ArgumentParser, *, allow_target: bool +-) -> None: ++def _add_full_audit_profile_options(parser: argparse.ArgumentParser, *, allow_target: bool) -> None: + parser.add_argument( + "-s", + "--show", +@@ -129,9 +127,7 @@ + help=AUDIT_COMMAND_HELP, + description=AUDIT_COMMAND_HELP, + ) +- audit_commands = audit.add_subparsers( +- dest="profile", title="audit profiles and targets" +- ) ++ audit_commands = audit.add_subparsers(dest="profile", title="audit profiles and targets") + security = audit_commands.add_parser( + "security", + help="Run the focused, read-only security report.", +@@ -170,8 +166,7 @@ + nargs="?", + metavar="PATH", + help=( +- "Audit directory, audit.values.json, or audit.out. " +- "Defaults to the newest saved audit." ++ "Audit directory, audit.values.json, or audit.out. Defaults to the newest saved audit." + ), + ) + review.add_argument( +@@ -455,9 +450,7 @@ + artifacts = ( + audit_review.resolve_source(Path(args.review_source)) + if args.review_source +- else audit_review.find_latest( +- [Path.cwd(), *Path.cwd().parents, rules_root] +- ) ++ else audit_review.find_latest([Path.cwd(), *Path.cwd().parents, rules_root]) + ) + return audit_review.run( + artifacts, + +--- cmax/criteria_links.py ++++ cmax/criteria_links.py +@@ -12,12 +12,9 @@ + "firmware-and-host-packages" + ) + _SECURITY_CONTAINER_TOOLKIT = ( +- "security-updated-nvidia-container-toolkit-preventing-cve-2024-0132-and-" +- "related-vulnerabilities" +-) +-_SECURITY_ESCALATION = ( +- "security-protection-against-container-escalation-vulnerabilities" ++ "security-updated-nvidia-container-toolkit-preventing-cve-2024-0132-and-related-vulnerabilities" + ) ++_SECURITY_ESCALATION = "security-protection-against-container-escalation-vulnerabilities" + _SECURITY_ISOLATION = ( + "security-strong-isolation-between-tenants-not-namespaces-or-container-" + "only-isolation-i-e-use-vcluster-private-nodes-instead-of-vcluster-shared-" +@@ -28,21 +25,13 @@ + "providing-a-comprehensive-set-of-security-features-required-for-secure-" + "multi-tenant-cloud-environments" + ) +-_LIFECYCLE_GPU_DIRECT = ( +- "lifecycle-out-of-the-box-gpudirect-rdma-between-nic-and-gpu-setup" +-) +-_LIFECYCLE_DRIVERS = ( +- "lifecycle-out-of-the-box-ib-rocev2-and-nvidia-drivers-configuration" +-) ++_LIFECYCLE_GPU_DIRECT = "lifecycle-out-of-the-box-gpudirect-rdma-between-nic-and-gpu-setup" ++_LIFECYCLE_DRIVERS = "lifecycle-out-of-the-box-ib-rocev2-and-nvidia-drivers-configuration" + _ORCHESTRATION_USERS = "orchestration-easy-process-for-adding-new-cluster-users" + _ORCHESTRATION_RBAC = "orchestration-rbac-and-sso-implementation" + _ORCHESTRATION_SSH = "orchestration-no-ssh-key-copying-required" +-_ORCHESTRATION_CUDA = ( +- "orchestration-cuda-visible-devices-properly-configured" +-) +-_ORCHESTRATION_TOPOLOGY = ( +- "orchestration-out-of-the-box-slurm-topology-configuration" +-) ++_ORCHESTRATION_CUDA = "orchestration-cuda-visible-devices-properly-configured" ++_ORCHESTRATION_TOPOLOGY = "orchestration-out-of-the-box-slurm-topology-configuration" + _ORCHESTRATION_MODULES = "orchestration-slurm-modules-availability" + _ORCHESTRATION_PYXIS = "orchestration-pyxis-container-plugin-support" + _STORAGE_RWX = ( +@@ -51,18 +40,13 @@ + ) + _NETWORKING_TOPOLOGY = "networking-out-of-the-box-slurm-topology-configuration" + _NETWORKING_NCCL_AUTOCONFIG = ( +- "networking-nccl-min-nchannels-nccl-proto-nccl-algo-not-set-auto-" +- "configuration" ++ "networking-nccl-min-nchannels-nccl-proto-nccl-algo-not-set-auto-configuration" + ) + _MONITORING_NCU = "monitoring-ncu-profiling-available-for-all-users" + _MONITORING_HEALTH = "monitoring-automated-active-and-passive-health-checks" + _MONITORING_GRAFANA = "monitoring-out-of-the-box-detailed-managed-grafana" +-_MONITORING_DCGM = ( +- "monitoring-dcgm-health-checks-plugged-into-the-slurm-healthcheckprogram" +-) +-_MONITORING_SACCT = ( +- "monitoring-sacct-integration-for-job-accounting-and-resource-utilization" +-) ++_MONITORING_DCGM = "monitoring-dcgm-health-checks-plugged-into-the-slurm-healthcheckprogram" ++_MONITORING_SACCT = "monitoring-sacct-integration-for-job-accounting-and-resource-utilization" + + + # Some release checks are more granular than the public requirements. Those + +--- cmax/minimum_refresh.py ++++ cmax/minimum_refresh.py +@@ -90,8 +90,7 @@ + ) + DOCKER_RELEASE_NOTES_PAGE = "https://docs.docker.com/engine/release-notes/{major}/" + DOCKER_DOCS_CONTENTS = ( +- "https://api.github.com/repos/docker/docs/contents" +- "/content/manuals/engine/release-notes" ++ "https://api.github.com/repos/docker/docs/contents/content/manuals/engine/release-notes" + ) + NVHPC_RELEASES_PAGE = "https://developer.nvidia.com/hpc-sdk/releases" + NVHPC_RELEASE_RE = re.compile(r"\bHPC SDK\s+([0-9]{2}\.[0-9]{1,2})\b", re.IGNORECASE) +@@ -199,9 +198,7 @@ + # OSV pages large result sets. Follow every page, and stop rather than build a + # ladder from a truncated advisory set. + OSV_MAX_PAGES = 20 +-RUNC_ADVISORY = ( +- "https://github.com/opencontainers/runc/security/advisories/GHSA-9493-h29p-rfm2" +-) ++RUNC_ADVISORY = "https://github.com/opencontainers/runc/security/advisories/GHSA-9493-h29p-rfm2" + + NVIDIA_DRIVER_RELEASE_REPO = "NVIDIA/open-gpu-kernel-modules" + NVIDIA_CONTAINER_TOOLKIT_RELEASE_REPO = "NVIDIA/nvidia-container-toolkit" +@@ -233,9 +230,7 @@ + # One release on a Docker release-note page: a `## X.Y.Z` heading, a + # release-date shortcode below it, and a `### Security` subsection when the + # release ships a security fix. +-DOCKER_RELEASE_HEADING = re.compile( +- r"^## +(\d+\.\d+\.\d+(?:-rc\.?\d+)?)\s*$", re.MULTILINE +-) ++DOCKER_RELEASE_HEADING = re.compile(r"^## +(\d+\.\d+\.\d+(?:-rc\.?\d+)?)\s*$", re.MULTILINE) + DOCKER_RELEASE_DATE = re.compile(r'release-date\s+date="(\d{4}-\d{2}-\d{2})"') + DOCKER_SECURITY_HEADING = re.compile(r"^### +Security\s*$", re.MULTILINE) + DOCKER_NEXT_HEADING = re.compile(r"^#{2,3} ", re.MULTILINE) +@@ -252,9 +247,7 @@ + # above a minimum this series forces. + AMD_FEED = "amd-security-bulletin" + AMD_SECURITY_INDEX = "https://www.amd.com/en/resources/product-security.html" +-AMD_BULLETIN_PAGE = ( +- "https://www.amd.com/en/resources/product-security/bulletin/{sb_id}.html" +-) ++AMD_BULLETIN_PAGE = "https://www.amd.com/en/resources/product-security/bulletin/{sb_id}.html" + AMD_BULLETINS: tuple[str, ...] = ("amd-sb-6018", "amd-sb-6024", "amd-sb-6027") + + # An index row whose title matches this pattern concerns GPUs, so +@@ -288,73 +281,48 @@ + # AMD_BULLETINS forces, so tracking it cannot move any minimum up. + AMD_DEFERRED_BULLETINS: dict[str, str] = { + "amd-sb-1000": ( +- "Windows 10 graphics driver bulletin for client GPUs. It names no " +- "ROCm release." ++ "Windows 10 graphics driver bulletin for client GPUs. It names no ROCm release." + ), + "amd-sb-1029": ( +- "Client graphics driver bulletin from November 2022. It names no " +- "ROCm release." ++ "Client graphics driver bulletin from November 2022. It names no ROCm release." + ), + "amd-sb-6003": ( +- "Client graphics driver bulletin from November 2023. It names no " +- "ROCm release." ++ "Client graphics driver bulletin from November 2023. It names no ROCm release." + ), + "amd-sb-6005": ( + "Consolidated bulletin from August 2024 in the older transposed " + "table format. Its highest ROCm release is 6.3.2, below the tracked " + "minimums." + ), +- "amd-sb-6007": ( +- "Radeon Software Crimson bulletin for client GPUs. It names no ROCm " +- "release." +- ), +- "amd-sb-6008": ( +- "Consolidated bulletin from February 2025. It names no ROCm release." +- ), +- "amd-sb-6009": ( +- "Radeon kernel driver bulletin for client GPUs. It names no ROCm " +- "release." +- ), ++ "amd-sb-6007": ("Radeon Software Crimson bulletin for client GPUs. It names no ROCm release."), ++ "amd-sb-6008": ("Consolidated bulletin from February 2025. It names no ROCm release."), ++ "amd-sb-6009": ("Radeon kernel driver bulletin for client GPUs. It names no ROCm release."), + "amd-sb-6010": ( + "GPU memory leak bulletin in the older per-environment table format. " + "Its highest ROCm release is 6.3.1, below the tracked minimums." +- ), +- "amd-sb-6011": ( +- "WebGPU browser side-channel note. It names no ROCm release." + ), +- "amd-sb-6012": ( +- "Radeon DirectX 11 shader bulletin for client GPUs. It names no ROCm " +- "release." +- ), ++ "amd-sb-6011": ("WebGPU browser side-channel note. It names no ROCm release."), ++ "amd-sb-6012": ("Radeon DirectX 11 shader bulletin for client GPUs. It names no ROCm release."), + "amd-sb-6013": ( + "Uninitialized GPU register bulletin in the older per-environment " + "table format. Its highest ROCm release is 6.3.1, below the tracked " + "minimums." + ), + "amd-sb-6015": ( +- "Graphics driver installer bulletin for client GPUs. It names no " +- "ROCm release." +- ), +- "amd-sb-6016": ( +- "Client GPU bulletin. It names no ROCm release." +- ), +- "amd-sb-6019": ( +- "Cross-process GPU memory disclosure note. It names no ROCm release." ++ "Graphics driver installer bulletin for client GPUs. It names no ROCm release." + ), ++ "amd-sb-6016": ("Client GPU bulletin. It names no ROCm release."), ++ "amd-sb-6019": ("Cross-process GPU memory disclosure note. It names no ROCm release."), + "amd-sb-6021": ( + "Linux graphics driver bulletin in the older format. Its highest " + "ROCm release is 6.2, below the tracked minimums." + ), +- "amd-sb-6026": ( +- "GPU timing side-channel research note. It names no ROCm release." +- ), ++ "amd-sb-6026": ("GPU timing side-channel research note. It names no ROCm release."), + "amd-sb-6031": ( + "Device Metrics Exporter bulletin. The fix is an exporter release, " + "and the audit does not grade the exporter version." + ), +- "amd-sb-7049": ( +- "GPUHammer research note. It names no ROCm release." +- ), ++ "amd-sb-7049": ("GPUHammer research note. It names no ROCm release."), + } + + DEFAULT_RELATIVE_PATH = runtime_paths.MINIMUMS_TABLE_RELATIVE +@@ -716,8 +684,7 @@ + "feed": "nvidia-csaf", + "aId": a_id, + "title": doc["document"]["title"], +- "released": tracking.get("initial_release_date") +- or tracking["current_release_date"], ++ "released": tracking.get("initial_release_date") or tracking["current_release_date"], + "url": bulletin_blob_url(year, a_id), + } + +@@ -868,8 +835,7 @@ + "kind": "releaseLines", + "lines": lines, + "floorAvailability": { +- line: {**availability, "version": entry["fixed"]} +- for line, entry in lines.items() ++ line: {**availability, "version": entry["fixed"]} for line, entry in lines.items() + }, + "cves": [item["cve"] for item in doc.get("vulnerabilities", []) if item.get("cve")], + "advisory": NVIDIA_ADVISORY.format(a_id=a_id), +@@ -965,9 +931,7 @@ + return blocks + + +-def osv_package_vulns( +- package: str, ecosystem: str, fetch: Fetcher | None = None +-) -> list[dict]: ++def osv_package_vulns(package: str, ecosystem: str, fetch: Fetcher | None = None) -> list[dict]: + """Return every OSV advisory for a package, across every page. + + The query carries no version. A version-scoped query returns only the +@@ -1264,7 +1228,7 @@ + heading = DOCKER_SECURITY_HEADING.search(body) + if not heading: + return None +- rest = body[heading.end():] ++ rest = body[heading.end() :] + next_heading = DOCKER_NEXT_HEADING.search(rest) + return rest[: next_heading.start()] if next_heading else rest + +@@ -1282,7 +1246,7 @@ + headings = list(DOCKER_RELEASE_HEADING.finditer(text)) + for index, match in enumerate(headings): + end = headings[index + 1].start() if index + 1 < len(headings) else len(text) +- body = text[match.end():end] ++ body = text[match.end() : end] + date_match = DOCKER_RELEASE_DATE.search(body) + security = _docker_security_section(body) + releases.append( +@@ -1393,9 +1357,7 @@ + that moving policy reproducible for every audit that consumes it. + """ + text = _fetcher(fetch).get_text(NVHPC_RELEASES_PAGE) +- releases = sorted( +- set(NVHPC_RELEASE_RE.findall(text)), key=version_key, reverse=True +- ) ++ releases = sorted(set(NVHPC_RELEASE_RE.findall(text)), key=version_key, reverse=True) + if len(releases) < 2: + raise MinimumRefreshError( + f"nvhpc: found fewer than two HPC SDK releases on {NVHPC_RELEASES_PAGE}; " +@@ -1573,8 +1535,7 @@ + other_tables.append(" ".join(" ".join(row) for row in table)) + if not rows: + raise MinimumRefreshError( +- f"rocm: found no dated mitigation table on {url}; " +- f"the table format probably changed" ++ f"rocm: found no dated mitigation table on {url}; the table format probably changed" + ) + for body in other_tables: + if AMD_ROCM_STRAY.search(AMD_TRADEMARKS.sub(" ", body)): +@@ -1625,8 +1586,7 @@ + for row in parsed["rows"]: + if len(row) < 4: + raise MinimumRefreshError( +- f"rocm: a mitigation row on {url} has fewer than four " +- f"cells: {row!r}" ++ f"rocm: a mitigation row on {url} has fewer than four cells: {row!r}" + ) + program_cell, cve_cell, mitigation_cell, date_cell = row[:4] + mitigation = " ".join(AMD_TRADEMARKS.sub(" ", mitigation_cell).split()) +@@ -1691,9 +1651,7 @@ + "kind": "programMap", + "programs": {program: records[program]["version"] for program in ordered}, + "programSources": {program: records[program]["aId"] for program in ordered}, +- "programCves": { +- program: sorted(records[program]["cves"]) for program in ordered +- }, ++ "programCves": {program: sorted(records[program]["cves"]) for program in ordered}, + "floorAvailability": { + program: { + "aId": records[program]["aId"], +@@ -1776,9 +1734,7 @@ + continue + if kind == "distroPackages": + for selector, package in (block.get("packages") or {}).items(): +- availability = ( +- package.get("fixAvailability") if isinstance(package, dict) else None +- ) ++ availability = package.get("fixAvailability") if isinstance(package, dict) else None + released = _minimum_bulletin_released(block, str(selector)) + if isinstance(availability, dict) and released: + availability["bulletinReleased"] = released +@@ -1808,7 +1764,7 @@ + conservative availability date. A later editorial revision must not restart + the vendor window for an unchanged minimum. + """ +- old_components = ((existing or {}).get("components") or {}) ++ old_components = (existing or {}).get("components") or {} + for name, block in (doc.get("components") or {}).items(): + old = old_components.get(name) or {} + kind = block.get("kind") +@@ -1816,9 +1772,7 @@ + if block.get("minimum") == old.get("minimum") and _confirmed_availability( + old.get("fixAvailability") + ): +- block["fixAvailability"] = _preserved_availability( +- old["fixAvailability"], old +- ) ++ block["fixAvailability"] = _preserved_availability(old["fixAvailability"], old) + continue + keyed_fields = { + "branchMap": "branches", +@@ -1840,9 +1794,7 @@ + if kind == "releaseLines" and isinstance(minimum, dict) + else minimum == old_minimum + ) +- if same_minimum and _confirmed_availability( +- old_availability.get(key) +- ): ++ if same_minimum and _confirmed_availability(old_availability.get(key)): + current_availability[key] = _preserved_availability( + old_availability[key], old, str(key) + ) +@@ -1996,7 +1948,7 @@ + release lines) and the single-value `minimum` components. + """ + problems: list[str] = [] +- old_components = ((existing or {}).get("components") or {}) ++ old_components = (existing or {}).get("components") or {} + for name, block in doc.get("components", {}).items(): + payload = _payload(block) + old_block = old_components.get(name) or {} +@@ -2008,27 +1960,19 @@ + continue + kind = block.get("kind") + if kind == "minimum": +- problems.extend( +- _availability_problems(name, block.get("fixAvailability")) +- ) ++ problems.extend(_availability_problems(name, block.get("fixAvailability"))) + elif kind in {"branchMap", "trainMap", "programMap", "ladder", "releaseLines"}: + availability = block.get("floorAvailability") or {} + for key in payload: +- problems.extend( +- _availability_problems(f"{name}.{key}", availability.get(str(key))) +- ) ++ problems.extend(_availability_problems(f"{name}.{key}", availability.get(str(key)))) + elif kind == "distroPackages": + for key, entry in payload.items(): + if entry.get("fixed"): + problems.extend( +- _availability_problems( +- f"{name}.{key}", entry.get("fixAvailability") +- ) ++ _availability_problems(f"{name}.{key}", entry.get("fixAvailability")) + ) + if kind == "branchMap": +- problems.extend( +- _minimum_downgrades(name, "branch", payload, old_block.get("branches")) +- ) ++ problems.extend(_minimum_downgrades(name, "branch", payload, old_block.get("branches"))) + continue + if kind == "trainMap": + problems.extend( +@@ -2057,9 +2001,7 @@ + current = block.get("current") + minimum = block.get("minimum") + if not current or version_key(str(current)) < version_key(str(minimum)): +- problems.append( +- f"{name}: current release {current!r} is below minimum {minimum!r}" +- ) ++ problems.append(f"{name}: current release {current!r} is below minimum {minimum!r}") + old_minimum = old_block.get("minimum") + if old_minimum and version_key(str(minimum)) < version_key(str(old_minimum)): + problems.append( +@@ -2071,9 +2013,7 @@ + old_lines = old_block.get("lines") or {} + for key, entry in payload.items(): + if not entry.get("fixed"): +- problems.append( +- f"{name}.{key}: no fixed release from {_source_url(block)}" +- ) ++ problems.append(f"{name}.{key}: no fixed release from {_source_url(block)}") + problems.extend( + _minimum_downgrades( + name, +@@ -2326,9 +2266,7 @@ + "reason": reason, + } + ) +- return sorted( +- found, key=lambda item: (item["status"], item["year"], str(item["id"])) +- ) ++ return sorted(found, key=lambda item: (item["status"], item["year"], str(item["id"]))) + + + # --------------------------------------------------------------------------- +@@ -2405,9 +2343,7 @@ + if new: + print("Untracked bulletins that match a graded product line:") + for item in new: +- print( +- f" {item['id']} {item['product']}: {item['title']} {item['url']}" +- ) ++ print(f" {item['id']} {item['product']}: {item['title']} {item['url']}") + print( + f"{len(new)} unknown bulletin(s) match a graded product line. " + "Track each NVIDIA bulletin in BULLETINS in " + +--- cmax/minimum_sync.py ++++ cmax/minimum_sync.py +@@ -127,9 +127,7 @@ + if not isinstance(data, dict) or data.get("url") != published_url(): + return {} + return { +- key: str(value) +- for key, value in data.items() +- if key in {"etag", "lastModified"} and value ++ key: str(value) for key, value in data.items() if key in {"etag", "lastModified"} and value + } + + +@@ -182,8 +180,7 @@ + raise MinimumSyncError("the published table carries no components") + if parse_stamp(table.get("generated")) is None: + raise MinimumSyncError( +- f"the published table has no usable generated timestamp: " +- f"{table.get('generated')!r}" ++ f"the published table has no usable generated timestamp: {table.get('generated')!r}" + ) + return table + +@@ -217,9 +214,7 @@ + # authenticated. Check the address the body actually came from. + final = response.geturl() + if not final.lower().startswith("https://"): +- raise MinimumSyncError( +- f"{url} redirected to an address that is not https: {final}" +- ) ++ raise MinimumSyncError(f"{url} redirected to an address that is not https: {final}") + body = response.read(MAX_BYTES + 1) + fresh = { + key: value +@@ -237,8 +232,7 @@ + raise MinimumSyncError(f"{url} could not be read: {exc}") from exc + if len(body) > MAX_BYTES: + raise MinimumSyncError( +- f"{url} returned more than {MAX_BYTES} bytes, so it is not the " +- f"minimum table" ++ f"{url} returned more than {MAX_BYTES} bytes, so it is not the minimum table" + ) + return body, fresh + +@@ -353,8 +347,7 @@ + True, + path, + str(table.get("generated")), +- f"fetched the published minimum table (generated {table.get('generated')}) " +- f"to {path}.", ++ f"fetched the published minimum table (generated {table.get('generated')}) to {path}.", + ) + + + +--- cmax/progress.py ++++ cmax/progress.py +@@ -74,9 +74,7 @@ + # helpers are called at the start of a line, sometimes indented inside a + # conditional branch, and a label that itself contains a double quote is passed + # in single quotes. +-_CALL_SITE = re.compile( +- r"""^[ \t]*print_(header|section)[ \t]+(?:"([^"]*)"|'([^']*)')""", re.M +-) ++_CALL_SITE = re.compile(r"""^[ \t]*print_(header|section)[ \t]+(?:"([^"]*)"|'([^']*)')""", re.M) + + # The same two helpers as they appear in the collector's output: print_header + # frames its title between two rules of box characters, print_section wraps its +@@ -168,9 +166,7 @@ + return rendered + + +-def _terminal_hyperlink( +- text: str, url: str, *, prompt_toolkit: bool = False +-) -> str: ++def _terminal_hyperlink(text: str, url: str, *, prompt_toolkit: bool = False) -> str: + opening = f"\x1b]8;;{url}\x1b\\" + closing = "\x1b]8;;\x1b\\" + if prompt_toolkit: +@@ -187,12 +183,7 @@ + return _PROMPT_TOOLKIT_LINK_CLOSE + " " if prompt_toolkit else text + if "\x1b]8;;" in text: + if prompt_toolkit: +- return ( +- _PROMPT_TOOLKIT_LINK_CLOSE +- + text +- + _PROMPT_TOOLKIT_LINK_CLOSE +- + " " +- ) ++ return _PROMPT_TOOLKIT_LINK_CLOSE + text + _PROMPT_TOOLKIT_LINK_CLOSE + " " + return text + + rendered = _CVE_LINK.sub( +@@ -239,12 +230,7 @@ + # prompt-toolkit can omit a zero-width escape at the end of a row + # because no terminal cell owns it. Put the close before a real space, + # and close again at column zero so a malformed prior row cannot leak. +- return ( +- _PROMPT_TOOLKIT_LINK_CLOSE +- + rendered +- + _PROMPT_TOOLKIT_LINK_CLOSE +- + " " +- ) ++ return _PROMPT_TOOLKIT_LINK_CLOSE + rendered + _PROMPT_TOOLKIT_LINK_CLOSE + " " + return rendered + + +@@ -325,9 +311,7 @@ + # Standalone and Kubernetes use thin wrappers that select the harness and + # exec the shared focused security collector. Follow that local target for + # planning so their progress bars have the same real phases as Slurm. +- wrapper = re.search( +- r'exec\s+bash\s+"\$WORKLOAD_DIR/([^"/]+\.sh)"', text +- ) ++ wrapper = re.search(r'exec\s+bash\s+"\$WORKLOAD_DIR/([^"/]+\.sh)"', text) + if wrapper is not None: + target = script.parent / wrapper.group(1) + if target.is_file() and target != script: +@@ -340,9 +324,7 @@ + if kind == "header": + group = label + continue +- steps.append( +- Step(group=group, label=_display_label(label), pattern=_label_pattern(label)) +- ) ++ steps.append(Step(group=group, label=_display_label(label), pattern=_label_pattern(label))) + return steps + + +@@ -710,9 +692,7 @@ + # tests from more than one group at the same time, and an operator who + # can see only one of them cannot tell what the allocation is doing. + expanded = { +- title +- for title, steps in groups +- if any(step.state == RUNNING for step in steps) ++ title for title, steps in groups if any(step.state == RUNNING for step in steps) + } + if not expanded and progress.current_group: + expanded = {progress.current_group} +@@ -781,10 +761,7 @@ + { + "kind": "note", + "lines": [ +- " " +- + theme.paint( +- f"{theme.ellipsis} {hidden} more group(s) to run", "90" +- ) ++ " " + theme.paint(f"{theme.ellipsis} {hidden} more group(s) to run", "90") + ], + } + ) +@@ -842,10 +819,7 @@ + the characters that are about to be written. + """ + theme = self.theme +- return [ +- _clip(theme.encodable(line), self.width, theme.ellipsis) +- for line in lines +- ] ++ return [_clip(theme.encodable(line), self.width, theme.ellipsis) for line in lines] + + # -- pieces ----------------------------------------------------------- + +@@ -901,9 +875,8 @@ + parts.append(theme.paint(f"{not_run} not run", "90")) + tail = ", ".join(parts) + elif state == DONE and skipped: +- tail = ( +- theme.paint(f"{passed}/{total} passed, ", "90") +- + theme.paint(f"{skipped} skipped", "33") ++ tail = theme.paint(f"{passed}/{total} passed, ", "90") + theme.paint( ++ f"{skipped} skipped", "33" + ) + else: + duration = format_duration(_group_duration(steps, now)) +@@ -923,17 +896,11 @@ + # Only steps that actually ran are worth a line. A group whose plan holds + # conditional checks would otherwise fill the window with the branches + # this cluster never took. +- finished = [ +- step +- for step in steps[:first_running] +- if step.state in {DONE, FAILED} +- ] ++ finished = [step for step in steps[:first_running] if step.state in {DONE, FAILED}] + visible = finished[-2:] + [steps[index] for index in running] + lines: list[str] = [] + for step in visible: +- duration = ( +- format_duration(step.duration(now)) if step.state == RUNNING else "" +- ) ++ duration = format_duration(step.duration(now)) if step.state == RUNNING else "" + if step.state == RUNNING and step.ceiling: + duration = f"{duration} / {format_duration(step.ceiling)}" + title = step.title +@@ -942,9 +909,7 @@ + style = ("36",) if step.state == RUNNING else ("90",) + column = max(24, self.width - 22) + title = _fit_text(title, column - 9, theme.ellipsis) +- body = ( +- " " + theme.glyph(step.state, tick) + " " + theme.paint(title, *style) +- ) ++ body = " " + theme.glyph(step.state, tick) + " " + theme.paint(title, *style) + lines.append(_pad(body, column) + theme.paint(duration, "90")) + return lines + +@@ -967,9 +932,7 @@ + # Output collection and input handling run on different threads. Never + # wait for the producer's lock here: retain the last complete snapshot + # if it is busy, and keep that snapshot stable while the user scrolls. +- if not self.display._scroll_input_active() and self.display._lock.acquire( +- blocking=False +- ): ++ if not self.display._scroll_input_active() and self.display._lock.acquire(blocking=False): + try: + dirty_from = self.display._timeline_dirty_from + if dirty_from is not None: +@@ -979,17 +942,11 @@ + # or the appended rows before dirty_from are skipped. + sync_from = min(dirty_from, len(self._render_lines)) + del self._render_lines[sync_from:] +- self._render_lines.extend( +- self.display._timeline_rendered[sync_from:] +- ) ++ self._render_lines.extend(self.display._timeline_rendered[sync_from:]) + self.display._timeline_dirty_from = None +- elif len(self._render_lines) < len( +- self.display._timeline_rendered +- ): ++ elif len(self._render_lines) < len(self.display._timeline_rendered): + start = len(self._render_lines) +- self._render_lines.extend( +- self.display._timeline_rendered[start:] +- ) ++ self._render_lines.extend(self.display._timeline_rendered[start:]) + elif len(self._render_lines) > len(self.display._timeline_rendered): + self._render_lines[:] = self.display._timeline_rendered + finally: +@@ -1159,10 +1116,7 @@ + except (AttributeError, OSError, ValueError): + pass + self._dedicated_tui_input = ( +- terminal +- and os.name == "posix" +- and termios is not None +- and tty is not None ++ terminal and os.name == "posix" and termios is not None and tty is not None + ) + self._tui = terminal + # After close() the final checklist owns the screen. A sweep's worker +@@ -1183,21 +1137,15 @@ + self._started = True + if self._tui: + self._enter_tui() +- self._thread = threading.Thread( +- target=self._run_tui, name="cmax-progress", daemon=True +- ) ++ self._thread = threading.Thread(target=self._run_tui, name="cmax-progress", daemon=True) + self._thread.start() + self._tui_ready.wait(timeout=2.0) + if self._tui_error is not None: +- raise RuntimeError( +- "prompt-toolkit display failed to start" +- ) from self._tui_error ++ raise RuntimeError("prompt-toolkit display failed to start") from self._tui_error + else: + self.stream.write("\x1b[?25l") + self.stream.flush() +- self._thread = threading.Thread( +- target=self._loop, name="cmax-progress", daemon=True +- ) ++ self._thread = threading.Thread(target=self._loop, name="cmax-progress", daemon=True) + self._thread.start() + + def feed(self, line: str) -> None: +@@ -1292,9 +1240,7 @@ + self.stream.write(rendered + "\n") + self.stream.flush() + return +- self._capture_locked( +- owner or self._current_owner(), text, transient=transient +- ) ++ self._capture_locked(owner or self._current_owner(), text, transient=transient) + + def activate_output(self, owner: str) -> None: + """Select the lower-pane owner when a runner stage starts.""" +@@ -1384,9 +1330,7 @@ + (step.duration(now) or 0.0 for step in self.progress.running_steps()), + default=0.0, + ) +- self._next_interval = ( +- self.slow_interval if longest >= self.slow_after_s else self.interval +- ) ++ self._next_interval = self.slow_interval if longest >= self.slow_after_s else self.interval + + def _erase(self) -> None: + if not self._painted: +@@ -1446,9 +1390,7 @@ + ): + self._mark_timeline_dirty_locked(len(self._timeline) - 1) + self._timeline[-1] = ("output", lines[-1]) +- self._timeline_rendered[-1] = self._render_timeline_entry( +- self._timeline[-1] +- ) ++ self._timeline_rendered[-1] = self._render_timeline_entry(self._timeline[-1]) + else: + self._append_timeline_locked([("output", lines[-1])]) + self._tail_transient_owner = owner +@@ -1469,9 +1411,7 @@ + def _append_timeline_locked(self, entries: Iterable[tuple[str, str]]) -> None: + pending = list(entries) + self._timeline.extend(pending) +- self._timeline_rendered.extend( +- self._render_timeline_entry(entry) for entry in pending +- ) ++ self._timeline_rendered.extend(self._render_timeline_entry(entry) for entry in pending) + + def _render_timeline_entry(self, entry: tuple[str, str]) -> list: + return to_formatted_text(ANSI(self._timeline_line(entry))) +@@ -1518,9 +1458,7 @@ + (Keys.Home, "home"), + (Keys.End, "end"), + ): +- bindings.add(key)( +- lambda event, selected=action: self._scroll_tui(selected) +- ) ++ bindings.add(key)(lambda event, selected=action: self._scroll_tui(selected)) + + @bindings.add("c-c") + def interrupt(_event: object) -> None: +@@ -1777,9 +1715,7 @@ + top_height = self._tui_top_capacity() + self._tick += 1 + now = self._clock() +- lines = self.renderer.live( +- self.progress, now, self._tick, top_height + 2 +- )[:top_height] ++ lines = self.renderer.live(self.progress, now, self._tick, top_height + 2)[:top_height] + longest = max( + (step.duration(now) or 0.0 for step in self.progress.running_steps()), + default=0.0, +@@ -1797,25 +1733,17 @@ + return self._tui_header_cache + try: + owner = self._active_owner or self._current_owner() +- self._tui_header_cache = ANSI( +- self.theme_line(f" RUNNING · {owner} ", "36") +- ) ++ self._tui_header_cache = ANSI(self.theme_line(f" RUNNING · {owner} ", "36")) + return self._tui_header_cache + finally: + self._lock.release() + + def theme_line(self, text: str, *codes: str) -> str: +- return self.renderer.theme.paint( +- self.renderer.theme.encodable(text), *codes +- ) ++ return self.renderer.theme.paint(self.renderer.theme.encodable(text), *codes) + +- def _safe_line( +- self, text: str, *, color: bool, prompt_toolkit: bool = False +- ) -> str: ++ def _safe_line(self, text: str, *, color: bool, prompt_toolkit: bool = False) -> str: + rendered = self.renderer.theme.encodable(text.replace("\r", "")) +- rendered = color_status_text( +- rendered, color=color and self.renderer.theme.color +- ) ++ rendered = color_status_text(rendered, color=color and self.renderer.theme.color) + return linkify_report_text(rendered, prompt_toolkit=prompt_toolkit) + + def _timeline_line(self, entry: tuple[str, str]) -> str: +@@ -1843,9 +1771,7 @@ + for line in lines + ) + if owner not in _UNLABELED_OUTPUT_OWNERS and not owns_heading: +- self.stream.write( +- self.theme_line(f"── {owner} ──", "36", "1") + "\n" +- ) ++ self.stream.write(self.theme_line(f"── {owner} ──", "36", "1") + "\n") + for line in lines: + self.stream.write(self._safe_line(line, color=True) + "\n") + +@@ -1876,9 +1802,7 @@ + return + self._last = step + index = self.progress.completed + 1 +- self.stream.write( +- f"[{index:>3}/{self.progress.total}] {step.group} / {step.title}\n" +- ) ++ self.stream.write(f"[{index:>3}/{self.progress.total}] {step.group} / {step.title}\n") + self.stream.flush() + + def close(self, ok: bool = True) -> None: +@@ -1979,9 +1903,7 @@ + proc.wait() + raise + display.update( +- lambda tracker: getattr(tracker, "finish_collector", lambda _ok: None)( +- returncode == 0 +- ) ++ lambda tracker: getattr(tracker, "finish_collector", lambda _ok: None)(returncode == 0) + ) + return returncode, "".join(chunks) + +@@ -1995,9 +1917,7 @@ + """ + target = stream if stream is not None else sys.stdout + lines = output.splitlines() +- color = bool(getattr(target, "isatty", lambda: False)()) and ( +- "NO_COLOR" not in os.environ +- ) ++ color = bool(getattr(target, "isatty", lambda: False)()) and ("NO_COLOR" not in os.environ) + for index, line in enumerate(lines): + if marker in strip_ansi(line): + for tail in lines[index:]: +@@ -2010,9 +1930,7 @@ + return False + + +-def print_failure_tail( +- output: str, stream: TextIO | None = None, lines: int = 40 +-) -> None: ++def print_failure_tail(output: str, stream: TextIO | None = None, lines: int = 40) -> None: + """Show the end of a failed run, which the display otherwise swallowed.""" + target = stream if stream is not None else sys.stdout + captured = output.splitlines() + +--- cmax/report_style.py ++++ cmax/report_style.py +@@ -71,20 +71,12 @@ + if assessment: + lines.append(f" {assessment}") + for name, value in details: +- lines.append( +- f" {paint(name + ':', 'bold', color=color)} {value}" +- ) ++ lines.append(f" {paint(name + ':', 'bold', color=color)} {value}") + if recommendation: +- lines.append( +- f" {paint('Recommendation:', 'bold', color=color)} " +- f"{recommendation}" +- ) ++ lines.append(f" {paint('Recommendation:', 'bold', color=color)} {recommendation}") + refs = list(references) + if refs: + lines.append(f" {paint('References:', 'bold', color=color)}") + for index, (reference_label, url) in enumerate(refs, start=1): +- lines.append( +- f" [{index}] " +- f"{hyperlink(reference_label, url, color=color)}" +- ) ++ lines.append(f" [{index}] {hyperlink(reference_label, url, color=color)}") + return lines + +--- cmax/scripts/1-audit/audit_findings.py ++++ cmax/scripts/1-audit/audit_findings.py +@@ -153,7 +153,9 @@ + nvidia runtime; in either case there is nothing left for the provider to + attest, so the worker-check attestation note should stay silent. + """ +- return _is_true(nested_get(audit, "containers", "nvidiaContainerToolkit")) or _nvidia_runtime_configured(audit) ++ return _is_true( ++ nested_get(audit, "containers", "nvidiaContainerToolkit") ++ ) or _nvidia_runtime_configured(audit) + + + def _numeric_above(threshold: float) -> Callable[[Any], bool]: +@@ -374,9 +376,8 @@ + unproven one. + """ + status = nested_get(audit, "securityVersions", "virtioNetBluefield", "floorStatus") +- return ( +- str(status).strip().lower() == "fail" +- and not _minimum_grace_active(audit, "virtioNetBluefield") ++ return str(status).strip().lower() == "fail" and not _minimum_grace_active( ++ audit, "virtioNetBluefield" + ) + + +@@ -500,10 +501,7 @@ + """True only when a completed inventory found no BlueField device.""" + isolation = nested_get(audit, "securityVersions", "dpuHostIsolation") + if isinstance(isolation, dict): +- if ( +- isolation.get("scanComplete") is True +- and isolation.get("bluefieldPresent") is False +- ): ++ if isolation.get("scanComplete") is True and isolation.get("bluefieldPresent") is False: + return True + controller = nested_get(audit, "securityVersions", "virtioNetBluefield") + if isinstance(controller, dict): +@@ -680,8 +678,7 @@ + "ClusterMAX fault and needs no provider action (CVE-2026-65094)", + CONFIG, + _status_is("latent"), +- guard=lambda audit: _virtio_minimums_unavailable(audit) +- and _virtio_latent_exposure(audit), ++ guard=lambda audit: _virtio_minimums_unavailable(audit) and _virtio_latent_exposure(audit), + harnesses=SCALE_OUT_HARNESSES, + ), + Rule( +@@ -698,9 +695,11 @@ + # attestation note below: a provider asked to attest a version the audit + # already holds, for a fault on our side. An attestation cannot be + # graded until the table is repaired. +- guard=lambda audit: _virtio_minimums_unavailable(audit) +- and not _virtio_latent_exposure(audit) +- and not _bluefield_absence_proven(audit), ++ guard=lambda audit: ( ++ _virtio_minimums_unavailable(audit) ++ and not _virtio_latent_exposure(audit) ++ and not _bluefield_absence_proven(audit) ++ ), + harnesses=SCALE_OUT_HARNESSES, + ), + Rule( +@@ -714,9 +713,11 @@ + # above owns and which is not a question for the provider at all. This + # note owns every remaining unknown: an unreadable version on a DPU-mode + # or unknown-mode card, and a fleet a coverage gap left uncleared. +- guard=lambda audit: not _virtio_latent_exposure(audit) +- and not _virtio_minimums_unavailable(audit) +- and not _bluefield_absence_proven(audit), ++ guard=lambda audit: ( ++ not _virtio_latent_exposure(audit) ++ and not _virtio_minimums_unavailable(audit) ++ and not _bluefield_absence_proven(audit) ++ ), + harnesses=SCALE_OUT_HARNESSES, + ), + Rule( +@@ -777,9 +778,11 @@ + "NVIDIA Container Toolkit not installed", + MISSING, + _is_false, +- guard=lambda audit: _not_amd_gpu(audit) +- and not _container_check_unavailable(audit) +- and not _nvidia_runtime_configured(audit), ++ guard=lambda audit: ( ++ _not_amd_gpu(audit) ++ and not _container_check_unavailable(audit) ++ and not _nvidia_runtime_configured(audit) ++ ), + ), + Rule( + "containers.workerCheckOk", +@@ -816,8 +819,9 @@ + "Nsight Compute profiling is not enabled for the audited user", + CONFIG, + _is_false, +- guard=lambda audit: _not_amd_gpu(audit) +- and _is_true(nested_get(audit, "software", "ncu", "installed")), ++ guard=lambda audit: ( ++ _not_amd_gpu(audit) and _is_true(nested_get(audit, "software", "ncu", "installed")) ++ ), + ), + Rule( + "software.lmod.modulesStatus", +@@ -860,8 +864,7 @@ + # nvidia-open is also present, the driver and consumers (e.g. NCCL) + # select dma_buf automatically, so a loaded nvidia_peermem module is + # harmless and may still serve legacy verbs consumers. +- guard=lambda audit: _not_amd_gpu(audit) +- and not _modern_gpudirect_path_present(audit), ++ guard=lambda audit: _not_amd_gpu(audit) and not _modern_gpudirect_path_present(audit), + harnesses=SCALE_OUT_HARNESSES, + ), + # --- GPU controls ----------------------------------------------------- +@@ -870,9 +873,11 @@ + "Vboost not enabled / not allowed", + CONFIG, + _is_false, +- guard=lambda audit: _not_amd_gpu(audit) +- and not _vboost_check_unavailable(audit) +- and not _vboost_unsupported_gpu(audit), ++ guard=lambda audit: ( ++ _not_amd_gpu(audit) ++ and not _vboost_check_unavailable(audit) ++ and not _vboost_unsupported_gpu(audit) ++ ), + ), + # --- GPU HBM memory exposure ------------------------------------------- + # The Tier-0 hbm_memory_exposure check (see tests/AUDIT-CRITERIA.md) applies +@@ -1158,15 +1163,13 @@ + _AMD_HARDWARE = GuardReason( + _is_amd_gpu, + NOT_APPLICABLE_KIND, +- "An AMD GPU is present, so this NVIDIA-specific check does not apply to " +- "this hardware.", ++ "An AMD GPU is present, so this NVIDIA-specific check does not apply to this hardware.", + ) + + _NO_BLUEFIELD = GuardReason( + _bluefield_absence_proven, + NOT_APPLICABLE_KIND, +- "No BlueField DPU was observed, so this check does not apply to this " +- "configuration.", ++ "No BlueField DPU was observed, so this check does not apply to this configuration.", + ) + + # Ordered suppression reasons for every guarded rule key. Kept beside RULES so +@@ -1240,9 +1243,7 @@ + "not apply; the separate installation check reports that gap.", + ), + GuardReason( +- lambda audit: not _is_true( +- nested_get(audit, "software", "ncu", "installed") +- ), ++ lambda audit: not _is_true(nested_get(audit, "software", "ncu", "installed")), + UNVERIFIED_KIND, + "The NCU check ended without an installation verdict, so whether " + "profiling permissions apply is unverified rather than evidence " +@@ -1298,8 +1299,6 @@ + return UNVERIFIED_KIND, "" + + +- +- + # Check-key -> dashboard criterion id (dashboard/src/data/audit-criteria.ts on + # master, vendored as criteria-checks.json next to this file). Several check + # keys can serve one criterion. None marks a CLI-only check with no criteria +@@ -1440,9 +1439,7 @@ + return "" + + +-def _version_context( +- audit: dict[str, Any], key: str +-) -> tuple[str, Any, Any]: ++def _version_context(audit: dict[str, Any], key: str) -> tuple[str, Any, Any]: + component = _finding_component(key) + if key.startswith("securityVersions.") and component: + block = nested_get(audit, "securityVersions", component, default={}) +@@ -1491,10 +1488,7 @@ + "Disable nested virtualization and remove guest access to /dev/kvm " + "until the provider confirms the physical-host patch." + ) +- elif ( +- rule.key == "securityVersions.docker.status" +- and str(value).strip().lower() == "unknown" +- ): ++ elif rule.key == "securityVersions.docker.status" and str(value).strip().lower() == "unknown": + client_version = nested_get(audit, "containers", "dockerVersion") + detected = "The Docker Engine server version could not be read." + if not _missing_string(client_version): +@@ -1509,18 +1503,12 @@ + rule.key == "securityVersions.dpuHostIsolation.status" + and str(value).strip().lower() == "fail" + ): +- isolation = nested_get( +- audit, "securityVersions", "dpuHostIsolation", default={} +- ) ++ isolation = nested_get(audit, "securityVersions", "dpuHostIsolation", default={}) + detected = ( +- str(isolation.get("detail") or "").strip() +- if isinstance(isolation, dict) +- else "" ++ str(isolation.get("detail") or "").strip() if isinstance(isolation, dict) else "" + ) or "The host side can reach the BlueField DPU control plane." + remediation = ( +- str(isolation.get("remediation") or "").strip() +- if isinstance(isolation, dict) +- else "" ++ str(isolation.get("remediation") or "").strip() if isinstance(isolation, dict) else "" + ) or ( + "mlxprivhost -d r --disable_rshim --disable_tracer " + "--disable_counter_rd --disable_port_owner" +@@ -1534,18 +1522,11 @@ + rule.key == "securityVersions.dpuHostIsolation.status" + and str(value).strip().lower() == "unknown" + ): +- isolation = nested_get( +- audit, "securityVersions", "dpuHostIsolation", default={} +- ) ++ isolation = nested_get(audit, "securityVersions", "dpuHostIsolation", default={}) + if _bluefield_observed(audit): + detected = ( +- str(isolation.get("detail") or "").strip() +- if isinstance(isolation, dict) +- else "" +- ) or ( +- "The audit found a BlueField device, but it could not verify " +- "DPU host isolation." +- ) ++ str(isolation.get("detail") or "").strip() if isinstance(isolation, dict) else "" ++ ) or ("The audit found a BlueField device, but it could not verify DPU host isolation.") + recommendation = ( + "Attest the isolation posture for every detected BlueField device. " + "Verify that RShim is restricted and that /dev/rshim0 and " +@@ -1566,9 +1547,7 @@ + and str(value).strip().lower() == "unknown" + ): + if _bluefield_observed(audit): +- mode = nested_get( +- audit, "securityVersions", "virtioNetBluefield", "platformMode" +- ) ++ mode = nested_get(audit, "securityVersions", "virtioNetBluefield", "platformMode") + mode_text = str(mode).strip().upper() + mode_detail = f" in {mode_text} mode" if mode_text in {"NIC", "DPU"} else "" + detected = ( +@@ -1593,9 +1572,7 @@ + elif rule.key == "kubelet_cpu_manager_policy.status": + policy = nested_get(audit, "kubelet_cpu_manager_policy", default={}) + detected = ( +- str(policy.get("message") or "").strip() +- if isinstance(policy, dict) +- else "" ++ str(policy.get("message") or "").strip() if isinstance(policy, dict) else "" + ) or str(value).capitalize() + recommendation = ( + "Set the kubelet CPU Manager policy to static on GPU workers. Drain " +@@ -1630,7 +1607,10 @@ + elif isinstance(value, bool) and value is False: + if rule.severity == MISSING or "not installed" in rule.title.lower(): + detected = "Not installed" +- elif any(word in rule.key.lower() for word in ("allowed", "configured")) or "enabled" in rule.title.lower(): ++ elif ( ++ any(word in rule.key.lower() for word in ("allowed", "configured")) ++ or "enabled" in rule.title.lower() ++ ): + detected = "Disabled" + else: + detected = "Check failed" +@@ -1649,10 +1629,7 @@ + else: + recommendation = "Enable the required configuration, and run the audit again." + elif str(value).strip().lower() == "unknown": +- detected = ( +- "The collector could not verify this check. Treat this value as " +- "unverified." +- ) ++ detected = "The collector could not verify this check. Treat this value as unverified." + subject = component_name or rule.title.split(" requires", 1)[0] + recommendation = ( + _sentence(f"{subject} {minimum} or later.") +@@ -1662,7 +1639,9 @@ + else: + detected = str(value).replace("_", " ").capitalize() + if "reboot required" in rule.title.lower(): +- recommendation = "Reboot the affected worker, and confirm that it runs the newer kernel." ++ recommendation = ( ++ "Reboot the affected worker, and confirm that it runs the newer kernel." ++ ) + elif "attestation" in rule.title.lower() or "could not be verified" in rule.title.lower(): + recommendation = "Ask the provider to attest this state with host evidence." + elif "nvidia_peermem" in rule.title.lower(): +@@ -1713,9 +1692,7 @@ + audit = _audit_data(values) + cluster = values.get("cluster") + harness = ( +- str(cluster.get("orchestrator") or "").strip().lower() +- if isinstance(cluster, dict) +- else "" ++ str(cluster.get("orchestrator") or "").strip().lower() if isinstance(cluster, dict) else "" + ) + findings: list[Finding] = [] + for rule in RULES: +@@ -1775,11 +1752,7 @@ + Terminals with hyperlink support show only `text` and make it clickable; + the URL itself never appears in the output. + """ +- return ( +- f"\x1b]8;;{url}\x1b\\" +- f"\x1b[4m{text}\x1b[24m" +- "\x1b]8;;\x1b\\" +- ) ++ return f"\x1b]8;;{url}\x1b\\\x1b[4m{text}\x1b[24m\x1b]8;;\x1b\\" + + + def _cve_label(cve: str, *, hyperlinks: bool) -> str: +@@ -1830,9 +1803,7 @@ + ] + + +-def format_report( +- findings: list[Finding], *, source: str = "", hyperlinks: bool = False +-) -> str: ++def format_report(findings: list[Finding], *, source: str = "", hyperlinks: bool = False) -> str: + """Render actionable findings without exposing internal evidence paths.""" + bar = "=" * _HEADER_WIDTH + lines: list[str] = [bar] +@@ -1855,9 +1826,7 @@ + cves = ", ".join(_cve_label(cve, hyperlinks=hyperlinks) for cve in finding.cves) + lines.append(f" CVEs: {cves}") + if finding.advisories: +- advisories = _advisory_labels( +- finding.advisories, hyperlinks=hyperlinks +- ) ++ advisories = _advisory_labels(finding.advisories, hyperlinks=hyperlinks) + lines.append(f" Advisories: {', '.join(advisories)}") + lines.append(bar) + return "\n".join(lines) + +--- cmax/scripts/1-audit/checks/fabric/nic-topology-check.py ++++ cmax/scripts/1-audit/checks/fabric/nic-topology-check.py +@@ -106,6 +106,7 @@ + if _FANOUT is None: + sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) + import _fanout ++ + _FANOUT = _fanout + return _FANOUT + +@@ -163,15 +164,19 @@ + header_idx = i + break + if header: +- for l in lines[header_idx + 1:]: ++ for l in lines[header_idx + 1 :]: + toks = l.split() + if not toks: + continue + row = toks[0] +- if not (row.startswith("NIC") or row.startswith("mlx") or +- row.startswith("ibp") or row.startswith("roce")): ++ if not ( ++ row.startswith("NIC") ++ or row.startswith("mlx") ++ or row.startswith("ibp") ++ or row.startswith("roce") ++ ): + continue +- cells = toks[1:1 + len(header)] ++ cells = toks[1 : 1 + len(header)] + aff = "far" + for h, c in zip(header, cells): + if h.startswith("GPU") and c in ("PIX", "PXB", "PHB"): +@@ -214,17 +219,16 @@ + return "bond" not in name and state == "ACTIVE" and rate >= 100 + + rails = [ +- name for name, rate, layer, state in hcas +- if "rail" in name and eligible(name, rate, state) ++ name for name, rate, layer, state in hcas if "rail" in name and eligible(name, rate, state) + ] + if rails: + return { +- name: "rail-named RDMA HCA; no GPU-affinity signal on this platform" +- for name in rails ++ name: "rail-named RDMA HCA; no GPU-affinity signal on this platform" for name in rails + } + + ib = [ +- (name, rate) for name, rate, layer, state in hcas ++ (name, rate) ++ for name, rate, layer, state in hcas + if layer == "InfiniBand" and rate >= 200 and eligible(name, rate, state) + ] + if not ib: +@@ -276,25 +280,29 @@ + role, reason = first_pass[name] + if name in promoted: + role, reason = "fabric", promoted[name] +- nics.append({ +- "name": name, +- "role": role, +- "rate_gbps": int(rate), +- "layer": layer, +- "state": state, +- "reason": reason, +- }) ++ nics.append( ++ { ++ "name": name, ++ "role": role, ++ "rate_gbps": int(rate), ++ "layer": layer, ++ "state": state, ++ "reason": reason, ++ } ++ ) + + # Non-RDMA netdevs - frontend by definition. + for name, speed_mbps in netdevs: +- nics.append({ +- "name": name, +- "role": "frontend", +- "rate_gbps": speed_mbps // 1000 if speed_mbps > 0 else 0, +- "layer": "ethernet", +- "state": "?", +- "reason": "no infiniband sysfs link", +- }) ++ nics.append( ++ { ++ "name": name, ++ "role": "frontend", ++ "rate_gbps": speed_mbps // 1000 if speed_mbps > 0 else 0, ++ "layer": "ethernet", ++ "state": "?", ++ "reason": "no infiniband sysfs link", ++ } ++ ) + return nics + + +@@ -408,7 +416,13 @@ + "--collect-host", + ] + try: +- proc = runner(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, timeout=SLURM_TIMEOUT_S) ++ proc = runner( ++ command, ++ stdout=subprocess.PIPE, ++ stderr=subprocess.PIPE, ++ text=True, ++ timeout=SLURM_TIMEOUT_S, ++ ) + except (OSError, subprocess.TimeoutExpired) as exc: + return [gather()], [f"srun NIC check failed; local host only: {exc}"] + +@@ -466,7 +480,16 @@ + def k8s_driver_pod(namespace: str, node: str, *, runner: Runner) -> dict[str, Any] | None: + try: + proc = kubectl( +- ["get", "pods", "-n", namespace, "--field-selector", f"spec.nodeName={node}", "-o", "json"], ++ [ ++ "get", ++ "pods", ++ "-n", ++ namespace, ++ "--field-selector", ++ f"spec.nodeName={node}", ++ "-o", ++ "json", ++ ], + runner=runner, + timeout=45, + ) +@@ -515,7 +538,7 @@ + # classify_nics() needs (topo text + per-HCA rate/layer/state + frontend + # netdevs), tab-delimited, and the orchestrator (which has python3) parses and + # classifies them. Mirrors the local sysfs reads in _collect_*_local(). +-K8S_COLLECTOR_SH = r''' ++K8S_COLLECTOR_SH = r""" + echo @@TOPO_BEGIN@@ + nvidia-smi topo -m 2>/dev/null + echo @@TOPO_END@@ +@@ -536,7 +559,7 @@ + sp=$(cat "$p/speed" 2>/dev/null) + printf '@@NET@@\t%s\t%s\n' "$n" "$sp" + done +-''' ++""" + + + def _parse_k8s_collector(text: str) -> tuple[str, list, list]: +@@ -562,7 +585,11 @@ + rate = parse_rate(parts[2]) + layer = parts[3].strip() or "?" + state_raw = parts[4] +- state = state_raw.split(":")[1].split()[0] if ":" in state_raw else (state_raw.strip() or "?") ++ state = ( ++ state_raw.split(":")[1].split()[0] ++ if ":" in state_raw ++ else (state_raw.strip() or "?") ++ ) + hcas.append((name, rate, layer, state)) + elif line.startswith("@@NET@@\t"): + parts = line.split("\t") +@@ -573,10 +600,15 @@ + return "\n".join(topo_lines), hcas, netdevs + + +-def run_k8s_node_check(namespace: str, node: str, *, runner: Runner) -> tuple[dict[str, Any] | None, str | None]: ++def run_k8s_node_check( ++ namespace: str, node: str, *, runner: Runner ++) -> tuple[dict[str, Any] | None, str | None]: + access = k8s_driver_pod(namespace, node, runner=runner) + if not access: +- return None, f"{node}: no running nvidia-smi-capable GPU Operator pod for NIC classification" ++ return ( ++ None, ++ f"{node}: no running nvidia-smi-capable GPU Operator pod for NIC classification", ++ ) + exec_args = ["exec", "-i", "-n", namespace, access["name"]] + if access["container"]: + exec_args.extend(["-c", access["container"]]) +@@ -633,14 +665,18 @@ + + namespace = k8s_gpu_namespace(env=env, runner=runner) + if not namespace: +- return [], ["no NVIDIA GPU Operator namespace found; cannot reach a driver pod for NIC classification"] ++ return [], [ ++ "no NVIDIA GPU Operator namespace found; cannot reach a driver pod for NIC classification" ++ ] + + try: + max_nodes = int(str(env.get("CLUSTERMAX_AUDIT_K8S_MAX_HOST_CHECKS"))) + except (TypeError, ValueError): + max_nodes = len(nodes) + +- return load_fanout().fan_out_k8s(_nic_per_node(namespace, runner=runner), nodes=nodes, max_nodes=max_nodes) ++ return load_fanout().fan_out_k8s( ++ _nic_per_node(namespace, runner=runner), nodes=nodes, max_nodes=max_nodes ++ ) + + + def build_check_payload( +@@ -675,8 +711,17 @@ + + def main(argv: list[str]) -> int: + parser = argparse.ArgumentParser(description="NIC topology check") +- parser.add_argument("--collect-host", action="store_true", help="emit one host record instead of the aggregate check object") +- parser.add_argument("--harness", default=os.environ.get("CLUSTERMAX_AUDIT_HARNESS") or os.environ.get("CLUSTERMAX_HARNESS") or "") ++ parser.add_argument( ++ "--collect-host", ++ action="store_true", ++ help="emit one host record instead of the aggregate check object", ++ ) ++ parser.add_argument( ++ "--harness", ++ default=os.environ.get("CLUSTERMAX_AUDIT_HARNESS") ++ or os.environ.get("CLUSTERMAX_HARNESS") ++ or "", ++ ) + args = parser.parse_args(argv) + + if args.collect_host: + +--- cmax/scripts/1-audit/checks/fabric/virtio-net-check.py ++++ cmax/scripts/1-audit/checks/fabric/virtio-net-check.py +@@ -482,9 +482,7 @@ + *, runner: Runner = subprocess.run, env: dict[str, str] = os.environ + ) -> dict[str, Any]: + text, error = run_tool(["lspci", "-Dnn"], runner=runner) +- return pci_scan_result( +- text, error, empty_bus_confirmed=sysfs_pci_bus_empty(env) +- ) ++ return pci_scan_result(text, error, empty_bus_confirmed=sysfs_pci_bus_empty(env)) + + + # --------------------------------------------------------------------------- +@@ -917,12 +915,8 @@ + # root caller, or on a container with no sudo, would put a false + # statement in evidence an operator pastes into provider feedback. + privilege = { +- ESCALATION_SUDO: ( +- "mlxconfig needs root and the sudo -n retry did not answer either" +- ), +- ESCALATION_UNAVAILABLE: ( +- "mlxconfig needs root and no sudo is installed to retry with" +- ), ++ ESCALATION_SUDO: ("mlxconfig needs root and the sudo -n retry did not answer either"), ++ ESCALATION_UNAVAILABLE: ("mlxconfig needs root and no sudo is installed to retry with"), + ESCALATION_ROOT: "mlxconfig ran with root rights and still reported no mode", + }.get(str(record.get("modeEscalation")), "mlxconfig needs root") + return ( +@@ -958,7 +952,11 @@ + ) -> dict[str, Any]: + pci = collect_pci(runner=runner, env=env) + devices = pci["devices"] +- mode = collect_mode(devices, runner=runner) if devices else {"mode": "unknown", "values": {}, "error": None} ++ mode = ( ++ collect_mode(devices, runner=runner) ++ if devices ++ else {"mode": "unknown", "values": {}, "error": None} ++ ) + rshim = collect_rshim(mode.get("values") or {}, env=env) + version = collect_version(rshim, env=env, runner=runner) + +@@ -1132,11 +1130,17 @@ + (record for record in records if record.get("version") == oldest and oldest is not None), + None, + ) +- source = str(oldest_record["versionSource"]) if oldest_record and oldest_record.get("versionSource") else None ++ source = ( ++ str(oldest_record["versionSource"]) ++ if oldest_record and oldest_record.get("versionSource") ++ else None ++ ) + # Only the line that belongs to the version being reported. A line read off + # a different node would grade this version against the wrong minimum. + line = str(oldest_record["line"]) if oldest_record and oldest_record.get("line") else None +- reasons = sorted({str(record.get("reason") or "") for record in records if record.get("state") == state}) ++ reasons = sorted( ++ {str(record.get("reason") or "") for record in records if record.get("state") == state} ++ ) + + # Cluster mode, worst case first: one node still in DPU mode keeps the + # bulletin in scope for the cluster. "absent" is claimed only when every +@@ -1146,7 +1150,9 @@ + cluster_mode = "dpu" + elif "nic" in modes: + cluster_mode = "nic" +- elif all(record.get("scanComplete") and not record.get("bluefield3Present") for record in records): ++ elif all( ++ record.get("scanComplete") and not record.get("bluefield3Present") for record in records ++ ): + cluster_mode = "absent" + else: + cluster_mode = None +@@ -1376,9 +1382,17 @@ + "--collect-host", + ] + try: +- proc = runner(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, timeout=SLURM_TIMEOUT_S) ++ proc = runner( ++ command, ++ stdout=subprocess.PIPE, ++ stderr=subprocess.PIPE, ++ text=True, ++ timeout=SLURM_TIMEOUT_S, ++ ) + except (OSError, subprocess.TimeoutExpired) as exc: +- return [gather(env=env, runner=runner)], [f"srun virtio-net check failed; local host only: {exc}"] ++ return [gather(env=env, runner=runner)], [ ++ f"srun virtio-net check failed; local host only: {exc}" ++ ] + records = load_fanout().parse_json_lines(proc.stdout, require_host=True) + if not records: + return [gather(env=env, runner=runner)], [ +@@ -1395,20 +1409,16 @@ + except (TypeError, ValueError): + wanted = len(hosts) + if len(hosts) < wanted: +- errors.append( +- f"srun virtio-net check returned {len(hosts)} of {wanted} host records" +- ) ++ errors.append(f"srun virtio-net check returned {len(hosts)} of {wanted} host records") + if proc.returncode != 0: +- errors.append( +- f"srun virtio-net check exited {proc.returncode}: {proc.stderr.strip()}" +- ) ++ errors.append(f"srun virtio-net check exited {proc.returncode}: {proc.stderr.strip()}") + return records, errors + + + # The driver daemonset image has no python3, so the per-node worker is POSIX sh + # emitting tagged lines that this orchestrator parses. It reads only; it never + # runs mlxprivhost and never writes to the RShim console. +-K8S_COLLECTOR_SH = r''' ++K8S_COLLECTOR_SH = r""" + if command -v lspci >/dev/null 2>&1; then + echo @@LSPCI_BEGIN@@ + lspci -Dnn 2>/dev/null +@@ -1433,7 +1443,7 @@ + virtnet version 2>/dev/null + echo @@VIRTNET_END@@ + fi +-''' ++""" + + + def parse_k8s_collector(text: str) -> dict[str, Any]: +@@ -1492,7 +1502,9 @@ + "rshimDeviceNode": "device" in parsed["rshim"], + "tmfifoNet0": "tmfifo" in parsed["rshim"], + "internalCpuRshim": mode_values.get("INTERNAL_CPU_RSHIM", "unknown"), +- "rshimRestricted": {"0": False, "1": True}.get(mode_values.get("INTERNAL_CPU_RSHIM", ""), None), ++ "rshimRestricted": {"0": False, "1": True}.get( ++ mode_values.get("INTERNAL_CPU_RSHIM", ""), None ++ ), + "dpuReachedFromHost": None, + } + version = parse_virtnet_version(parsed["virtnet"]) +@@ -1522,8 +1534,16 @@ + return record + + +-def kubectl(args: list[str], *, runner: Runner, timeout: int = KUBECTL_TIMEOUT_S) -> subprocess.CompletedProcess: +- return runner(["kubectl", *args], stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, timeout=timeout) ++def kubectl( ++ args: list[str], *, runner: Runner, timeout: int = KUBECTL_TIMEOUT_S ++) -> subprocess.CompletedProcess: ++ return runner( ++ ["kubectl", *args], ++ stdout=subprocess.PIPE, ++ stderr=subprocess.PIPE, ++ text=True, ++ timeout=timeout, ++ ) + + + def k8s_namespace(*, env: dict[str, str], runner: Runner) -> str | None: +@@ -1555,7 +1575,16 @@ + def k8s_driver_pod(namespace: str, node: str, *, runner: Runner) -> dict[str, Any] | None: + try: + proc = kubectl( +- ["get", "pods", "-n", namespace, "--field-selector", f"spec.nodeName={node}", "-o", "json"], ++ [ ++ "get", ++ "pods", ++ "-n", ++ namespace, ++ "--field-selector", ++ f"spec.nodeName={node}", ++ "-o", ++ "json", ++ ], + runner=runner, + timeout=45, + ) +@@ -1593,7 +1622,9 @@ + return {"name": name, "container": container, "hostRoot": host_root} + + +-def run_k8s_node_check(namespace: str, node: str, *, runner: Runner) -> tuple[dict[str, Any] | None, str | None]: ++def run_k8s_node_check( ++ namespace: str, node: str, *, runner: Runner ++) -> tuple[dict[str, Any] | None, str | None]: + access = k8s_driver_pod(namespace, node, runner=runner) + if not access: + return None, f"{node}: no running nvidia-driver pod for the virtio-net check" +@@ -1637,7 +1668,9 @@ + return [], ["no Kubernetes nodes advertise nvidia.com/gpu capacity"] + namespace = k8s_namespace(env=env, runner=runner) + if not namespace: +- return [], ["no NVIDIA GPU Operator namespace found; cannot reach a driver pod for the virtio-net check"] ++ return [], [ ++ "no NVIDIA GPU Operator namespace found; cannot reach a driver pod for the virtio-net check" ++ ] + try: + max_nodes = int(str(env.get("CLUSTERMAX_AUDIT_K8S_MAX_HOST_CHECKS"))) + except (TypeError, ValueError): +@@ -1666,7 +1699,10 @@ + print(f" WARNING: virtio_net_bluefield: {error}", file=sys.stderr) + return { + "virtio_net_bluefield": { +- "hosts": {str(record.get("host") or f"host-{index}"): record for index, record in enumerate(records)}, ++ "hosts": { ++ str(record.get("host") or f"host-{index}"): record ++ for index, record in enumerate(records) ++ }, + "summary": summary, + } + } +@@ -1741,18 +1777,14 @@ + } + + +-def load_or_build_payload( +- *, harness: str, env: dict[str, str] = os.environ +-) -> dict[str, Any]: ++def load_or_build_payload(*, harness: str, env: dict[str, str] = os.environ) -> dict[str, Any]: + """Reuse this run's full fleet result across collector and final checks.""" + cache_value = str(env.get(CHECK_CACHE_ENV) or "").strip() + cache_path = Path(cache_value) if cache_value else None + if cache_path is not None: + try: + cached = json.loads(cache_path.read_text()) +- if isinstance(cached, dict) and isinstance( +- cached.get("virtio_net_bluefield"), dict +- ): ++ if isinstance(cached, dict) and isinstance(cached.get("virtio_net_bluefield"), dict): + return cached + except (OSError, json.JSONDecodeError): + pass +@@ -1770,15 +1802,19 @@ + + def main(argv: list[str]) -> int: + parser = argparse.ArgumentParser(description="BlueField VIRTIO-Net controller check") +- parser.add_argument("--collect-host", action="store_true", help="emit one host record instead of the aggregate") + parser.add_argument( ++ "--collect-host", action="store_true", help="emit one host record instead of the aggregate" ++ ) ++ parser.add_argument( + "--summary", + action="store_true", + help="emit only the collector-facing rollup (virtioNet / virtioNetLine)", + ) + parser.add_argument( + "--harness", +- default=os.environ.get("CLUSTERMAX_AUDIT_HARNESS") or os.environ.get("CLUSTERMAX_HARNESS") or "", ++ default=os.environ.get("CLUSTERMAX_AUDIT_HARNESS") ++ or os.environ.get("CLUSTERMAX_HARNESS") ++ or "", + ) + args = parser.parse_args(argv) + + +--- cmax/scripts/1-audit/checks/gpu/vboost.py ++++ cmax/scripts/1-audit/checks/gpu/vboost.py +@@ -50,6 +50,7 @@ + if _FANOUT is None: + sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) + import _fanout ++ + _FANOUT = _fanout + return _FANOUT + +@@ -73,7 +74,9 @@ + return bool(_VBOOST_UNSUPPORTED_GPU_RE.search(model)) + + +-def vboost_command(which: Which = shutil.which, euid: Callable[[], int] = os.geteuid) -> tuple[list[str] | None, str]: ++def vboost_command( ++ which: Which = shutil.which, euid: Callable[[], int] = os.geteuid ++) -> tuple[list[str] | None, str]: + nvidia_smi = which("nvidia-smi") + if not nvidia_smi: + return None, "nvidia_smi_missing" +@@ -234,7 +237,16 @@ + def k8s_driver_pod(namespace: str, node: str, *, runner: Runner) -> str | None: + try: + proc = kubectl( +- ["get", "pods", "-n", namespace, "--field-selector", f"spec.nodeName={node}", "-o", "json"], ++ [ ++ "get", ++ "pods", ++ "-n", ++ namespace, ++ "--field-selector", ++ f"spec.nodeName={node}", ++ "-o", ++ "json", ++ ], + runner=runner, + timeout=45, + ) +@@ -261,7 +273,9 @@ + return min(candidates)[1] if candidates else None + + +-def k8s_vboost_node(namespace: str, node: str, *, runner: Runner) -> tuple[dict[str, Any] | None, str | None]: ++def k8s_vboost_node( ++ namespace: str, node: str, *, runner: Runner ++) -> tuple[dict[str, Any] | None, str | None]: + """Try to set vboost from inside the node's nvidia-driver daemonset pod. + + The driver pod runs privileged with nvidia-smi on PATH, so this exercises +@@ -327,7 +341,9 @@ + + namespace = k8s_gpu_namespace(env=env, runner=runner) + if not namespace: +- return [], ["no NVIDIA GPU Operator namespace found; cannot reach a driver pod to test vboost"] ++ return [], [ ++ "no NVIDIA GPU Operator namespace found; cannot reach a driver pod to test vboost" ++ ] + + try: + max_nodes = int(str(env.get("CLUSTERMAX_AUDIT_K8S_MAX_HOST_CHECKS"))) +@@ -429,7 +445,9 @@ + print(json.dumps(local_vboost_result(), sort_keys=True)) + return 0 + +- harness = os.environ.get("CLUSTERMAX_AUDIT_HARNESS") or os.environ.get("CLUSTERMAX_HARNESS") or "" ++ harness = ( ++ os.environ.get("CLUSTERMAX_AUDIT_HARNESS") or os.environ.get("CLUSTERMAX_HARNESS") or "" ++ ) + print(json.dumps(build_check_payload(harness=harness), sort_keys=True)) + return 0 + + +--- cmax/scripts/1-audit/checks/platform_config.py ++++ cmax/scripts/1-audit/checks/platform_config.py +@@ -412,7 +412,10 @@ + + + def iommu_disabled_requested(params: dict[str, str]) -> bool: +- return any(params.get(key, "").lower() in {"off", "disabled"} for key in ("iommu", "intel_iommu", "amd_iommu")) ++ return any( ++ params.get(key, "").lower() in {"off", "disabled"} ++ for key in ("iommu", "intel_iommu", "amd_iommu") ++ ) + + + def classify_iommu_mode( +@@ -464,7 +467,9 @@ + return "other" + + +-def collect_pci_devices(root: Path, *, include_rdma: bool = True) -> tuple[list[dict[str, Any]], bool]: ++def collect_pci_devices( ++ root: Path, *, include_rdma: bool = True ++) -> tuple[list[dict[str, Any]], bool]: + """Selected GPU and RDMA NIC PCI functions with their IOMMU group. + + ``iommu_group`` is a symlink into ``/sys/kernel/iommu_groups/``; an empty +@@ -660,7 +665,8 @@ + sorted( + entry.name + for entry in under_root(root, path).iterdir() +- if entry.is_symlink() and not entry.name.startswith(("bind", "uevent", "unbind", "module")) ++ if entry.is_symlink() ++ and not entry.name.startswith(("bind", "uevent", "unbind", "module")) + ) + ) + except OSError: +@@ -670,7 +676,11 @@ + devicetree = devicetree_has_cmdqv(root) + # Anything that names the CMDQV driver or node is itself proof of Grace, so + # the platform markers only have to answer for a host that exposes neither. +- grace = bool(driver_dirs or bound_devices or devicetree) or devicetree_names_grace(root) or dmi_names_grace(root) ++ grace = ( ++ bool(driver_dirs or bound_devices or devicetree) ++ or devicetree_names_grace(root) ++ or dmi_names_grace(root) ++ ) + + return { + "machine": machine, +@@ -738,7 +748,9 @@ + "Arm virtual machine guest uses SMMUv3 without CMDQV / VCMDQ; " + "guest invalidations serialize through one host command queue" + ), +- warnings=[f"SMMUv3 unit(s) {', '.join(smmu.get('smmuv3_units') or [])} with no tegra241-cmdqv evidence"], ++ warnings=[ ++ f"SMMUv3 unit(s) {', '.join(smmu.get('smmuv3_units') or [])} with no tegra241-cmdqv evidence" ++ ], + evidence=evidence, + ) + return summary( +@@ -769,7 +781,12 @@ + """Enroot files that could bind-mount the topology file into the container.""" + paths: list[str] = [] + mentions_topology: list[str] = [] +- for relative in ("/etc/enroot/enroot.conf", "/etc/enroot/enroot.conf.d", "/etc/enroot/hooks.d", "/etc/enroot/mounts.d"): ++ for relative in ( ++ "/etc/enroot/enroot.conf", ++ "/etc/enroot/enroot.conf.d", ++ "/etc/enroot/hooks.d", ++ "/etc/enroot/mounts.d", ++ ): + entry = under_root(root, relative) + # pathlib's recursive glob swallows PermissionError, but not the rest of + # OSError, and neither do exists() and is_dir(). This scan runs before +@@ -828,7 +845,9 @@ + "devices": devices, + "sysfs_read": sysfs_read, + "link_layers": unique_layers, +- "fabric": unique_layers[0] if len(unique_layers) == 1 else ("mixed" if unique_layers else "none"), ++ "fabric": unique_layers[0] ++ if len(unique_layers) == 1 ++ else ("mixed" if unique_layers else "none"), + } + + +@@ -847,7 +866,9 @@ + # ``is not False`` keeps an entry the stat could not answer for. Dropping it + # would let an unreadable entry stand for an absent file, which is the + # asserted absence this check must never make. +- return [f"{TOPO_FILE_GLOB_DIR}/{entry.name}" for entry in entries if file_state(entry) is not False] ++ return [ ++ f"{TOPO_FILE_GLOB_DIR}/{entry.name}" for entry in entries if file_state(entry) is not False ++ ] + + + def describe_topo_file(path: Path) -> dict[str, Any]: +@@ -916,7 +937,9 @@ + "topo_file_conf": conf_topo, + "topo_file_declared": declared, + "topo_files_readable": readable, +- "topo_file_evidence": {path: describe_topo_file(under_root(root, path)) for path in readable}, ++ "topo_file_evidence": { ++ path: describe_topo_file(under_root(root, path)) for path in readable ++ }, + "topo_candidates_unreachable": unreachable, + "declared_topo_file_readable": bool(declared) and declared in readable, + "nccl_conf_present": bool(conf_text), +@@ -978,7 +1001,7 @@ + conf = safe_candidates([conf_path]) + loop = "" + if quoted: +- loop = f"for candidate in {quoted}; do\n" ' report "$candidate"\n' "done\n" ++ loop = f'for candidate in {quoted}; do\n report "$candidate"\ndone\n' + # NCCL reads /etc/nccl.conf as well as the environment, so a container that + # ships the conf resolves the topology file with no variable set. The sed + # pair strips a trailing comment, surrounding whitespace, and one balanced +@@ -1100,7 +1123,9 @@ + ) + + +-def run_container_check(*, harness: str, env: dict[str, str], candidates: list[str], runner=run_command) -> dict[str, Any]: ++def run_container_check( ++ *, harness: str, env: dict[str, str], candidates: list[str], runner=run_command ++) -> dict[str, Any]: + """Read NCCL_TOPO_FILE from inside a container started by the same launcher. + + Only slurm/pyxis gives the audit a container that the site's own launcher +@@ -1192,7 +1217,12 @@ + "stderr": stderr, + **attestation, + } +- return {"available": True, "stdout": (proc.stdout or "").strip()[:8000], **attestation, **parsed} ++ return { ++ "available": True, ++ "stdout": (proc.stdout or "").strip()[:8000], ++ **attestation, ++ **parsed, ++ } + + + def vantage_is_local_only(reports: list[dict[str, Any]]) -> bool: +@@ -1216,10 +1246,20 @@ + errors: list[str], + ) -> dict[str, Any]: + hosts = [report.get("nccl", {}) for report in reports] +- declared = next((host.get("topo_file_declared") for host in hosts if host.get("topo_file_declared")), "") ++ declared = next( ++ (host.get("topo_file_declared") for host in hosts if host.get("topo_file_declared")), "" ++ ) + host_env = next((host.get("topo_file_env") for host in hosts if host.get("topo_file_env")), "") +- host_files = sorted({path for host in hosts for path in (host.get("topo_files_readable") or [])}) +- enroot_hooks = sorted({path for host in hosts for path in (host.get("enroot", {}).get("files_mentioning_topology") or [])}) ++ host_files = sorted( ++ {path for host in hosts for path in (host.get("topo_files_readable") or [])} ++ ) ++ enroot_hooks = sorted( ++ { ++ path ++ for host in hosts ++ for path in (host.get("enroot", {}).get("files_mentioning_topology") or []) ++ } ++ ) + host_evidence: dict[str, Any] = {} + for host in hosts: + host_evidence.update(host.get("topo_file_evidence") or {}) +@@ -1251,7 +1291,9 @@ + # A declaration is not a file. A stale NCCL_TOPO_FILE or nccl.conf entry + # pointing at a path that holds nothing leaves nothing to mount, so the + # check has no subject and must not hard-fail on a missing mount. +- unreachable = sorted({path for host in hosts for path in (host.get("topo_candidates_unreachable") or [])}) ++ unreachable = sorted( ++ {path for host in hosts for path in (host.get("topo_candidates_unreachable") or [])} ++ ) + if unreachable: + # A path the vantage cannot reach is not a path with no file on it. + detail["unreachable_topo_candidates"] = unreachable +@@ -1346,7 +1388,9 @@ + if seen.get("size") == 0: + failures.append(f"container resolves {container_declared} but the file is empty") + if seen.get("xml") == "bad": +- failures.append(f"container resolves {container_declared} but its contents do not parse as XML") ++ failures.append( ++ f"container resolves {container_declared} but its contents do not parse as XML" ++ ) + # A topology file describes the node it was generated for, so two nodes + # legitimately publish different content at the same path. The container + # runs on one node of the allocation and these reports cover the +@@ -1359,14 +1403,21 @@ + host_digests: set[str] = set() + host_missing_digest = False + for host in hosts: +- digest = ((host.get("topo_file_evidence") or {}).get(container_declared) or {}).get("sha256") ++ digest = ((host.get("topo_file_evidence") or {}).get(container_declared) or {}).get( ++ "sha256" ++ ) + if digest: + host_digests.add(digest) + else: + host_missing_digest = True + detail["host_topo_file_digests"] = sorted(host_digests) + container_digest = seen.get("sha256") +- if container_digest and host_digests and not host_missing_digest and container_digest not in host_digests: ++ if ( ++ container_digest ++ and host_digests ++ and not host_missing_digest ++ and container_digest not in host_digests ++ ): + failures.append( + f"container resolves {container_declared} but its contents match no checked host's file at that " + "path; the container sees a stale or shadowed topology file" +@@ -1384,7 +1435,9 @@ + return detail + + +-def classify_fabric_shape(*, fabric_tiers: int, node_count: int, clos_node_threshold: int) -> tuple[bool | None, str]: ++def classify_fabric_shape( ++ *, fabric_tiers: int, node_count: int, clos_node_threshold: int ++) -> tuple[bool | None, str]: + """Whether the fabric has a spine tier, and the evidence that decided it. + + An exact tier count wins. Without one the GPU node count stands in, because +@@ -1412,11 +1465,15 @@ + errors: list[str], + ) -> dict[str, Any]: + hosts = [report.get("nccl", {}) for report in reports] +- qps_values = sorted({as_int(host.get("qps_per_connection"), DEFAULT_QPS_PER_CONNECTION) for host in hosts}) ++ qps_values = sorted( ++ {as_int(host.get("qps_per_connection"), DEFAULT_QPS_PER_CONNECTION) for host in hosts} ++ ) + qps = qps_values[0] if qps_values else DEFAULT_QPS_PER_CONNECTION + source = next((host.get("qps_source") for host in hosts if host.get("qps_source")), "default") + fabrics = sorted({str(host.get("rdma", {}).get("fabric") or "none") for host in hosts}) +- rdma_devices = sorted({device for host in hosts for device in (host.get("rdma", {}).get("devices") or [])}) ++ rdma_devices = sorted( ++ {device for host in hosts for device in (host.get("rdma", {}).get("devices") or [])} ++ ) + # "No host has an RDMA device" is a claim about every host, so one host that + # could not be read withdraws it. A sibling that read a clean empty class + # says nothing about the host nobody could read. +@@ -1600,7 +1657,10 @@ + the hosts that were read actually have, and every per-host status stays + visible under ``hosts``. + """ +- statuses = [str(report.get("summaries", {}).get(key, {}).get("status") or "unknown") for report in reports] ++ statuses = [ ++ str(report.get("summaries", {}).get(key, {}).get("status") or "unknown") ++ for report in reports ++ ] + if "fail" in statuses: + status = "fail" + elif "warning" in statuses: +@@ -1673,7 +1733,9 @@ + status="unknown", + message="only a stand-in host was checked; the platform of the compute nodes was not read", + ) +- payload["nccl_topo_file"] = summarize_nccl_topo_file(reports=reports, container=container, errors=errors) ++ payload["nccl_topo_file"] = summarize_nccl_topo_file( ++ reports=reports, container=container, errors=errors ++ ) + payload["nccl_ib_qps"] = summarize_nccl_ib_qps( + reports=reports, + node_count=node_count, +@@ -1865,7 +1927,9 @@ + reports: list[dict[str, Any]] = [] + errors: list[str] = [] + if proc.returncode != 0: +- errors.append(f"srun host check exited {proc.returncode}; parsing any completed host output") ++ errors.append( ++ f"srun host check exited {proc.returncode}; parsing any completed host output" ++ ) + for value in load_fanout().parse_json_lines(proc.stdout, require_host=True): + reports.append(value) + if proc.stderr and (proc.returncode != 0 or not reports): +@@ -1905,7 +1969,9 @@ + { + "name": "check", + "image": image, +- "imagePullPolicy": os.environ.get("CLUSTERMAX_AUDIT_K8S_HOST_CHECK_PULL_POLICY", "IfNotPresent"), ++ "imagePullPolicy": os.environ.get( ++ "CLUSTERMAX_AUDIT_K8S_HOST_CHECK_PULL_POLICY", "IfNotPresent" ++ ), + "command": ["sh", "-c", "sleep 600"], + "env": [{"name": "NODE_NAME", "value": node["name"]}], + "securityContext": {"privileged": True, "runAsUser": 0}, +@@ -1944,14 +2010,20 @@ + try: + apply_proc = kubectl(["apply", "-f", "-"], timeout=45, input_text=manifest) + if apply_proc.returncode != 0: +- return None, f"{node['name']}: failed to create host check pod: {apply_proc.stderr.strip()}" ++ return ( ++ None, ++ f"{node['name']}: failed to create host check pod: {apply_proc.stderr.strip()}", ++ ) + + wait_proc = kubectl( + ["wait", f"pod/{pod_name}", "-n", namespace, "--for=condition=Ready", "--timeout=90s"], + timeout=100, + ) + if wait_proc.returncode != 0: +- return None, f"{node['name']}: host check pod did not become Ready: {wait_proc.stderr.strip()}" ++ return ( ++ None, ++ f"{node['name']}: host check pod did not become Ready: {wait_proc.stderr.strip()}", ++ ) + + check_args = [ + "exec", +@@ -1991,7 +2063,18 @@ + return None, f"{node['name']}: host check failed: {exc}" + finally: + try: +- kubectl(["delete", "pod", pod_name, "-n", namespace, "--ignore-not-found=true", "--wait=false"], timeout=20) ++ kubectl( ++ [ ++ "delete", ++ "pod", ++ pod_name, ++ "-n", ++ namespace, ++ "--ignore-not-found=true", ++ "--wait=false", ++ ], ++ timeout=20, ++ ) + except (OSError, subprocess.SubprocessError): + pass + +@@ -2035,18 +2118,12 @@ + ) + + +-def run_default_check( +- harness: str, requested_keys: set[str] | None = None +-) -> dict[str, Any]: ++def run_default_check(harness: str, requested_keys: set[str] | None = None) -> dict[str, Any]: + requested = set(ALL_CHECK_KEYS if requested_keys is None else requested_keys) + unknown = requested.difference(ALL_CHECK_KEYS) + if unknown: +- raise ValueError( +- f"unknown platform configuration check(s): {', '.join(sorted(unknown))}" +- ) +- include_scale_out = bool( +- requested.intersection({"nccl_topo_file", "nccl_ib_qps"}) +- ) ++ raise ValueError(f"unknown platform configuration check(s): {', '.join(sorted(unknown))}") ++ include_scale_out = bool(requested.intersection({"nccl_topo_file", "nccl_ib_qps"})) + include_rdma_iommu = "vm_iommu" in requested and harness != "standalone" + errors: list[str] = [] + node_count = 1 +@@ -2093,7 +2170,10 @@ + ) + report["check_scope"] = "local" + reports = [report] +- errors = [*discovery_errors, "no GPU node was checked; the local host was checked instead"] ++ errors = [ ++ *discovery_errors, ++ "no GPU node was checked; the local host was checked instead", ++ ] + else: + reports = [ + collect_host( +@@ -2107,15 +2187,17 @@ + # Only a file a host actually holds gives the container arm a subject. A + # declaration pointing at nothing grades not_applicable whatever the + # container says, so it must not start one. +- host_topo_files = sorted( +- { +- path +- for report in reports +- for path in ( +- report.get("nccl", {}).get("topo_files_readable") or [] +- ) +- } +- ) if "nccl_topo_file" in requested else [] ++ host_topo_files = ( ++ sorted( ++ { ++ path ++ for report in reports ++ for path in (report.get("nccl", {}).get("topo_files_readable") or []) ++ } ++ ) ++ if "nccl_topo_file" in requested ++ else [] ++ ) + topo_candidates = sorted( + { + path +@@ -2142,7 +2224,9 @@ + "reason_code": "no_host_topo_file", + "reason": "no readable NCCL topology file on the checked hosts; the container check was skipped", + } +- threshold = as_int(os.environ.get("CLUSTERMAX_AUDIT_CLOS_NODE_THRESHOLD"), default=DEFAULT_CLOS_NODE_THRESHOLD) ++ threshold = as_int( ++ os.environ.get("CLUSTERMAX_AUDIT_CLOS_NODE_THRESHOLD"), default=DEFAULT_CLOS_NODE_THRESHOLD ++ ) + payload = build_payload( + reports=reports, + errors=errors, +@@ -2181,11 +2265,9 @@ + + if payload is None: + requested_value = os.environ.get(REQUESTED_CHECKS_ENV, "") +- requested = { +- key.strip() +- for key in requested_value.split(",") +- if key.strip() +- } or {check_key} ++ requested = {key.strip() for key in requested_value.split(",") if key.strip()} or { ++ check_key ++ } + payload = run_default_check(harness, requested) + if cache_path is not None: + cache_path.parent.mkdir(parents=True, exist_ok=True) +@@ -2201,10 +2283,16 @@ + + def main(argv: list[str]) -> int: + parser = argparse.ArgumentParser(description="Shared platform configuration collection") +- parser.add_argument("--collect-host", action="store_true", help="emit one host report instead of aggregate check JSON") ++ parser.add_argument( ++ "--collect-host", ++ action="store_true", ++ help="emit one host report instead of aggregate check JSON", ++ ) + parser.add_argument("--root", default="/", help="host root path for proc/sys/etc reads") +- parser.add_argument("--harness", default=os.environ.get("CLUSTERMAX_AUDIT_HARNESS", "standalone")) + parser.add_argument( ++ "--harness", default=os.environ.get("CLUSTERMAX_AUDIT_HARNESS", "standalone") ++ ) ++ parser.add_argument( + "--skip-scale-out", + action="store_true", + help="omit NCCL and RDMA host collection", + +--- cmax/scripts/1-audit/checks/system/hbm_memory_exposure.py ++++ cmax/scripts/1-audit/checks/system/hbm_memory_exposure.py +@@ -62,6 +62,7 @@ + if _FANOUT is None: + sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) + import _fanout ++ + _FANOUT = _fanout + return _FANOUT + +@@ -142,7 +143,9 @@ + return gpus + + +-def run_command(command: list[str], *, timeout: int = 30, input_text: str | None = None) -> subprocess.CompletedProcess[str]: ++def run_command( ++ command: list[str], *, timeout: int = 30, input_text: str | None = None ++) -> subprocess.CompletedProcess[str]: + return subprocess.run( + command, + input=input_text, +@@ -332,14 +335,18 @@ + ) + + if coherent_candidate and cdmm_mode != "driver": +- msg = f"coherent GPU platform is not in CDMM driver mode (CoherentGPUMemoryMode={cdmm_mode})" ++ msg = ( ++ f"coherent GPU platform is not in CDMM driver mode (CoherentGPUMemoryMode={cdmm_mode})" ++ ) + if harness == "k8s": + failures.append(msg) + else: + warnings.append(msg) + + if report.get("memory_only_numa_nodes") and not hbm_nodes: +- warnings.append("host has memory-only NUMA node(s), but they were not classified as GPU HBM") ++ warnings.append( ++ "host has memory-only NUMA node(s), but they were not classified as GPU HBM" ++ ) + + status = "pass" + if failures: +@@ -407,18 +414,25 @@ + gpu_memory_mb_hint: int = 0, + gpu_count_hint: int = 0, + ) -> dict[str, Any]: +- hostname = os.environ.get("NODE_NAME") or read_text(under_root(root, "/proc/sys/kernel/hostname")).strip() ++ hostname = ( ++ os.environ.get("NODE_NAME") ++ or read_text(under_root(root, "/proc/sys/kernel/hostname")).strip() ++ ) + if not hostname: + hostname = socket.gethostname() + + smi_gpus = query_nvidia_smi() if root == Path("/") else [] + proc_gpus = parse_gpu_information(root) +- gpu_models = [str(gpu.get("model") or gpu.get("model_name") or "") for gpu in smi_gpus + proc_gpus] ++ gpu_models = [ ++ str(gpu.get("model") or gpu.get("model_name") or "") for gpu in smi_gpus + proc_gpus ++ ] + if gpu_model_hint: + gpu_models.append(gpu_model_hint) + gpu_models = sorted({model for model in gpu_models if model}) + +- gpu_memories = [as_int(gpu.get("memory_mb")) for gpu in smi_gpus if as_int(gpu.get("memory_mb")) > 0] ++ gpu_memories = [ ++ as_int(gpu.get("memory_mb")) for gpu in smi_gpus if as_int(gpu.get("memory_mb")) > 0 ++ ] + if gpu_memory_mb_hint > 0: + gpu_memories.append(gpu_memory_mb_hint) + gpu_memory_mb = max(gpu_memories) if gpu_memories else 0 +@@ -433,7 +447,9 @@ + ) + + meminfo_total_kb = parse_memtotal_kb(read_text(under_root(root, "/proc/meminfo"))) +- cpu_numa_mem_kb = sum(as_int(node.get("mem_total_kb")) for node in numa_nodes if node.get("has_cpus")) ++ cpu_numa_mem_kb = sum( ++ as_int(node.get("mem_total_kb")) for node in numa_nodes if node.get("has_cpus") ++ ) + memory_only_mem_kb = sum(as_int(node.get("mem_total_kb")) for node in memory_only_nodes) + hbm_like_mem_kb = sum(as_int(node.get("mem_total_kb")) for node in hbm_like_nodes) + node_mem_total_kb = sum(as_int(node.get("mem_total_kb")) for node in numa_nodes) +@@ -461,10 +477,12 @@ + "node_memory_total_kb": node_mem_total_kb, + "meminfo_matches_node_total": ( + node_mem_total_kb > 0 +- and abs(meminfo_total_kb - node_mem_total_kb) <= max(int(node_mem_total_kb * 0.02), GIB_KB) ++ and abs(meminfo_total_kb - node_mem_total_kb) ++ <= max(int(node_mem_total_kb * 0.02), GIB_KB) + ), + "meminfo_includes_hbm_like_memory": ( +- hbm_like_mem_kb > 0 and meminfo_total_kb >= cpu_numa_mem_kb + int(hbm_like_mem_kb * 0.8) ++ hbm_like_mem_kb > 0 ++ and meminfo_total_kb >= cpu_numa_mem_kb + int(hbm_like_mem_kb * 0.8) + ), + }, + "memory_only_numa_nodes": memory_only_nodes, +@@ -531,7 +549,9 @@ + "failures": failures, + "warnings": warnings, + "hosts_checked": len(reports), +- "hosts": {str(report.get("host") or f"host-{idx}"): report for idx, report in enumerate(reports)}, ++ "hosts": { ++ str(report.get("host") or f"host-{idx}"): report for idx, report in enumerate(reports) ++ }, + } + + +@@ -591,7 +611,9 @@ + reports: list[dict[str, Any]] = [] + errors: list[str] = [] + if proc.returncode != 0: +- errors.append(f"srun host check exited {proc.returncode}; parsing any completed host output") ++ errors.append( ++ f"srun host check exited {proc.returncode}; parsing any completed host output" ++ ) + + for line in proc.stdout.splitlines(): + line = line.strip() +@@ -621,7 +643,9 @@ + return reports, errors + + +-def kubectl(command: list[str], *, timeout: int = 60, input_text: str | None = None) -> subprocess.CompletedProcess[str]: ++def kubectl( ++ command: list[str], *, timeout: int = 60, input_text: str | None = None ++) -> subprocess.CompletedProcess[str]: + return run_command(["kubectl", *command], timeout=timeout, input_text=input_text) + + +@@ -652,14 +676,20 @@ + { + "name": "check", + "image": image, +- "imagePullPolicy": os.environ.get("CLUSTERMAX_AUDIT_K8S_HOST_CHECK_PULL_POLICY", "IfNotPresent"), ++ "imagePullPolicy": os.environ.get( ++ "CLUSTERMAX_AUDIT_K8S_HOST_CHECK_PULL_POLICY", "IfNotPresent" ++ ), + "command": ["sh", "-c", "sleep 600"], + "env": _k8s_check_env(node), + "securityContext": {"privileged": True, "runAsUser": 0}, + "volumeMounts": [ + {"name": "host-proc", "mountPath": "/host/proc", "readOnly": True}, + {"name": "host-sys", "mountPath": "/host/sys", "readOnly": True}, +- {"name": "host-kubelet", "mountPath": "/host/var/lib/kubelet", "readOnly": True}, ++ { ++ "name": "host-kubelet", ++ "mountPath": "/host/var/lib/kubelet", ++ "readOnly": True, ++ }, + ], + } + ], +@@ -672,24 +702,46 @@ + } + + +-def run_k8s_host_check(namespace: str, node: dict[str, Any], image: str) -> tuple[dict[str, Any] | None, str | None]: ++def run_k8s_host_check( ++ namespace: str, node: dict[str, Any], image: str ++) -> tuple[dict[str, Any] | None, str | None]: + suffix = uuid.uuid4().hex[:8] + pod_name = f"clustermax-hbm-{suffix}" + manifest = json.dumps(pod_manifest(namespace, node, image, pod_name)) + try: + apply_proc = kubectl(["apply", "-f", "-"], timeout=45, input_text=manifest) + if apply_proc.returncode != 0: +- return None, f"{node['name']}: failed to create host check pod: {apply_proc.stderr.strip()}" ++ return ( ++ None, ++ f"{node['name']}: failed to create host check pod: {apply_proc.stderr.strip()}", ++ ) + + wait_proc = kubectl( + ["wait", f"pod/{pod_name}", "-n", namespace, "--for=condition=Ready", "--timeout=90s"], + timeout=100, + ) + if wait_proc.returncode != 0: +- return None, f"{node['name']}: host check pod did not become Ready: {wait_proc.stderr.strip()}" ++ return ( ++ None, ++ f"{node['name']}: host check pod did not become Ready: {wait_proc.stderr.strip()}", ++ ) + + exec_proc = kubectl( +- ["exec", "-i", "-n", namespace, pod_name, "--", "python3", "-", "--collect-host", "--root", "/host", "--harness", "k8s"], ++ [ ++ "exec", ++ "-i", ++ "-n", ++ namespace, ++ pod_name, ++ "--", ++ "python3", ++ "-", ++ "--collect-host", ++ "--root", ++ "/host", ++ "--harness", ++ "k8s", ++ ], + timeout=60, + input_text=Path(__file__).read_text(), + ) +@@ -707,7 +759,18 @@ + return None, f"{node['name']}: host check failed: {exc}" + finally: + try: +- kubectl(["delete", "pod", pod_name, "-n", namespace, "--ignore-not-found=true", "--wait=false"], timeout=20) ++ kubectl( ++ [ ++ "delete", ++ "pod", ++ pod_name, ++ "-n", ++ namespace, ++ "--ignore-not-found=true", ++ "--wait=false", ++ ], ++ timeout=20, ++ ) + except Exception: + pass + +@@ -737,7 +800,9 @@ + namespace = os.environ.get("CLUSTERMAX_AUDIT_K8S_NAMESPACE", "default") + image = os.environ.get("CLUSTERMAX_AUDIT_K8S_HOST_CHECK_IMAGE", "python:3.12-alpine") + max_nodes = as_int(os.environ.get("CLUSTERMAX_AUDIT_K8S_MAX_HOST_CHECKS"), default=len(nodes)) +- return load_fanout().fan_out_k8s(_hbm_per_node(namespace, image), nodes=nodes, max_nodes=max_nodes) ++ return load_fanout().fan_out_k8s( ++ _hbm_per_node(namespace, image), nodes=nodes, max_nodes=max_nodes ++ ) + + + def run_default_check(harness: str) -> dict[str, Any]: +@@ -752,17 +817,21 @@ + "hbm_memory_exposure": aggregate_reports(reports, errors), + } + if harness == "k8s": +- result["kubelet_cpu_manager_policy"] = aggregate_cpu_manager_reports( +- reports, errors +- ) ++ result["kubelet_cpu_manager_policy"] = aggregate_cpu_manager_reports(reports, errors) + return result + + + def main(argv: list[str]) -> int: + parser = argparse.ArgumentParser(description=__doc__) +- parser.add_argument("--collect-host", action="store_true", help="emit one host report instead of aggregate check JSON") ++ parser.add_argument( ++ "--collect-host", ++ action="store_true", ++ help="emit one host report instead of aggregate check JSON", ++ ) + parser.add_argument("--root", default="/", help="host root path for proc/sys/kubelet reads") +- parser.add_argument("--harness", default=os.environ.get("CLUSTERMAX_AUDIT_HARNESS", "standalone")) ++ parser.add_argument( ++ "--harness", default=os.environ.get("CLUSTERMAX_AUDIT_HARNESS", "standalone") ++ ) + args = parser.parse_args(argv) + + if args.collect_host: + +--- cmax/scripts/1-audit/merge_audit.py ++++ cmax/scripts/1-audit/merge_audit.py +@@ -291,16 +291,16 @@ + # --- GDRCopy (library present + gdrdrv module) --- + gdr_lib = check.get("WORKER_GDRCOPY_LIB") + gdrdrv_loaded = ( +- _truthy(check.get("WORKER_GDRCOPY_GDRDRV")) +- if "WORKER_GDRCOPY_GDRDRV" in check +- else None ++ _truthy(check.get("WORKER_GDRCOPY_GDRDRV")) if "WORKER_GDRCOPY_GDRDRV" in check else None + ) + if gdr_lib is not None or gdrdrv_loaded is not None: + gdrcopy = _ensure_dict(gpus, "gdrcopy") +- lib_present = ( +- gdr_lib is not None +- and str(gdr_lib).strip().lower() not in {"", "not-found", "none", "unknown"} +- ) ++ lib_present = gdr_lib is not None and str(gdr_lib).strip().lower() not in { ++ "", ++ "not-found", ++ "none", ++ "unknown", ++ } + # The libgdrapi file lives in the node image; on k8s the check can run in + # a container whose root lacks it even though gdrdrv is loaded on the + # host. The gdrdrv kernel module is host-global (/proc/modules is not +@@ -379,9 +379,7 @@ + for check_key, canonical_key in _MEM_KEY_MAP: + if check_key in check: + _set_default(memory, canonical_key, str(check.get(check_key))) +- _set_default( +- memory, "source", str(check.get("WORKER_MEM_SOURCE", "unknown")) +- ) ++ _set_default(memory, "source", str(check.get("WORKER_MEM_SOURCE", "unknown"))) + + # --- BMC / IPMI exposure (mirror the slurm security.bmcIpmi fold) --- + user_access = check.get("WORKER_IPMI_USER_ACCESS") +@@ -431,7 +429,9 @@ + if "jobAttribution" in gme: + _set_default(stack, "gpuMetricsJobAttribution", _truthy(gme.get("jobAttribution"))) + if gme.get("jobAttributionMethod"): +- _set_default(stack, "gpuMetricsJobAttributionMethod", gme.get("jobAttributionMethod")) ++ _set_default( ++ stack, "gpuMetricsJobAttributionMethod", gme.get("jobAttributionMethod") ++ ) + if "amdDeviceMetricsExporter" in gme: + amd_exp = gme.get("amdDeviceMetricsExporter") + if isinstance(comps, dict): +@@ -440,7 +440,9 @@ + if amd_exp is None and "amdDeviceMetricsExporter" in comps: + amd_exp = comps.get("amdDeviceMetricsExporter") + if "nodeProblemDetector" in comps: +- _set_default(stack, "nodeProblemDetector", _truthy(comps.get("nodeProblemDetector"))) ++ _set_default( ++ stack, "nodeProblemDetector", _truthy(comps.get("nodeProblemDetector")) ++ ) + if amd_exp is not None: + _set_default(stack, "amdDeviceMetricsExporter", _truthy(amd_exp)) + +@@ -454,7 +456,9 @@ + lmod["installed"] = bool(str(check.get("WORKER_LMOD_PATH") or "").strip()) + module_inventory = audit.get("lmod") + module_values = ( +- tuple(module_inventory.get(key) for key in ("hasCudaModule", "hasHpcxModule", "hasNcclModule")) ++ tuple( ++ module_inventory.get(key) for key in ("hasCudaModule", "hasHpcxModule", "hasNcclModule") ++ ) + if isinstance(module_inventory, dict) + else () + ) +@@ -523,14 +527,18 @@ + return + layers = { + str(nic.get("layer", "")).lower() +- for nics in nic_topology.values() if isinstance(nics, list) +- for nic in nics if isinstance(nic, dict) and nic.get("role") == "fabric" ++ for nics in nic_topology.values() ++ if isinstance(nics, list) ++ for nic in nics ++ if isinstance(nic, dict) and nic.get("role") == "fabric" + } + if not layers: # fall back to all NICs if none were tagged "fabric" + layers = { + str(nic.get("layer", "")).lower() +- for nics in nic_topology.values() if isinstance(nics, list) +- for nic in nics if isinstance(nic, dict) ++ for nics in nic_topology.values() ++ if isinstance(nics, list) ++ for nic in nics ++ if isinstance(nic, dict) + } + if "infiniband" in layers: + networking["rdmaType"] = "infiniband" +@@ -571,7 +579,9 @@ + total_gpus = as_int(nested_get(audit, "gpus", "total")) + inventory_nodes = as_int(nested_get(audit, "nodes", "total"), default=1) + gpu_nodes = as_int( +- first_known(primary_profile.get("nodeCount"), nested_get(audit, "gpus", "nodeCount"), default=0) ++ first_known( ++ primary_profile.get("nodeCount"), nested_get(audit, "gpus", "nodeCount"), default=0 ++ ) + ) + cluster_nodes = gpu_nodes if gpu_nodes > 0 else inventory_nodes + # Prefer the audited per-node GPU count (from node labels/capacity). Dividing +@@ -588,7 +598,9 @@ + gpus_per_node = 0 + if gpu_nodes > 0: + total_cpus = as_int( +- first_known(primary_profile.get("totalCpus"), nested_get(audit, "gpus", "totalCpus"), default=0) ++ first_known( ++ primary_profile.get("totalCpus"), nested_get(audit, "gpus", "totalCpus"), default=0 ++ ) + ) + total_memory_gb = as_int( + first_known( +@@ -606,16 +618,16 @@ + if total_cpus <= 0: + total_cpus = as_int(nested_get(audit, "nodes", "sampleCpu")) * cluster_nodes + if total_memory_gb <= 0: +- sample_memory_gib = kubernetes_memory_gib( +- nested_get(audit, "nodes", "sampleMemory") +- ) ++ sample_memory_gib = kubernetes_memory_gib(nested_get(audit, "nodes", "sampleMemory")) + total_memory_gb = int(sample_memory_gib * cluster_nodes) + + cluster: dict[str, Any] = { + # Record the bare accelerator name (B300, GB300, H100, MI300X, ...). + # The raw nvidia-smi / label string stays under audit_data.gpus.model. + "gpu_model": normalize_chip_name( +- first_known(primary_profile.get("model"), nested_get(audit, "gpus", "model"), default="unknown") ++ first_known( ++ primary_profile.get("model"), nested_get(audit, "gpus", "model"), default="unknown" ++ ) + ), + "nodes": cluster_nodes, + "gpus_per_node": gpus_per_node, +@@ -652,7 +664,9 @@ + } + + +-def print_summary(out_path: Path, values: dict[str, Any], nic_topology: dict[str, list[dict[str, Any]]]) -> None: ++def print_summary( ++ out_path: Path, values: dict[str, Any], nic_topology: dict[str, list[dict[str, Any]]] ++) -> None: + cluster = values["cluster"] + audit = values["audit_data"] + print(f"Wrote {out_path}") +@@ -683,7 +697,9 @@ + for nics in nic_topology.values(): + for nic in nics: + by_role[nic["role"]] = by_role.get(nic["role"], 0) + 1 +- print(f" nic_topology = {len(nic_topology)} node(s), {total_nics} NIC(s) total: {by_role}") ++ print( ++ f" nic_topology = {len(nic_topology)} node(s), {total_nics} NIC(s) total: {by_role}" ++ ) + + + def main(argv: list[str]) -> int: + +--- cmax/scripts/1-audit/minimum_versions.py ++++ cmax/scripts/1-audit/minimum_versions.py +@@ -31,12 +31,8 @@ + DEFAULT_GRACE_PERIOD_DAYS = 3 + GRACE_REMINDER = "(passes as bulletin released within past 3 days)" + FIX_GRACE_REMINDER = "(passes as fix became available within past 3 days)" +-FIX_UNCONFIRMED_REMINDER = ( +- "(passes because fixed release availability is not yet confirmed)" +-) +-MINIMUM_NOT_EFFECTIVE_REMINDER = ( +- "(passes because the minimum version is not yet effective)" +-) ++FIX_UNCONFIRMED_REMINDER = "(passes because fixed release availability is not yet confirmed)" ++MINIMUM_NOT_EFFECTIVE_REMINDER = "(passes because the minimum version is not yet effective)" + + _CACHE: dict[str, Any] | None = None + +@@ -226,9 +222,7 @@ + bulletin_stamps = [stamp for stamp in bulletins if stamp is not None] + reference = today or datetime.now(timezone.utc).date() + days = grace_period_days(path) +- availability_records = [ +- minimum_fix_availability(name, value, path) for value in selectors +- ] ++ availability_records = [minimum_fix_availability(name, value, path) for value in selectors] + parsed_availability = [ + _parse_timestamp((availability or {}).get("available")) + for availability in availability_records +@@ -261,14 +255,10 @@ + + candidates = [ + (bulletin, available) +- for bulletin, available in zip( +- bulletin_stamps, parsed_availability, strict=True +- ) ++ for bulletin, available in zip(bulletin_stamps, parsed_availability, strict=True) + if available is not None + ] +- bulletin_released, available = max( +- candidates, key=lambda item: max(item[0], item[1]) +- ) ++ bulletin_released, available = max(candidates, key=lambda item: max(item[0], item[1])) + grace_started = max(bulletin_released, available) + grace_date = grace_started.astimezone(timezone.utc).date() + elapsed = (reference - grace_date).days +@@ -295,9 +285,7 @@ + "gracePeriodDays": days, + "graceStart": grace_started.isoformat().replace("+00:00", "Z"), + "enforcementDate": (grace_date + timedelta(days=days + 1)).isoformat(), +- "message": ( +- FIX_GRACE_REMINDER if available > bulletin_released else GRACE_REMINDER +- ), ++ "message": (FIX_GRACE_REMINDER if available > bulletin_released else GRACE_REMINDER), + } + + +@@ -331,9 +319,7 @@ + audit_dir = parent / "cmax" / "scripts" / "1-audit" + if (parent / ".git").exists() and audit_dir.is_dir(): + return "Run `git pull` to get the current minimums" +- return ( +- "Run `cmax audit security` to fetch the published minimums" +- ) ++ return "Run `cmax audit security` to fetch the published minimums" + + + def staleness_message(path: Path | None = None, *, now: datetime | None = None) -> str | None: +@@ -414,7 +400,9 @@ + value = get(args.get, path) + if value is None: + return 1 +- print(value if not isinstance(value, (dict, list)) else json.dumps(value, sort_keys=True)) ++ print( ++ value if not isinstance(value, (dict, list)) else json.dumps(value, sort_keys=True) ++ ) + elif args.age_days: + age = age_days(path) + print(f"{age:.2f}" if age is not None else "unknown") + +--- cmax/scripts/1-audit/plan_audit.py ++++ cmax/scripts/1-audit/plan_audit.py +@@ -16,18 +16,18 @@ + + + def command_ok(command: list[str]) -> bool: +- return subprocess.run( +- command, +- stdout=subprocess.DEVNULL, +- stderr=subprocess.DEVNULL, +- ).returncode == 0 ++ return ( ++ subprocess.run( ++ command, ++ stdout=subprocess.DEVNULL, ++ stderr=subprocess.DEVNULL, ++ ).returncode ++ == 0 ++ ) + + + def detect_harness() -> str: +- override = ( +- os.environ.get("CLUSTERMAX_AUDIT_HARNESS") +- or os.environ.get("CLUSTERMAX_HARNESS") +- ) ++ override = os.environ.get("CLUSTERMAX_AUDIT_HARNESS") or os.environ.get("CLUSTERMAX_HARNESS") + if override: + return override + +@@ -117,9 +117,7 @@ + run_results_dir = os.environ.get("RUN_RESULTS_DIR") + if run_results_dir: + out_dir = Path(run_results_dir) +- slug = os.environ.get("CLUSTER_SLUG") or slug_from_results_dir( +- repo_root, out_dir +- ) ++ slug = os.environ.get("CLUSTER_SLUG") or slug_from_results_dir(repo_root, out_dir) + else: + slug = cluster_slug() + out_dir = repo_root / "runs" / slug / now_ts() / "audit" + +--- cmax/scripts/1-audit/run_checks.py ++++ cmax/scripts/1-audit/run_checks.py +@@ -16,12 +16,8 @@ + CHECK_PROFILES = { + "fabric/nccl-ib-qps-check.py": frozenset({"networking"}), + "fabric/nccl-topology-file-check.py": frozenset({"networking"}), +- "fabric/nic-topology-check.py": frozenset( +- {"security", "versions", "isolation", "networking"} +- ), +- "fabric/virtio-net-check.py": frozenset( +- {"security", "versions", "isolation"} +- ), ++ "fabric/nic-topology-check.py": frozenset({"security", "versions", "isolation", "networking"}), ++ "fabric/virtio-net-check.py": frozenset({"security", "versions", "isolation"}), + "gpu/vboost.py": frozenset({"hardware"}), + "system/arm-smmu-virtualization-check.py": frozenset({"hardware"}), + "system/hbm_memory_exposure.py": frozenset({"hardware", "orchestration"}), +@@ -33,9 +29,7 @@ + "fabric/nic-topology-check.py": frozenset({"slurm", "k8s"}), + "fabric/virtio-net-check.py": frozenset({"slurm", "k8s"}), + "gpu/vboost.py": frozenset({"standalone", "slurm", "k8s"}), +- "system/arm-smmu-virtualization-check.py": frozenset( +- {"standalone", "slurm", "k8s"} +- ), ++ "system/arm-smmu-virtualization-check.py": frozenset({"standalone", "slurm", "k8s"}), + "system/hbm_memory_exposure.py": frozenset({"standalone", "slurm", "k8s"}), + "system/vm-iommu-check.py": frozenset({"standalone", "slurm", "k8s"}), + } +@@ -88,8 +82,7 @@ + and os.access(path, os.X_OK) + and "__pycache__" not in path.parts + and not path.name.startswith(".") +- and harness +- in CHECK_HARNESSES.get(str(path.relative_to(root)), VALID_HARNESSES) ++ and harness in CHECK_HARNESSES.get(str(path.relative_to(root)), VALID_HARNESSES) + ] + if profile == "full": + return checks +@@ -202,10 +195,8 @@ + PLATFORM_CHECK_KEYS[relative] + for check in checks + if (relative := str(check.relative_to(check_root))) in PLATFORM_CHECK_KEYS +- ) +- gpu_operator_namespace = ( +- discover_gpu_operator_namespace() if harness == "k8s" else None + ) ++ gpu_operator_namespace = discover_gpu_operator_namespace() if harness == "k8s" else None + if checks: + print("") + for check in checks: +@@ -222,11 +213,7 @@ + # A checkout shared from Windows can carry CRLF even when the Git blob + # uses LF. Launch Python checks through this interpreter so Linux never + # asks /usr/bin/env to resolve the shebang as the invalid `python3\r`. +- argv = ( +- [sys.executable, str(check)] +- if check.suffix == ".py" +- else [str(check)] +- ) ++ argv = [sys.executable, str(check)] if check.suffix == ".py" else [str(check)] + proc = subprocess.run( + argv, + stdout=subprocess.PIPE, + +--- cmax/scripts/1-audit/security_version_audit.py ++++ cmax/scripts/1-audit/security_version_audit.py +@@ -217,10 +217,7 @@ + for vuln_id, entries in sorted(ranges.items()): + if not isinstance(entries, list): + continue +- if not any( +- isinstance(entry, dict) and _version_in_range(key, entry) +- for entry in entries +- ): ++ if not any(isinstance(entry, dict) and _version_in_range(key, entry) for entry in entries): + continue + fixes = [ + str(entry["fixed"]) +@@ -491,9 +488,7 @@ + return _minimums_unavailable(version, MinimumDataError(str(exc))) + + advisory = _advisory(block) +- listed = " or ".join( +- branches[branch] for branch in sorted(branches, reverse=True) +- ) ++ listed = " or ".join(branches[branch] for branch in sorted(branches, reverse=True)) + if version.strip().lower() in NOT_INSTALLED_VALUES: + return _not_installed(version, listed, advisory, "runc") + parsed = numeric_version(version) +@@ -578,7 +573,9 @@ + if isinstance(retired, str) and numeric_version(retired, parts=3): + legacy[name] = retired + if not fixed: +- raise MinimumDataError("minimum version table has no fixed release lines for virtioNetBluefield") ++ raise MinimumDataError( ++ "minimum version table has no fixed release lines for virtioNetBluefield" ++ ) + return fixed, legacy + + +@@ -619,9 +616,7 @@ + return candidates + return tuple( + sorted( +- name +- for name, value in fixed.items() +- if numeric_version(value, parts=3)[0] == parsed[0] ++ name for name, value in fixed.items() if numeric_version(value, parts=3)[0] == parsed[0] + ) + ) + +@@ -1019,13 +1014,11 @@ + from one host's reading, so "no controller installed" or "could not be read" + on that host says nothing at all about the hosts that never answered. + """ +- return _weigh_coverage_gap( +- verdict, withdraw=("pass", "unknown", "not_applicable") +- ) ++ return _weigh_coverage_gap(verdict, withdraw=("pass", "unknown", "not_applicable")) + + + def _worst_observed_verdict( +- entries: Iterable[dict[str, Any]] ++ entries: Iterable[dict[str, Any]], + ) -> tuple[Verdict, dict[str, Any]] | None: + """Grade every host reading and return the worst, with the reading behind it. + +@@ -1062,18 +1055,18 @@ + if numeric_version(version, parts=3) is None: + continue + mode = str(entry.get("mode") or "").strip().lower() +- verdict = virtio_net_verdict( +- version, line=entry.get("line"), mode=entry.get("mode") +- ) ++ verdict = virtio_net_verdict(version, line=entry.get("line"), mode=entry.get("mode")) + minimum = _virtio_running_verdict(version, line=entry.get("line")).status +- ranked.append(( +- _VIRTIO_SEVERITY.get(verdict.status, 2), +- _VIRTIO_MINIMUM_SEVERITY.get(minimum, 1), +- _VIRTIO_MODE_SEVERITY.get(mode, 1), +- -index, +- verdict, +- entry, +- )) ++ ranked.append( ++ ( ++ _VIRTIO_SEVERITY.get(verdict.status, 2), ++ _VIRTIO_MINIMUM_SEVERITY.get(minimum, 1), ++ _VIRTIO_MODE_SEVERITY.get(mode, 1), ++ -index, ++ verdict, ++ entry, ++ ) ++ ) + if not ranked: + return None + _, _, _, _, verdict, entry = max(ranked, key=lambda item: item[:4]) +@@ -1249,9 +1242,7 @@ + running = {"dpu": True, "nic": False, "absent": False}.get(normalized) + graded = numeric_version(graded_version, parts=3) is not None + grace_component = ( +- "virtioNetBluefield" +- if coverage_complete and every_bluefield_host_read +- else None ++ "virtioNetBluefield" if coverage_complete and every_bluefield_host_read else None + ) + grace_selectors = _virtio_minimum_selectors(graded_version, graded_line) + grace_selector: str | tuple[str, ...] | None = None +@@ -1386,9 +1377,7 @@ + device_node = rshim.get("rshimDeviceNode") + tmfifo = rshim.get("tmfifoNet0") + reachable = [ +- name +- for name, seen in (("/dev/rshim0", device_node), ("tmfifo_net0", tmfifo)) +- if seen ++ name for name, seen in (("/dev/rshim0", device_node), ("tmfifo_net0", tmfifo)) if seen + ] + + # The reachability proof is read before the scan gate below, because the two +@@ -1497,9 +1486,7 @@ + # Hosts the check could not assess. Recorded so a fail carries its + # coverage gap as structured data that a consumer can filter on. + "unassessedHosts": [ +- str(host).strip() +- for host in (facts.get("unassessedHosts") or []) +- if str(host).strip() ++ str(host).strip() for host in (facts.get("unassessedHosts") or []) if str(host).strip() + ], + "remediation": MLXPRIVHOST_REMEDIATION, + } +@@ -1615,9 +1602,7 @@ + None, + ) + if minimum_record is None: +- minimum_record = _verdict_record( +- connectx_firmware_verdict("unknown"), "connectxFirmware" +- ) ++ minimum_record = _verdict_record(connectx_firmware_verdict("unknown"), "connectxFirmware") + result: dict[str, object] = { + "status": status, + "minimum": minimum_record["minimum"], +@@ -1748,13 +1733,9 @@ + driver_result, + "nvidiaDriver", + str(driver_parsed[0]) if driver_parsed else None, +- ), +- "nvidiaContainerToolkit": _verdict_record( +- nct_result, "nvidiaContainerToolkit" +- ), +- "cudaToolkit": _verdict_record( +- cuda_toolkit_verdict(cuda), "cudaToolkit" + ), ++ "nvidiaContainerToolkit": _verdict_record(nct_result, "nvidiaContainerToolkit"), ++ "cudaToolkit": _verdict_record(cuda_toolkit_verdict(cuda), "cudaToolkit"), + "docker": _verdict_record(docker_verdict(docker), "docker"), + "runc": _verdict_record( + runc_verdict(runc), +@@ -1764,14 +1745,10 @@ + "connectxFirmware": aggregate_connectx( + connectx_firmware, inventory_complete=connectx_inventory_complete + ), +- "virtioNetBluefield": explain_unreadable_virtio_version( +- virtio_record, isolation_record +- ), ++ "virtioNetBluefield": explain_unreadable_virtio_version(virtio_record, isolation_record), + "dpuHostIsolation": isolation_record, + "dcgm": _verdict_record(dcgm_result, "dcgm"), +- "dcgmExporter": _verdict_record( +- dcgm_exporter_result, "dcgmExporter" +- ), ++ "dcgmExporter": _verdict_record(dcgm_exporter_result, "dcgmExporter"), + "floorsMetadata": minimums_metadata(), + } + +@@ -1799,7 +1776,9 @@ + parsed = json.loads(raw) + except json.JSONDecodeError: + return [] +- return [entry for entry in parsed if isinstance(entry, dict)] if isinstance(parsed, list) else [] ++ return ( ++ [entry for entry in parsed if isinstance(entry, dict)] if isinstance(parsed, list) else [] ++ ) + + + def main() -> int: + +--- cmax/scripts/1-audit/validate_audit.py ++++ cmax/scripts/1-audit/validate_audit.py +@@ -71,7 +71,10 @@ + return True + + print("", file=sys.stderr) +- print(f"ERROR: audit incomplete (software.workerCheckOk={str(worker_check_ok).lower()})", file=sys.stderr) ++ print( ++ f"ERROR: audit incomplete (software.workerCheckOk={str(worker_check_ok).lower()})", ++ file=sys.stderr, ++ ) + print(" GPU / driver / NCCL fields would be 'unknown' or missing.", file=sys.stderr) + print(f" Audit raw JSON saved at: {audit_path}", file=sys.stderr) + print(" Investigate worker reachability and re-run; not publishing.", file=sys.stderr) + +--- cmax/security.py ++++ cmax/security.py +@@ -107,10 +107,7 @@ + value = path.read_text(errors="replace").lower() + except OSError: + continue +- if any( +- token in value +- for token in ("docker", "containerd", "kubepods", "libpod", "lxc") +- ): ++ if any(token in value for token in ("docker", "containerd", "kubepods", "libpod", "lxc")): + return True + return bool(os.environ.get("container")) + +@@ -156,9 +153,7 @@ + ) + for path in dmi_paths: + try: +- if any( +- marker in path.read_text(errors="replace").lower() for marker in markers +- ): ++ if any(marker in path.read_text(errors="replace").lower() for marker in markers): + return True + except OSError: + continue +@@ -262,9 +257,7 @@ + try: + config = yaml.safe_load(config_path.read_text()) or {} + except OSError as exc: +- raise SecurityAuditError( +- f"security audit configuration not found: {config_path}" +- ) from exc ++ raise SecurityAuditError(f"security audit configuration not found: {config_path}") from exc + except yaml.YAMLError as exc: + raise SecurityAuditError( + f"invalid security audit configuration: {config_path}: {exc}" +@@ -321,9 +314,7 @@ + } + + +-def format_security_plan_yaml( +- target: SecurityTarget, *, repo: str | None = None +-) -> str: ++def format_security_plan_yaml(target: SecurityTarget, *, repo: str | None = None) -> str: + runtime_root = find_runtime_root(repo) + yaml = load_yaml_module(runtime_root) + plan = build_security_plan(runtime_root, target) +@@ -383,9 +374,7 @@ + detail or "The component is not applicable to this host.", + ) + if status == "pass": +- suffix = detail or ( +- "The observed version meets the minimum version." +- ) ++ suffix = detail or ("The observed version meets the minimum version.") + return PASS, observed, suffix + if status == "fail": + return ( +@@ -396,8 +385,7 @@ + return ( + WARNING, + observed, +- detail +- or "The host version could not be verified, so exposure cannot be ruled out.", ++ detail or "The host version could not be verified, so exposure cannot be ruled out.", + ) + + return evaluate +@@ -421,8 +409,7 @@ + return [ + f"{_display(device.get('device'))}={_display(device.get('version'))}" + for device in devices +- if isinstance(device, dict) +- and str(device.get("status") or "unknown").lower() in wanted ++ if isinstance(device, dict) and str(device.get("status") or "unknown").lower() in wanted + ] + + if status in {"not_applicable", "not-applicable"}: +@@ -509,9 +496,7 @@ + observed = _display(value) + if isinstance(parent, dict): + fields = [ +- (name, item) +- for name, item in parent.items() +- if not isinstance(item, (dict, list)) ++ (name, item) for name, item in parent.items() if not isinstance(item, (dict, list)) + ] + fields.sort(key=lambda item: (item[0] != "status", item[0])) + observed = "; ".join(f"{name}={_display(item)}" for name, item in fields) +@@ -699,9 +684,7 @@ + ) + + +-def _boolean_exposure( +- path: str, label: str +-) -> Callable[[dict[str, Any]], tuple[str, str, str]]: ++def _boolean_exposure(path: str, label: str) -> Callable[[dict[str, Any]], tuple[str, str, str]]: + def evaluate(audit: dict[str, Any]) -> tuple[str, str, str]: + value = _get(audit, path) + if value is True or str(value).lower() == "true": +@@ -773,9 +756,7 @@ + ) + + +-def _manual_boundary( +- path: str, label: str +-) -> Callable[[dict[str, Any]], tuple[str, str, str]]: ++def _manual_boundary(path: str, label: str) -> Callable[[dict[str, Any]], tuple[str, str, str]]: + def evaluate(audit: dict[str, Any]) -> tuple[str, str, str]: + value = _get(audit, path) + if value is True or str(value).lower() == "true": +@@ -1083,9 +1064,7 @@ + return block if isinstance(block, dict) else {} + + +-def minimum_freshness( +- *, repo: str | None = None, now: datetime | None = None +-) -> dict[str, Any]: ++def minimum_freshness(*, repo: str | None = None, now: datetime | None = None) -> dict[str, Any]: + """Report the age of the generated minimum table this report graded against. + + Freshness is a statement about this tool rather than about the cluster, so +@@ -1418,8 +1397,7 @@ + ), + "cuda-toolkit": ( + "cudaToolkit", +- "Upgrade the CUDA Toolkit to {minimum} or a newer vendor-supported " +- "release.", ++ "Upgrade the CUDA Toolkit to {minimum} or a newer vendor-supported release.", + "NVIDIA CUDA Toolkit bulletin", + ), + "runc": ( +@@ -1438,8 +1416,7 @@ + "connectx-firmware": ( + "connectxFirmware", + # Per firmware train, so no single minimum to name. +- "Apply the fixed NVIDIA firmware release for every detected ConnectX " +- "or BlueField device.", ++ "Apply the fixed NVIDIA firmware release for every detected ConnectX or BlueField device.", + "NVIDIA networking security bulletin", + ), + "virtio-net-bluefield": ( +@@ -1472,18 +1449,12 @@ + if isinstance(advisory, str) and advisory: + references.append(SecurityReference(bulletin_label, advisory)) + for cve in block.get("cves") or (): +- references.append( +- SecurityReference(str(cve), f"https://nvd.nist.gov/vuln/detail/{cve}") +- ) ++ references.append(SecurityReference(str(cve), f"https://nvd.nist.gov/vuln/detail/{cve}")) + return remediation, (*references, *spec.references) + + + def evaluate_security(values: dict[str, Any]) -> list[SecurityCheck]: +- audit = ( +- values.get("audit_data") +- if isinstance(values.get("audit_data"), dict) +- else values +- ) ++ audit = values.get("audit_data") if isinstance(values.get("audit_data"), dict) else values + checks: list[SecurityCheck] = [] + grace_paths = { + "nvidia-driver": "securityVersions.nvidiaDriver", +@@ -1589,9 +1560,7 @@ + assessment = check.observed + if verbosity >= 3: + assessment = check.assessment +- details.extend( +- (("Observed", check.observed), ("Why", check.importance)) +- ) ++ details.extend((("Observed", check.observed), ("Why", check.importance))) + # Remediation is for a check that found something to fix. A pass + # has nothing, and neither does a criterion for hardware this + # machine does not have: printing "apply the zero-trust host +@@ -1603,8 +1572,7 @@ + elif verbosity >= 3 and check.status not in (PASS, NOT_APPLICABLE): + recommendation = check.remediation + all_references = tuple( +- (reference.label, reference.url) +- for reference in check.references ++ (reference.label, reference.url) for reference in check.references + ) + references = minimum_links.canonical_references(all_references) + if verbosity < 2 and ( +@@ -1626,12 +1594,9 @@ + + summary = ", ".join( + ( ++ report_style.count(f"{total[CRITICAL]} failed", CRITICAL, color=color), + report_style.count( +- f"{total[CRITICAL]} failed", CRITICAL, color=color +- ), +- report_style.count( +- f"{total[WARNING]} warning" +- f"{'' if total[WARNING] == 1 else 's'}", ++ f"{total[WARNING]} warning{'' if total[WARNING] == 1 else 's'}", + WARNING, + color=color, + ), +@@ -1657,9 +1622,7 @@ + "host or provider verification." + ) + else: +- lines.append( +- _paint("No known critical exposure detected.", "green", color=color) +- ) ++ lines.append(_paint("No known critical exposure detected.", "green", color=color)) + + # Minimum-data staleness sits next to the summary, at every verbosity level: + # an operator who reads only the last few lines must still see that the +@@ -1669,9 +1632,7 @@ + if notice: + lines.append(f"{_paint('Minimum data:', 'yellow', color=color)} {notice}") + +- lines.append( +- _paint(f"report saved to {log_dir / 'security-report.log'}", "dim", color=color) +- ) ++ lines.append(_paint(f"report saved to {log_dir / 'security-report.log'}", "dim", color=color)) + if verbosity < 3: + hint = { + 1: "run with -vv for CVE and documentation links, -vvv for issue details and remediation", +@@ -1681,9 +1642,7 @@ + return "\n".join(lines) + + +-def write_reports( +- checks: list[SecurityCheck], target: SecurityTarget, log_dir: Path +-) -> None: ++def write_reports(checks: list[SecurityCheck], target: SecurityTarget, log_dir: Path) -> None: + payload = { + "schema_version": 1, + "target": asdict(target), +@@ -1773,9 +1732,7 @@ + installed_path: Path | None = None + try: + installed_path = minimum_table_path(repo) +- installed_generated = str( +- json.loads(installed_path.read_text()).get("generated") or "" +- ) ++ installed_generated = str(json.loads(installed_path.read_text()).get("generated") or "") + except (SecurityAuditError, OSError, json.JSONDecodeError, AttributeError): + # The installed table is missing or unreadable. The fetch is then the + # only path to a usable table, so it continues with no age minimum. +@@ -1805,9 +1762,7 @@ + flush=True, + ) + return True +- suffix = ( +- f" (generated {installed_generated})" if installed_generated else "" +- ) ++ suffix = f" (generated {installed_generated})" if installed_generated else "" + print( + f"minimum fetch warning: could not update the minimum table. " + f"Grading against the installed table{suffix}.", +@@ -1858,9 +1813,7 @@ + flush=True, + ) + return False +- writable = ( +- os.access(path, os.W_OK) if path.exists() else os.access(path.parent, os.W_OK) +- ) ++ writable = os.access(path, os.W_OK) if path.exists() else os.access(path.parent, os.W_OK) + if not writable: + print( + f"minimum refresh could not write {path}: the installed minimum table is " +@@ -1879,8 +1832,7 @@ + table = minimum_refresh.build_minimums(existing=existing or None) + except (Exception, SystemExit) as exc: # network, parse, or fail-closed abort + print( +- f"minimum refresh failed: {exc}. Continuing with the committed minimum " +- f"table {path}.", ++ f"minimum refresh failed: {exc}. Continuing with the committed minimum table {path}.", + flush=True, + ) + return False + +--- cmax/target_selection.py ++++ cmax/target_selection.py +@@ -90,9 +90,7 @@ + if not context and environ.get("KUBERNETES_SERVICE_HOST"): + context = "in-cluster service account" + if not context: +- raise TargetSelectionError( +- "Kubernetes was selected, but kubectl has no current context." +- ) ++ raise TargetSelectionError("Kubernetes was selected, but kubectl has no current context.") + + access = _run( + [kubectl, "--request-timeout=8s", "cluster-info"], +@@ -242,9 +240,7 @@ + environ=environ, + ) + contexts = list( +- dict.fromkeys( +- line.strip() for line in listed.stdout.splitlines() if line.strip() +- ) ++ dict.fromkeys(line.strip() for line in listed.stdout.splitlines() if line.strip()) + ) + if listed.returncode != 0 or not contexts: + raise TargetSelectionError( + +23 files would be reformatted, 16 files already formatted diff --git a/reports/static-analysis/ruff.txt b/reports/static-analysis/ruff.txt new file mode 100644 index 0000000..d083319 --- /dev/null +++ b/reports/static-analysis/ruff.txt @@ -0,0 +1,2739 @@ +SIM114 [*] Combine `if` branches using logical `or` operator + --> cmax/audit_report.py:1079:9 + | +1077 | if any(rule.severity in {findings.MISSING, findings.VERSION} for rule in failing): +1078 | status = FAIL +1079 | / elif failing: +1080 | | status = WARNING +1081 | | elif unverified: +1082 | | status = WARNING + | |____________________________^ +1083 | else: +1084 | status = PASS + | +help: Combine `if` branches + | +1078 | status = FAIL + - elif failing: + - status = WARNING + - elif unverified: +1079 + elif failing or unverified: +1080 | status = WARNING + | + +E501 Line too long (125 > 100) + --> cmax/audit_report.py:1214:101 + | +1212 | if verbosity < 3: +1213 | hint = { +1214 | 1: f"run '{command} -vv' for CVE and documentation links, -vvv for issue details, reproduction, and remediation", + | ^^^^^^^^^^^^^^^^^^^^^^^^^ +1215 | 2: f"run '{command} -vvv' for issue details, reproduction, and remediation", +1216 | }[verbosity] + | + +UP035 [*] Import from `collections.abc` instead: `Callable` + --> cmax/audit_review.py:11:1 + | + 9 | from dataclasses import dataclass +10 | from pathlib import Path +11 | from typing import Callable, TextIO + | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +12 | +13 | from cmax import audit_report + | +help: Import from `collections.abc` + | +10 | from pathlib import Path + - from typing import Callable, TextIO +11 + from typing import TextIO +12 + from collections.abc import Callable +13 | + | + +UP035 [*] Import from `collections.abc` instead: `Sequence` + --> cmax/banner.py:40:1 + | +38 | import sys +39 | from dataclasses import dataclass +40 | from typing import Sequence, TextIO + | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +41 | +42 | from cmax import logo + | +help: Import from `collections.abc` + | +39 | from dataclasses import dataclass + - from typing import Sequence, TextIO +40 + from typing import TextIO +41 + from collections.abc import Sequence +42 | + | + +F401 [*] `cmax.logo.BASIC` imported but unused + --> cmax/banner.py:43:23 + | +42 | from cmax import logo +43 | from cmax.logo import BASIC, PALETTE256, TRUECOLOR, Ink, color_depth + | ^^^^^ +44 | from cmax.progress import Theme, progress_enabled + | +help: Remove unused import + | +42 | from cmax import logo + - from cmax.logo import BASIC, PALETTE256, TRUECOLOR, Ink, color_depth +43 + from cmax.logo import Ink, color_depth +44 | from cmax.progress import Theme, progress_enabled + | + +F401 [*] `cmax.logo.PALETTE256` imported but unused + --> cmax/banner.py:43:30 + | +42 | from cmax import logo +43 | from cmax.logo import BASIC, PALETTE256, TRUECOLOR, Ink, color_depth + | ^^^^^^^^^^ +44 | from cmax.progress import Theme, progress_enabled + | +help: Remove unused import + | +42 | from cmax import logo + - from cmax.logo import BASIC, PALETTE256, TRUECOLOR, Ink, color_depth +43 + from cmax.logo import Ink, color_depth +44 | from cmax.progress import Theme, progress_enabled + | + +F401 [*] `cmax.logo.TRUECOLOR` imported but unused + --> cmax/banner.py:43:42 + | +42 | from cmax import logo +43 | from cmax.logo import BASIC, PALETTE256, TRUECOLOR, Ink, color_depth + | ^^^^^^^^^ +44 | from cmax.progress import Theme, progress_enabled + | +help: Remove unused import + | +42 | from cmax import logo + - from cmax.logo import BASIC, PALETTE256, TRUECOLOR, Ink, color_depth +43 + from cmax.logo import Ink, color_depth +44 | from cmax.progress import Theme, progress_enabled + | + +SIM300 [*] Yoda condition detected + --> cmax/banner.py:356:8 + | +354 | """ +355 | found = logo.defects() +356 | if logo.HEIGHT != BLOCK_FONT.height: + | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +357 | found.append( +358 | f"logo is {logo.HEIGHT} rows and the block font is " + | +help: Rewrite as `BLOCK_FONT.height != logo.HEIGHT` + | +355 | found = logo.defects() + - if logo.HEIGHT != BLOCK_FONT.height: +356 + if BLOCK_FONT.height != logo.HEIGHT: +357 | found.append( + | + +C408 Unnecessary `dict()` call (rewrite as a literal) + --> cmax/cli.py:431:26 + | +429 | if target is None: +430 | return AUDIT_CANCEL_EXIT +431 | runner_options = dict( + | __________________________^ +432 | | repo=args.repo, +433 | | verbosity=_audit_verbosity(args.verbose), +434 | | category=category, +435 | | resolved_target=target, +436 | | ) + | |_________^ +437 | if exit_on_fail: +438 | runner_options["exit_on_fail"] = True + | +help: Rewrite as a literal + +I001 [*] Import block is un-sorted or un-formatted + --> cmax/criteria_links.py:3:1 + | +1 | """Stable links from audit checks to the public evaluation criteria.""" +2 | +3 | from __future__ import annotations + | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +help: Organize imports + | +4 | + - +5 | CRITERIA_URL = "https://www.clustermax.ai/criteria" + | + +I001 [*] Import block is un-sorted or un-formatted + --> cmax/minimum_links.py:3:1 + | +1 | """Stable links from audit checks to the published minimum-version table.""" +2 | +3 | / from __future__ import annotations +4 | | +5 | | from collections.abc import Iterable + | |____________________________________^ +help: Organize imports + | +6 | + - +7 | MINIMUM_VERSIONS_URL = "https://www.clustermax.ai/minimum-versions" + | + +UP035 [*] Import from `collections.abc` instead: `Iterator`, `Sequence` + --> cmax/minimum_refresh.py:70:1 + | +68 | from html.parser import HTMLParser +69 | from pathlib import Path +70 | from typing import Any, Iterator, Sequence + | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +71 | +72 | from cmax import runtime_paths + | +help: Import from `collections.abc` + | +69 | from pathlib import Path + - from typing import Any, Iterator, Sequence +70 + from typing import Any +71 + from collections.abc import Iterator, Sequence +72 | + | + +UP035 [*] Import from `collections.abc` instead: `Callable`, `Iterable`, `Sequence` + --> cmax/progress.py:37:1 + | +35 | from dataclasses import dataclass +36 | from pathlib import Path +37 | from typing import Callable, Iterable, Sequence, TextIO + | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +38 | +39 | from prompt_toolkit.application import Application, get_app + | +help: Import from `collections.abc` + | +36 | from pathlib import Path + - from typing import Callable, Iterable, Sequence, TextIO +37 + from typing import TextIO +38 + from collections.abc import Callable, Iterable, Sequence +39 | + | + +B023 Function definition does not bind loop variable `code` + --> cmax/progress.py:155:54 + | +153 | rendered = text +154 | for pattern, code in _STATUS_WORDS: +155 | rendered = pattern.sub(lambda match: f"\x1b[{code}m{match.group(0)}\x1b[0m", rendered) + | ^^^^ +156 | # Findings use a padded seven-character category. Color only the text so +157 | # `[CONFIG ]` keeps the alignment operators rely on in screenshots. + | + +UP037 [*] Remove quotes from type annotation + --> cmax/progress.py:955:33 + | +953 | """Expose preformatted log rows through an independently moved viewport.""" +954 | +955 | def __init__(self, display: "LiveDisplay") -> None: + | ^^^^^^^^^^^^^ +956 | self.display = display +957 | self._cursor_line = max(0, len(display._timeline) - 1) + | +help: Remove quotes + | +954 | + - def __init__(self, display: "LiveDisplay") -> None: +955 + def __init__(self, display: LiveDisplay) -> None: +956 | self.display = display + | + +UP037 [*] Remove quotes from type annotation + --> cmax/progress.py:1174:28 + | +1172 | self._closed = False +1173 | +1174 | def __enter__(self) -> "LiveDisplay": + | ^^^^^^^^^^^^^ +1175 | self.start() +1176 | return self + | +help: Remove quotes + | +1173 | + - def __enter__(self) -> "LiveDisplay": +1174 + def __enter__(self) -> LiveDisplay: +1175 | self.start() + | + +SIM105 Use `contextlib.suppress(OSError, ValueError)` instead of `try`-`except`-`pass` + --> cmax/progress.py:1662:13 + | +1660 | self._tui_input_thread = None +1661 | if self._tui_stdin_attrs is not None and termios is not None: +1662 | / try: +1663 | | termios.tcsetattr( +1664 | | sys.stdin.fileno(), +1665 | | termios.TCSANOW, +1666 | | self._tui_stdin_attrs, +1667 | | ) +1668 | | except (OSError, ValueError): +1669 | | pass + | |____________________^ +1670 | self._tui_stdin_attrs = None + | +help: Replace `try`-`except`-`pass` with `with contextlib.suppress(OSError, ValueError): ...` + +SIM105 Use `contextlib.suppress(RuntimeError)` instead of `try`-`except`-`pass` + --> cmax/progress.py:1686:9 + | +1684 | app.exit() +1685 | +1686 | / try: +1687 | | app.loop.call_soon_threadsafe(exit_app) +1688 | | except RuntimeError: +1689 | | pass + | |________________^ +1690 | +1691 | def _invalidate_tui(self) -> None: + | +help: Replace `try`-`except`-`pass` with `with contextlib.suppress(RuntimeError): ...` + +UP037 [*] Remove quotes from type annotation + --> cmax/progress.py:1861:28 + | +1859 | self._last: Step | None = None +1860 | +1861 | def __enter__(self) -> "PlainDisplay": + | ^^^^^^^^^^^^^^ +1862 | self.start() +1863 | return self + | +help: Remove quotes + | +1860 | + - def __enter__(self) -> "PlainDisplay": +1861 + def __enter__(self) -> PlainDisplay: +1862 | self.start() + | + +I001 [*] Import block is un-sorted or un-formatted + --> cmax/report_style.py:3:1 + | +1 | """Shared terminal styling for ClusterMAX audit reports.""" +2 | +3 | / from __future__ import annotations +4 | | +5 | | from collections.abc import Iterable + | |____________________________________^ +help: Organize imports + | +6 | + - +7 | _ANSI = { + | + +I001 [*] Import block is un-sorted or un-formatted + --> cmax/runtime_paths.py:3:1 + | +1 | """Canonical paths for the audit runtime that ships with ClusterMAX.""" +2 | +3 | / from importlib import resources +4 | | from pathlib import Path + | |________________________^ +help: Organize imports + | +5 | + - +6 | AUDIT_RELATIVE = Path("scripts/1-audit") + | + +UP035 [*] Import from `collections.abc` instead: `Callable` + --> cmax/scripts/1-audit/audit_findings.py:39:1 + | +37 | from dataclasses import dataclass +38 | from pathlib import Path +39 | from typing import Any, Callable + | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +40 | from urllib.parse import unquote, urlparse + | +help: Import from `collections.abc` + | +38 | from pathlib import Path + - from typing import Any, Callable +39 + from typing import Any +40 + from collections.abc import Callable +41 | from urllib.parse import unquote, urlparse + | + +E501 Line too long (115 > 100) + --> cmax/scripts/1-audit/audit_findings.py:156:101 + | +154 | attest, so the worker-check attestation note should stay silent. +155 | """ +156 | return _is_true(nested_get(audit, "containers", "nvidiaContainerToolkit")) or _nvidia_runtime_configured(audit) + | ^^^^^^^^^^^^^^^ + +SIM102 Use a single `if` statement instead of nested `if` statements + --> cmax/scripts/1-audit/audit_findings.py:502:5 + | +500 | """True only when a completed inventory found no BlueField device.""" +501 | isolation = nested_get(audit, "securityVersions", "dpuHostIsolation") +502 | / if isinstance(isolation, dict): +503 | | if ( +504 | | isolation.get("scanComplete") is True +505 | | and isolation.get("bluefieldPresent") is False +506 | | ): + | |__________^ +507 | return True +508 | controller = nested_get(audit, "securityVersions", "virtioNetBluefield") + | +help: Combine `if` statements using `and` + +E501 Line too long (153 > 100) + --> cmax/scripts/1-audit/audit_findings.py:786:101 + | +784 | … +785 | … +786 | …time unverified from this vantage; requires provider attestation (not evidence of absence)", + | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +787 | … +788 | … + | + +E501 Line too long (114 > 100) + --> cmax/scripts/1-audit/audit_findings.py:933:101 + | +931 | Rule( +932 | "nccl_topo_file.status", +933 | "Host NCCL topology file is present but the container mount is unverified; requires provider attestation", + | ^^^^^^^^^^^^^^ +934 | CONFIG, +935 | _status_is("unknown"), + | + +E501 Line too long (104 > 100) + --> cmax/scripts/1-audit/audit_findings.py:941:101 + | +939 | Rule( +940 | "nccl_ib_qps.status", +941 | "NCCL_IB_QPS_PER_CONNECTION is at the default on a multi-tier Clos fabric; sweep it (advisory)", + | ^^^^ +942 | CONFIG, +943 | _status_is("warning"), + | + +E501 Line too long (116 > 100) + --> cmax/scripts/1-audit/audit_findings.py:1633:101 + | +1631 | if rule.severity == MISSING or "not installed" in rule.title.lower(): +1632 | detected = "Not installed" +1633 | elif any(word in rule.key.lower() for word in ("allowed", "configured")) or "enabled" in rule.title.lower(): + | ^^^^^^^^^^^^^^^^ +1634 | detected = "Disabled" +1635 | else: + | + +E501 Line too long (101 > 100) + --> cmax/scripts/1-audit/audit_findings.py:1665:101 + | +1663 | detected = str(value).replace("_", " ").capitalize() +1664 | if "reboot required" in rule.title.lower(): +1665 | recommendation = "Reboot the affected worker, and confirm that it runs the newer kernel." + | ^ +1666 | elif "attestation" in rule.title.lower() or "could not be verified" in rule.title.lower(): +1667 | recommendation = "Ask the provider to attest this state with host evidence." + | + +UP035 [*] Import from `collections.abc` instead: `Callable` + --> cmax/scripts/1-audit/checks/_fanout.py:28:1 + | +26 | import json +27 | from concurrent.futures import ThreadPoolExecutor +28 | from typing import Any, Callable, Optional + | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +29 | +30 | # (node) -> (record | None, error | None). ``node`` is whatever k8s_gpu_nodes + | +help: Import from `collections.abc` + | +27 | from concurrent.futures import ThreadPoolExecutor + - from typing import Any, Callable, Optional +28 + from typing import Any, Optional +29 + from collections.abc import Callable +30 | + | + +UP045 [*] Use `X | None` for type annotations + --> cmax/scripts/1-audit/checks/_fanout.py:32:34 + | +30 | # (node) -> (record | None, error | None). ``node`` is whatever k8s_gpu_nodes +31 | # produced; the caller's closure decides how to reach and check it. +32 | PerNode = Callable[[Any], "tuple[Optional[dict[str, Any]], Optional[str]]"] + | ^^^^^^^^^^^^^^^^^^^^^^^^ +33 | MAX_K8S_FANOUT_WORKERS = 8 + | +help: Convert to `X | None` + | +31 | # produced; the caller's closure decides how to reach and check it. + - PerNode = Callable[[Any], "tuple[Optional[dict[str, Any]], Optional[str]]"] +32 + PerNode = Callable[[Any], "tuple[dict[str, Any] | None, Optional[str]]"] +33 | MAX_K8S_FANOUT_WORKERS = 8 + | + +UP045 [*] Use `X | None` for type annotations + --> cmax/scripts/1-audit/checks/_fanout.py:32:60 + | +30 | # (node) -> (record | None, error | None). ``node`` is whatever k8s_gpu_nodes +31 | # produced; the caller's closure decides how to reach and check it. +32 | PerNode = Callable[[Any], "tuple[Optional[dict[str, Any]], Optional[str]]"] + | ^^^^^^^^^^^^^ +33 | MAX_K8S_FANOUT_WORKERS = 8 + | +help: Convert to `X | None` + | +31 | # produced; the caller's closure decides how to reach and check it. + - PerNode = Callable[[Any], "tuple[Optional[dict[str, Any]], Optional[str]]"] +32 + PerNode = Callable[[Any], "tuple[Optional[dict[str, Any]], str | None]"] +33 | MAX_K8S_FANOUT_WORKERS = 8 + | + +E501 Line too long (104 > 100) + --> cmax/scripts/1-audit/checks/_fanout.py:123:101 + | +121 | if max_nodes < len(nodes): +122 | errors.append( +123 | f"checked {max_nodes} of {len(nodes)} GPU nodes due to CLUSTERMAX_AUDIT_K8S_MAX_HOST_CHECKS" + | ^^^^ +124 | ) +125 | return records, errors + | + +I001 [*] Import block is un-sorted or un-formatted + --> cmax/scripts/1-audit/checks/fabric/nccl-ib-qps-check.py:12:1 + | +11 | sys.path.insert(0, str(Path(__file__).resolve().parents[1])) +12 | from platform_config import run_named_check + | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +help: Organize imports + | +13 | + - +14 | if __name__ == "__main__": + | + +I001 [*] Import block is un-sorted or un-formatted + --> cmax/scripts/1-audit/checks/fabric/nccl-topology-file-check.py:12:1 + | +11 | sys.path.insert(0, str(Path(__file__).resolve().parents[1])) +12 | from platform_config import run_named_check + | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +help: Organize imports + | +13 | + - +14 | if __name__ == "__main__": + | + +UP035 [*] Import from `collections.abc` instead: `Callable` + --> cmax/scripts/1-audit/checks/fabric/nic-topology-check.py:78:1 + | +76 | import sys +77 | from pathlib import Path +78 | from typing import Any, Callable + | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +79 | +80 | Runner = Callable[..., subprocess.CompletedProcess] + | +help: Import from `collections.abc` + | +77 | from pathlib import Path + - from typing import Any, Callable +78 + from typing import Any +79 + from collections.abc import Callable +80 | + | + +E741 Ambiguous variable name: `l` + --> cmax/scripts/1-audit/checks/fabric/nic-topology-check.py:158:33 + | +156 | header_idx = -1 +157 | if topo: +158 | lines = [l.rstrip() for l in topo.splitlines()] + | ^ +159 | for i, l in enumerate(lines): +160 | toks = l.split() + | + +E741 Ambiguous variable name: `l` + --> cmax/scripts/1-audit/checks/fabric/nic-topology-check.py:159:16 + | +157 | if topo: +158 | lines = [l.rstrip() for l in topo.splitlines()] +159 | for i, l in enumerate(lines): + | ^ +160 | toks = l.split() +161 | if toks and toks[0].startswith("GPU0"): + | + +E741 Ambiguous variable name: `l` + --> cmax/scripts/1-audit/checks/fabric/nic-topology-check.py:166:17 + | +164 | break +165 | if header: +166 | for l in lines[header_idx + 1:]: + | ^ +167 | toks = l.split() +168 | if not toks: + | + +B905 `zip()` without an explicit `strict=` parameter + --> cmax/scripts/1-audit/checks/fabric/nic-topology-check.py:176:29 + | +174 | cells = toks[1:1 + len(header)] +175 | aff = "far" +176 | for h, c in zip(header, cells): + | ^^^^^^^^^^^^^^^^^^ +177 | if h.startswith("GPU") and c in ("PIX", "PXB", "PHB"): +178 | aff = "close" + | +help: Add explicit value for parameter `strict=` + +C420 [*] Unnecessary dict comprehension for iterable; use `dict.fromkeys` instead + --> cmax/scripts/1-audit/checks/fabric/nic-topology-check.py:221:16 + | +219 | ] +220 | if rails: +221 | return { + | ________________^ +222 | | name: "rail-named RDMA HCA; no GPU-affinity signal on this platform" +223 | | for name in rails +224 | | } + | |_________^ +225 | +226 | ib = [ + | +help: Replace with `dict.fromkeys(iterable)`) + | +220 | if rails: + - return { + - name: "rail-named RDMA HCA; no GPU-affinity signal on this platform" + - for name in rails + - } +221 + return dict.fromkeys(rails, "rail-named RDMA HCA; no GPU-affinity signal on this platform") +222 | + | + +B007 Loop control variable `state` not used within loop body + --> cmax/scripts/1-audit/checks/fabric/nic-topology-check.py:267:28 + | +266 | first_pass = {} +267 | for name, rate, layer, state in hcas: + | ^^^^^ +268 | first_pass[name] = classify(name, rate, layer) + | +help: Rename unused `state` to `_state` + +SIM115 Use a context manager for opening files + --> cmax/scripts/1-audit/checks/fabric/nic-topology-check.py:309:31 + | +307 | continue +308 | try: +309 | rate = parse_rate(open(path + "/ports/1/rate").read()) + | ^^^^ +310 | except OSError: +311 | rate = 0.0 + | + +SIM115 Use a context manager for opening files + --> cmax/scripts/1-audit/checks/fabric/nic-topology-check.py:313:21 + | +311 | rate = 0.0 +312 | try: +313 | layer = open(path + "/ports/1/link_layer").read().strip() + | ^^^^ +314 | except OSError: +315 | layer = "?" + | + +SIM115 Use a context manager for opening files + --> cmax/scripts/1-audit/checks/fabric/nic-topology-check.py:317:21 + | +315 | layer = "?" +316 | try: +317 | state = open(path + "/ports/1/state").read().split(":")[1].split()[0] + | ^^^^ +318 | except (OSError, IndexError): +319 | state = "?" + | + +SIM115 Use a context manager for opening files + --> cmax/scripts/1-audit/checks/fabric/nic-topology-check.py:336:26 + | +334 | continue +335 | try: +336 | speed_mbps = open("/sys/class/net/" + n + "/speed").read().strip() + | ^^^^ +337 | speed_mbps = int(speed_mbps) if speed_mbps.lstrip("-").isdigit() else 0 +338 | except (OSError, ValueError): + | + +E501 Line too long (114 > 100) + --> cmax/scripts/1-audit/checks/fabric/nic-topology-check.py:411:101 + | +409 | ] +410 | try: +411 | proc = runner(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, timeout=SLURM_TIMEOUT_S) + | ^^^^^^^^^^^^^^ +412 | except (OSError, subprocess.TimeoutExpired) as exc: +413 | return [gather()], [f"srun NIC check failed; local host only: {exc}"] + | + +E501 Line too long (104 > 100) + --> cmax/scripts/1-audit/checks/fabric/nic-topology-check.py:469:101 + | +467 | try: +468 | proc = kubectl( +469 | ["get", "pods", "-n", namespace, "--field-selector", f"spec.nodeName={node}", "-o", "json"], + | ^^^^ +470 | runner=runner, +471 | timeout=45, + | + +E501 Line too long (110 > 100) + --> cmax/scripts/1-audit/checks/fabric/nic-topology-check.py:565:101 + | +563 | layer = parts[3].strip() or "?" +564 | state_raw = parts[4] +565 | state = state_raw.split(":")[1].split()[0] if ":" in state_raw else (state_raw.strip() or "?") + | ^^^^^^^^^^ +566 | hcas.append((name, rate, layer, state)) +567 | elif line.startswith("@@NET@@\t"): + | + +E501 Line too long (113 > 100) + --> cmax/scripts/1-audit/checks/fabric/nic-topology-check.py:576:101 + | +576 | def run_k8s_node_check(namespace: str, node: str, *, runner: Runner) -> tuple[dict[str, Any] | None, str | None]: + | ^^^^^^^^^^^^^ +577 | access = k8s_driver_pod(namespace, node, runner=runner) +578 | if not access: + | + +E501 Line too long (101 > 100) + --> cmax/scripts/1-audit/checks/fabric/nic-topology-check.py:579:101 + | +577 | access = k8s_driver_pod(namespace, node, runner=runner) +578 | if not access: +579 | return None, f"{node}: no running nvidia-smi-capable GPU Operator pod for NIC classification" + | ^ +580 | exec_args = ["exec", "-i", "-n", namespace, access["name"]] +581 | if access["container"]: + | + +E501 Line too long (111 > 100) + --> cmax/scripts/1-audit/checks/fabric/nic-topology-check.py:636:101 + | +634 | namespace = k8s_gpu_namespace(env=env, runner=runner) +635 | if not namespace: +636 | return [], ["no NVIDIA GPU Operator namespace found; cannot reach a driver pod for NIC classification"] + | ^^^^^^^^^^^ +637 | +638 | try: + | + +E501 Line too long (111 > 100) + --> cmax/scripts/1-audit/checks/fabric/nic-topology-check.py:643:101 + | +641 | max_nodes = len(nodes) +642 | +643 | return load_fanout().fan_out_k8s(_nic_per_node(namespace, runner=runner), nodes=nodes, max_nodes=max_nodes) + | ^^^^^^^^^^^ + +E501 Line too long (129 > 100) + --> cmax/scripts/1-audit/checks/fabric/nic-topology-check.py:678:101 + | +676 | def main(argv: list[str]) -> int: +677 | parser = argparse.ArgumentParser(description="NIC topology check") +678 | parser.add_argument("--collect-host", action="store_true", help="emit one host record instead of the aggregate check object") + | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +679 | parser.add_argument("--harness", default=os.environ.get("CLUSTERMAX_AUDIT_HARNESS") or os.environ.get("CLUSTERMAX_HARNESS") or "") +680 | args = parser.parse_args(argv) + | + +E501 Line too long (134 > 100) + --> cmax/scripts/1-audit/checks/fabric/nic-topology-check.py:679:101 + | +677 | parser = argparse.ArgumentParser(description="NIC topology check") +678 | parser.add_argument("--collect-host", action="store_true", help="emit one host record instead of the aggregate check object") +679 | parser.add_argument("--harness", default=os.environ.get("CLUSTERMAX_AUDIT_HARNESS") or os.environ.get("CLUSTERMAX_HARNESS") or "") + | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +680 | args = parser.parse_args(argv) + | + +UP035 [*] Import from `collections.abc` instead: `Callable` + --> cmax/scripts/1-audit/checks/fabric/virtio-net-check.py:137:1 + | +135 | import sys +136 | from pathlib import Path +137 | from typing import Any, Callable + | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +138 | +139 | Runner = Callable[..., subprocess.CompletedProcess] + | +help: Import from `collections.abc` + | +136 | from pathlib import Path + - from typing import Any, Callable +137 + from typing import Any +138 + from collections.abc import Callable +139 | + | + +E501 Line too long (112 > 100) + --> cmax/scripts/1-audit/checks/fabric/virtio-net-check.py:961:101 + | +959 | pci = collect_pci(runner=runner, env=env) +960 | devices = pci["devices"] +961 | mode = collect_mode(devices, runner=runner) if devices else {"mode": "unknown", "values": {}, "error": None} + | ^^^^^^^^^^^^ +962 | rshim = collect_rshim(mode.get("values") or {}, env=env) +963 | version = collect_version(rshim, env=env, runner=runner) + | + +F841 Local variable `states` is assigned to but never used + --> cmax/scripts/1-audit/checks/fabric/virtio-net-check.py:1120:5 + | +1119 | observed = observed_controllers(records) +1120 | states = {str(record.get("state") or STATE_INCOMPLETE) for record in records} + | ^^^^^^ +1121 | modes = {str(record.get("mode") or "unknown") for record in records} +1122 | versions = [str(record["version"]) for record in records if record.get("version")] + | +help: Remove assignment to unused variable `states` + +E501 Line too long (114 > 100) + --> cmax/scripts/1-audit/checks/fabric/virtio-net-check.py:1135:101 + | +1133 | None, +1134 | ) +1135 | source = str(oldest_record["versionSource"]) if oldest_record and oldest_record.get("versionSource") else None + | ^^^^^^^^^^^^^^ +1136 | # Only the line that belongs to the version being reported. A line read off +1137 | # a different node would grade this version against the wrong minimum. + | + +E501 Line too long (109 > 100) + --> cmax/scripts/1-audit/checks/fabric/virtio-net-check.py:1139:101 + | +1137 | # a different node would grade this version against the wrong minimum. +1138 | line = str(oldest_record["line"]) if oldest_record and oldest_record.get("line") else None +1139 | reasons = sorted({str(record.get("reason") or "") for record in records if record.get("state") == state}) + | ^^^^^^^^^ +1140 | +1141 | # Cluster mode, worst case first: one node still in DPU mode keeps the + | + +E501 Line too long (103 > 100) + --> cmax/scripts/1-audit/checks/fabric/virtio-net-check.py:1149:101 + | +1147 | elif "nic" in modes: +1148 | cluster_mode = "nic" +1149 | elif all(record.get("scanComplete") and not record.get("bluefield3Present") for record in records): + | ^^^ +1150 | cluster_mode = "absent" +1151 | else: + | + +E501 Line too long (114 > 100) + --> cmax/scripts/1-audit/checks/fabric/virtio-net-check.py:1379:101 + | +1377 | ] +1378 | try: +1379 | proc = runner(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, timeout=SLURM_TIMEOUT_S) + | ^^^^^^^^^^^^^^ +1380 | except (OSError, subprocess.TimeoutExpired) as exc: +1381 | return [gather(env=env, runner=runner)], [f"srun virtio-net check failed; local host only: {exc}"] + | + +E501 Line too long (106 > 100) + --> cmax/scripts/1-audit/checks/fabric/virtio-net-check.py:1381:101 + | +1379 | proc = runner(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, timeout=SLURM_TIMEOUT_S) +1380 | except (OSError, subprocess.TimeoutExpired) as exc: +1381 | return [gather(env=env, runner=runner)], [f"srun virtio-net check failed; local host only: {exc}"] + | ^^^^^^ +1382 | records = load_fanout().parse_json_lines(proc.stdout, require_host=True) +1383 | if not records: + | + +E501 Line too long (105 > 100) + --> cmax/scripts/1-audit/checks/fabric/virtio-net-check.py:1421:101 + | +1419 | printf '@@SCAN_ERROR@@\t%s\n' "lspci is not installed" +1420 | fi +1421 | slot=$(lspci -Dn 2>/dev/null | grep -Eio '^[0-9a-f:.]+ [^ ]* 15b3:(a2dc|a2d9)' | head -1 | cut -d' ' -f1) + | ^^^^^ +1422 | if [ -n "$slot" ] && command -v mlxconfig >/dev/null 2>&1; then +1423 | echo @@MLXCONFIG_BEGIN@@ + | + +SIM910 [*] Use `dict.get()` without default value + --> cmax/scripts/1-audit/checks/fabric/virtio-net-check.py:1495:28 + | +1493 | "tmfifoNet0": "tmfifo" in parsed["rshim"], +1494 | "internalCpuRshim": mode_values.get("INTERNAL_CPU_RSHIM", "unknown"), +1495 | "rshimRestricted": {"0": False, "1": True}.get(mode_values.get("INTERNAL_CPU_RSHIM", ""), None), + | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +1496 | "dpuReachedFromHost": None, +1497 | } + | +help: Remove default value + | +1494 | "internalCpuRshim": mode_values.get("INTERNAL_CPU_RSHIM", "unknown"), + - "rshimRestricted": {"0": False, "1": True}.get(mode_values.get("INTERNAL_CPU_RSHIM", ""), None), +1495 + "rshimRestricted": {"0": False, "1": True}.get(mode_values.get("INTERNAL_CPU_RSHIM", "")), +1496 | "dpuReachedFromHost": None, + | + +E501 Line too long (104 > 100) + --> cmax/scripts/1-audit/checks/fabric/virtio-net-check.py:1495:101 + | +1493 | "tmfifoNet0": "tmfifo" in parsed["rshim"], +1494 | "internalCpuRshim": mode_values.get("INTERNAL_CPU_RSHIM", "unknown"), +1495 | "rshimRestricted": {"0": False, "1": True}.get(mode_values.get("INTERNAL_CPU_RSHIM", ""), None), + | ^^^^ +1496 | "dpuReachedFromHost": None, +1497 | } + | + +E501 Line too long (113 > 100) + --> cmax/scripts/1-audit/checks/fabric/virtio-net-check.py:1525:101 + | +1525 | def kubectl(args: list[str], *, runner: Runner, timeout: int = KUBECTL_TIMEOUT_S) -> subprocess.CompletedProcess: + | ^^^^^^^^^^^^^ +1526 | return runner(["kubectl", *args], stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, timeout=timeout) + | + +E501 Line too long (113 > 100) + --> cmax/scripts/1-audit/checks/fabric/virtio-net-check.py:1526:101 + | +1525 | def kubectl(args: list[str], *, runner: Runner, timeout: int = KUBECTL_TIMEOUT_S) -> subprocess.CompletedProcess: +1526 | return runner(["kubectl", *args], stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, timeout=timeout) + | ^^^^^^^^^^^^^ + +E501 Line too long (104 > 100) + --> cmax/scripts/1-audit/checks/fabric/virtio-net-check.py:1558:101 + | +1556 | try: +1557 | proc = kubectl( +1558 | ["get", "pods", "-n", namespace, "--field-selector", f"spec.nodeName={node}", "-o", "json"], + | ^^^^ +1559 | runner=runner, +1560 | timeout=45, + | + +E501 Line too long (113 > 100) + --> cmax/scripts/1-audit/checks/fabric/virtio-net-check.py:1596:101 + | +1596 | def run_k8s_node_check(namespace: str, node: str, *, runner: Runner) -> tuple[dict[str, Any] | None, str | None]: + | ^^^^^^^^^^^^^ +1597 | access = k8s_driver_pod(namespace, node, runner=runner) +1598 | if not access: + | + +E501 Line too long (113 > 100) + --> cmax/scripts/1-audit/checks/fabric/virtio-net-check.py:1640:101 + | +1638 | namespace = k8s_namespace(env=env, runner=runner) +1639 | if not namespace: +1640 | return [], ["no NVIDIA GPU Operator namespace found; cannot reach a driver pod for the virtio-net check"] + | ^^^^^^^^^^^^^ +1641 | try: +1642 | max_nodes = int(str(env.get("CLUSTERMAX_AUDIT_K8S_MAX_HOST_CHECKS"))) + | + +E501 Line too long (114 > 100) + --> cmax/scripts/1-audit/checks/fabric/virtio-net-check.py:1669:101 + | +1667 | return { +1668 | "virtio_net_bluefield": { +1669 | "hosts": {str(record.get("host") or f"host-{index}"): record for index, record in enumerate(records)}, + | ^^^^^^^^^^^^^^ +1670 | "summary": summary, +1671 | } + | + +SIM105 Use `contextlib.suppress(OSError)` instead of `try`-`except`-`pass` + --> cmax/scripts/1-audit/checks/fabric/virtio-net-check.py:1762:9 + | +1760 | payload = build_check_payload(harness=harness, env=env) +1761 | if cache_path is not None: +1762 | / try: +1763 | | cache_path.write_text(json.dumps(payload, sort_keys=True)) +1764 | | except OSError: +1765 | | # The cache is only a runtime optimization. Collection remains +1766 | | # authoritative when the temporary directory cannot be written. +1767 | | pass + | |________________^ +1768 | return payload + | +help: Replace `try`-`except`-`pass` with `with contextlib.suppress(OSError): ...` + +E501 Line too long (116 > 100) + --> cmax/scripts/1-audit/checks/fabric/virtio-net-check.py:1773:101 + | +1771 | def main(argv: list[str]) -> int: +1772 | parser = argparse.ArgumentParser(description="BlueField VIRTIO-Net controller check") +1773 | parser.add_argument("--collect-host", action="store_true", help="emit one host record instead of the aggregate") + | ^^^^^^^^^^^^^^^^ +1774 | parser.add_argument( +1775 | "--summary", + | + +E501 Line too long (105 > 100) + --> cmax/scripts/1-audit/checks/fabric/virtio-net-check.py:1781:101 + | +1779 | parser.add_argument( +1780 | "--harness", +1781 | default=os.environ.get("CLUSTERMAX_AUDIT_HARNESS") or os.environ.get("CLUSTERMAX_HARNESS") or "", + | ^^^^^ +1782 | ) +1783 | args = parser.parse_args(argv) + | + +I001 [*] Import block is un-sorted or un-formatted + --> cmax/scripts/1-audit/checks/gpu/vboost.py:4:1 + | + 2 | """Check whether the current provider allows changing NVIDIA vboost.""" + 3 | + 4 | / from __future__ import annotations + 5 | | + 6 | | import json + 7 | | import os + 8 | | import re + 9 | | import shutil +10 | | import socket +11 | | import subprocess +12 | | import sys +13 | | from pathlib import Path +14 | | from typing import Any, Callable, Optional + | |__________________________________________^ +help: Organize imports + | +15 | + - +16 | TARGET_VBOOST = 2 + | + +UP035 [*] Import from `collections.abc` instead: `Callable` + --> cmax/scripts/1-audit/checks/gpu/vboost.py:14:1 + | +12 | import sys +13 | from pathlib import Path +14 | from typing import Any, Callable, Optional + | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +help: Import from `collections.abc` + | +13 | from pathlib import Path + - from typing import Any, Callable, Optional +14 + from typing import Any, Optional +15 + from collections.abc import Callable +16 | + | + +UP045 [*] Use `X | None` for type annotations + --> cmax/scripts/1-audit/checks/gpu/vboost.py:36:25 + | +35 | Runner = Callable[..., subprocess.CompletedProcess[str]] +36 | Which = Callable[[str], Optional[str]] + | ^^^^^^^^^^^^^ +37 | +38 | _FANOUT = None + | +help: Convert to `X | None` + | +35 | Runner = Callable[..., subprocess.CompletedProcess[str]] + - Which = Callable[[str], Optional[str]] +36 + Which = Callable[[str], str | None] +37 | + | + +E501 Line too long (118 > 100) + --> cmax/scripts/1-audit/checks/gpu/vboost.py:76:101 + | +76 | def vboost_command(which: Which = shutil.which, euid: Callable[[], int] = os.geteuid) -> tuple[list[str] | None, str]: + | ^^^^^^^^^^^^^^^^^^ +77 | nvidia_smi = which("nvidia-smi") +78 | if not nvidia_smi: + | + +E501 Line too long (104 > 100) + --> cmax/scripts/1-audit/checks/gpu/vboost.py:237:101 + | +235 | try: +236 | proc = kubectl( +237 | ["get", "pods", "-n", namespace, "--field-selector", f"spec.nodeName={node}", "-o", "json"], + | ^^^^ +238 | runner=runner, +239 | timeout=45, + | + +E501 Line too long (110 > 100) + --> cmax/scripts/1-audit/checks/gpu/vboost.py:264:101 + | +264 | def k8s_vboost_node(namespace: str, node: str, *, runner: Runner) -> tuple[dict[str, Any] | None, str | None]: + | ^^^^^^^^^^ +265 | """Try to set vboost from inside the node's nvidia-driver daemonset pod. + | + +E501 Line too long (103 > 100) + --> cmax/scripts/1-audit/checks/gpu/vboost.py:330:101 + | +328 | namespace = k8s_gpu_namespace(env=env, runner=runner) +329 | if not namespace: +330 | return [], ["no NVIDIA GPU Operator namespace found; cannot reach a driver pod to test vboost"] + | ^^^ +331 | +332 | try: + | + +E501 Line too long (102 > 100) + --> cmax/scripts/1-audit/checks/gpu/vboost.py:432:101 + | +430 | return 0 +431 | +432 | harness = os.environ.get("CLUSTERMAX_AUDIT_HARNESS") or os.environ.get("CLUSTERMAX_HARNESS") or "" + | ^^ +433 | print(json.dumps(build_check_payload(harness=harness), sort_keys=True)) +434 | return 0 + | + +E501 Line too long (112 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:115:101 + | +113 | # fails, which is the point. +114 | TOPO_REMEDIATION = ( +115 | f"Provider fix: add an enroot mount hook or a pyxis default mount so {TOPO_FILE_GLOB_DIR}/{TOPO_FILE_GLOB} " + | ^^^^^^^^^^^^ +116 | "is mounted into every container with no per-job flag. Per-job workaround, which this check does not " +117 | "score as a pass: --mount type=bind,source=/etc/nccl/topo.xml,target=/etc/nccl/topo.xml" + | + +E501 Line too long (106 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:116:101 + | +114 | TOPO_REMEDIATION = ( +115 | f"Provider fix: add an enroot mount hook or a pyxis default mount so {TOPO_FILE_GLOB_DIR}/{TOPO_FILE_GLOB} " +116 | "is mounted into every container with no per-job flag. Per-job workaround, which this check does not " + | ^^^^^^ +117 | "score as a pass: --mount type=bind,source=/etc/nccl/topo.xml,target=/etc/nccl/topo.xml" +118 | ) + | + +UP022 Prefer `capture_output` over sending `stdout` and `stderr` to `PIPE` + --> cmax/scripts/1-audit/checks/platform_config.py:252:12 + | +251 | def run_command(command: list[str], *, timeout: int = 30, input_text: str | None = None): +252 | return subprocess.run( + | ____________^ +253 | | command, +254 | | input=input_text, +255 | | stdout=subprocess.PIPE, +256 | | stderr=subprocess.PIPE, +257 | | text=True, +258 | | timeout=timeout, +259 | | ) + | |_____^ +help: Replace with `capture_output` keyword argument + +E501 Line too long (115 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:415:101 + | +414 | def iommu_disabled_requested(params: dict[str, str]) -> bool: +415 | return any(params.get(key, "").lower() in {"off", "disabled"} for key in ("iommu", "intel_iommu", "amd_iommu")) + | ^^^^^^^^^^^^^^^ + +E501 Line too long (103 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:467:101 + | +467 | def collect_pci_devices(root: Path, *, include_rdma: bool = True) -> tuple[list[dict[str, Any]], bool]: + | ^^^ +468 | """Selected GPU and RDMA NIC PCI functions with their IOMMU group. + | + +E501 Line too long (113 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:572:101 + | +570 | return summary( +571 | "unknown", +572 | "the PCI bus is not readable from where this check ran; GPU and RDMA device presence is unknown", + | ^^^^^^^^^^^^^ +573 | evidence=evidence, +574 | ) + | + +E501 Line too long (101 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:603:101 + | +601 | ), +602 | warnings=[ +603 | f"IOMMU mode is translated with kernel parameters {iommu.get('cmdline_params') or '{}'}", + | ^ +604 | ], +605 | evidence=evidence, + | + +SIM110 Use `return any(CMDQV_COMPATIBLE in read_text(candidate) for candidate in candidates)` instead of `for` loop + --> cmax/scripts/1-audit/checks/platform_config.py:620:5 + | +618 | except OSError: +619 | return False +620 | / for candidate in candidates: +621 | | if CMDQV_COMPATIBLE in read_text(candidate): +622 | | return True +623 | | return False + | |________________^ +help: Replace with `return any(CMDQV_COMPATIBLE in read_text(candidate) for candidate in candidates)` + +E501 Line too long (111 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:663:101 + | +661 | entry.name +662 | for entry in under_root(root, path).iterdir() +663 | if entry.is_symlink() and not entry.name.startswith(("bind", "uevent", "unbind", "module")) + | ^^^^^^^^^^^ +664 | ) +665 | ) + | + +E501 Line too long (117 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:673:101 + | +671 | # Anything that names the CMDQV driver or node is itself proof of Grace, so +672 | # the platform markers only have to answer for a host that exposes neither. +673 | grace = bool(driver_dirs or bound_devices or devicetree) or devicetree_names_grace(root) or dmi_names_grace(root) + | ^^^^^^^^^^^^^^^^^ +674 | +675 | return { + | + +E501 Line too long (109 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:700:101 + | +698 | return summary( +699 | "not_applicable", +700 | f"{machine} host; Arm SMMU command queue virtualization applies to Arm and Grace platforms only", + | ^^^^^^^^^ +701 | evidence=evidence, +702 | ) + | + +E501 Line too long (101 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:730:101 + | +728 | "unknown", +729 | ( +730 | "Arm virtual machine guest uses SMMUv3, and this vantage could not identify the " + | ^ +731 | "platform as Grace; CMDQV / VCMDQ exists only on Grace" +732 | ), + | + +E501 Line too long (117 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:741:101 + | +739 | "guest invalidations serialize through one host command queue" +740 | ), +741 | warnings=[f"SMMUv3 unit(s) {', '.join(smmu.get('smmuv3_units') or [])} with no tegra241-cmdqv evidence"], + | ^^^^^^^^^^^^^^^^^ +742 | evidence=evidence, +743 | ) + | + +E501 Line too long (124 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:772:101 + | +770 | paths: list[str] = [] +771 | mentions_topology: list[str] = [] +772 | for relative in ("/etc/enroot/enroot.conf", "/etc/enroot/enroot.conf.d", "/etc/enroot/hooks.d", "/etc/enroot/mounts.d"): + | ^^^^^^^^^^^^^^^^^^^^^^^^ +773 | entry = under_root(root, relative) +774 | # pathlib's recursive glob swallows PermissionError, but not the rest of + | + +E501 Line too long (106 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:831:101 + | +829 | "sysfs_read": sysfs_read, +830 | "link_layers": unique_layers, +831 | "fabric": unique_layers[0] if len(unique_layers) == 1 else ("mixed" if unique_layers else "none"), + | ^^^^^^ +832 | } + | + +E501 Line too long (104 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:850:101 + | +848 | # would let an unreadable entry stand for an absent file, which is the +849 | # asserted absence this check must never make. +850 | return [f"{TOPO_FILE_GLOB_DIR}/{entry.name}" for entry in entries if file_state(entry) is not False] + | ^^^^ + +E501 Line too long (102 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:919:101 + | +917 | "topo_file_declared": declared, +918 | "topo_files_readable": readable, +919 | "topo_file_evidence": {path: describe_topo_file(under_root(root, path)) for path in readable}, + | ^^ +920 | "topo_candidates_unreachable": unreachable, +921 | "declared_topo_file_readable": bool(declared) and declared in readable, + | + +E501 Line too long (104 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:990:101 + | +988 | conf_block = ( +989 | f"if [ -r '{conf[0]}' ]; then\n" +990 | " conf_topo=$(sed -n 's/#.*//; s/^[[:space:]]*NCCL_TOPO_FILE[[:space:]]*=[[:space:]]*//p' " + | ^^^^ +991 | f"'{conf[0]}'" +992 | " | sed -e 's/[[:space:]]*$//' -e 's/^\"\\(.*\\)\"$/\\1/' -e \"s/^'\\(.*\\)'\\$/\\1/\" | tail -n 1)\n" + | + +E501 Line too long (114 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:992:101 + | +990 | " conf_topo=$(sed -n 's/#.*//; s/^[[:space:]]*NCCL_TOPO_FILE[[:space:]]*=[[:space:]]*//p' " +991 | f"'{conf[0]}'" +992 | " | sed -e 's/[[:space:]]*$//' -e 's/^\"\\(.*\\)\"$/\\1/' -e \"s/^'\\(.*\\)'\\$/\\1/\" | tail -n 1)\n" + | ^^^^^^^^^^^^^^ +993 | ' if [ -n "$conf_topo" ]; then\n' +994 | ' printf "topo_file_conf=%s\\n" "$conf_topo"\n' + | + +E501 Line too long (123 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:1103:101 + | +1103 | def run_container_check(*, harness: str, env: dict[str, str], candidates: list[str], runner=run_command) -> dict[str, Any]: + | ^^^^^^^^^^^^^^^^^^^^^^^ +1104 | """Read NCCL_TOPO_FILE from inside a container started by the same launcher. + | + +E501 Line too long (104 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:1175:101 + | +1173 | "available": False, +1174 | "reason_code": "launch_failed", +1175 | "reason": f"pyxis container check did not run ({exc}); the mount is unverified, not absent", + | ^^^^ +1176 | **attestation, +1177 | } + | + +E501 Line too long (116 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:1184:101 + | +1182 | if stderr_says_no_pyxis(stderr): +1183 | reason_code = "no_pyxis" +1184 | reason = "pyxis/enroot is not installed on this cluster; the container arm of this check does not apply" + | ^^^^^^^^^^^^^^^^ +1185 | else: +1186 | reason_code = "check_incomplete" + | + +E501 Line too long (101 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:1195:101 + | +1193 | **attestation, +1194 | } +1195 | return {"available": True, "stdout": (proc.stdout or "").strip()[:8000], **attestation, **parsed} + | ^ + +E501 Line too long (109 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:1219:101 + | +1217 | ) -> dict[str, Any]: +1218 | hosts = [report.get("nccl", {}) for report in reports] +1219 | declared = next((host.get("topo_file_declared") for host in hosts if host.get("topo_file_declared")), "") + | ^^^^^^^^^ +1220 | host_env = next((host.get("topo_file_env") for host in hosts if host.get("topo_file_env")), "") +1221 | host_files = sorted({path for host in hosts for path in (host.get("topo_files_readable") or [])}) + | + +E501 Line too long (101 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:1221:101 + | +1219 | declared = next((host.get("topo_file_declared") for host in hosts if host.get("topo_file_declared")), "") +1220 | host_env = next((host.get("topo_file_env") for host in hosts if host.get("topo_file_env")), "") +1221 | host_files = sorted({path for host in hosts for path in (host.get("topo_files_readable") or [])}) + | ^ +1222 | enroot_hooks = sorted({path for host in hosts for path in (host.get("enroot", {}).get("files_mentioning_topology") or [])}) +1223 | host_evidence: dict[str, Any] = {} + | + +E501 Line too long (127 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:1222:101 + | +1220 | host_env = next((host.get("topo_file_env") for host in hosts if host.get("topo_file_env")), "") +1221 | host_files = sorted({path for host in hosts for path in (host.get("topo_files_readable") or [])}) +1222 | enroot_hooks = sorted({path for host in hosts for path in (host.get("enroot", {}).get("files_mentioning_topology") or [])}) + | ^^^^^^^^^^^^^^^^^^^^^^^^^^^ +1223 | host_evidence: dict[str, Any] = {} +1224 | for host in hosts: + | + +E501 Line too long (114 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:1254:101 + | +1252 | # pointing at a path that holds nothing leaves nothing to mount, so the +1253 | # check has no subject and must not hard-fail on a missing mount. +1254 | unreachable = sorted({path for host in hosts for path in (host.get("topo_candidates_unreachable") or [])}) + | ^^^^^^^^^^^^^^ +1255 | if unreachable: +1256 | # A path the vantage cannot reach is not a path with no file on it. + | + +E501 Line too long (104 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:1324:101 + | +1322 | if host_env and not container_env: +1323 | failures.append( +1324 | f"the host sets NCCL_TOPO_FILE={host_env} but the container environment does not carry it; " + | ^^^^ +1325 | "the launcher does not propagate the job environment into the container" +1326 | ) + | + +E501 Line too long (118 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:1331:101 + | +1329 | if container_declared: +1330 | failures.append( +1331 | f"container declares NCCL topology file {container_declared} in {TOPO_SOURCE_LABELS[source]} but the " + | ^^^^^^^^^^^^^^^^^^ +1332 | "file is not readable inside the container; NCCL falls back to automatic detection without a warning" +1333 | ) + | + +E501 Line too long (117 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:1332:101 + | +1330 | failures.append( +1331 | f"container declares NCCL topology file {container_declared} in {TOPO_SOURCE_LABELS[source]} but the " +1332 | "file is not readable inside the container; NCCL falls back to automatic detection without a warning" + | ^^^^^^^^^^^^^^^^^ +1333 | ) +1334 | else: + | + +E501 Line too long (105 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:1336:101 + | +1334 | else: +1335 | failures.append( +1336 | f"host has NCCL topology file {target} but the container sets NCCL_TOPO_FILE in neither " + | ^^^^^ +1337 | f"the environment nor {NCCL_CONF_PATH}, and has no readable {NCCL_DEFAULT_TOPO_FILE}; " +1338 | "the file is not mounted automatically and NCCL falls back to automatic detection without a warning" + | + +E501 Line too long (103 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:1337:101 + | +1335 | failures.append( +1336 | f"host has NCCL topology file {target} but the container sets NCCL_TOPO_FILE in neither " +1337 | f"the environment nor {NCCL_CONF_PATH}, and has no readable {NCCL_DEFAULT_TOPO_FILE}; " + | ^^^ +1338 | "the file is not mounted automatically and NCCL falls back to automatic detection without a warning" +1339 | ) + | + +E501 Line too long (116 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:1338:101 + | +1336 | f"host has NCCL topology file {target} but the container sets NCCL_TOPO_FILE in neither " +1337 | f"the environment nor {NCCL_CONF_PATH}, and has no readable {NCCL_DEFAULT_TOPO_FILE}; " +1338 | "the file is not mounted automatically and NCCL falls back to automatic detection without a warning" + | ^^^^^^^^^^^^^^^^ +1339 | ) +1340 | else: + | + +E501 Line too long (108 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:1349:101 + | +1347 | failures.append(f"container resolves {container_declared} but the file is empty") +1348 | if seen.get("xml") == "bad": +1349 | failures.append(f"container resolves {container_declared} but its contents do not parse as XML") + | ^^^^^^^^ +1350 | # A topology file describes the node it was generated for, so two nodes +1351 | # legitimately publish different content at the same path. The container + | + +E501 Line too long (105 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:1362:101 + | +1360 | host_missing_digest = False +1361 | for host in hosts: +1362 | digest = ((host.get("topo_file_evidence") or {}).get(container_declared) or {}).get("sha256") + | ^^^^^ +1363 | if digest: +1364 | host_digests.add(digest) + | + +E501 Line too long (114 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:1369:101 + | +1367 | detail["host_topo_file_digests"] = sorted(host_digests) +1368 | container_digest = seen.get("sha256") +1369 | if container_digest and host_digests and not host_missing_digest and container_digest not in host_digests: + | ^^^^^^^^^^^^^^ +1370 | failures.append( +1371 | f"container resolves {container_declared} but its contents match no checked host's file at that " + | + +E501 Line too long (113 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:1371:101 + | +1369 | if container_digest and host_digests and not host_missing_digest and container_digest not in host_digests: +1370 | failures.append( +1371 | f"container resolves {container_declared} but its contents match no checked host's file at that " + | ^^^^^^^^^^^^^ +1372 | "path; the container sees a stale or shadowed topology file" +1373 | ) + | + +E501 Line too long (112 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:1382:101 + | +1380 | detail.update( +1381 | status="pass", +1382 | message=f"container resolves NCCL topology file {container_declared} from {TOPO_SOURCE_LABELS[source]}", + | ^^^^^^^^^^^^ +1383 | ) +1384 | return detail + | + +E501 Line too long (118 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:1387:101 + | +1387 | def classify_fabric_shape(*, fabric_tiers: int, node_count: int, clos_node_threshold: int) -> tuple[bool | None, str]: + | ^^^^^^^^^^^^^^^^^^ +1388 | """Whether the fabric has a spine tier, and the evidence that decided it. + | + +E501 Line too long (111 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:1415:101 + | +1413 | ) -> dict[str, Any]: +1414 | hosts = [report.get("nccl", {}) for report in reports] +1415 | qps_values = sorted({as_int(host.get("qps_per_connection"), DEFAULT_QPS_PER_CONNECTION) for host in hosts}) + | ^^^^^^^^^^^ +1416 | qps = qps_values[0] if qps_values else DEFAULT_QPS_PER_CONNECTION +1417 | source = next((host.get("qps_source") for host in hosts if host.get("qps_source")), "default") + | + +E501 Line too long (111 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:1419:101 + | +1417 | source = next((host.get("qps_source") for host in hosts if host.get("qps_source")), "default") +1418 | fabrics = sorted({str(host.get("rdma", {}).get("fabric") or "none") for host in hosts}) +1419 | rdma_devices = sorted({device for host in hosts for device in (host.get("rdma", {}).get("devices") or [])}) + | ^^^^^^^^^^^ +1420 | # "No host has an RDMA device" is a claim about every host, so one host that +1421 | # could not be read withdraws it. A sibling that read a clean empty class + | + +E501 Line too long (112 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:1464:101 + | +1462 | status="unknown", +1463 | message=( +1464 | f"/sys/class/infiniband is not readable on {unread_hosts} of {len(hosts)} checked host(s); " + | ^^^^^^^^^^^^ +1465 | "RDMA device presence is unknown" +1466 | ), + | + +E501 Line too long (102 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:1493:101 + | +1491 | status="unknown", +1492 | message=( +1493 | f"NCCL_IB_QPS_PER_CONNECTION={qps} on a {node_count}-node {node_count_scope} with no " + | ^^ +1494 | "topology data; the shape of the whole fabric is not known, so the advisory cannot be judged" +1495 | ), + | + +E501 Line too long (109 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:1494:101 + | +1492 | message=( +1493 | f"NCCL_IB_QPS_PER_CONNECTION={qps} on a {node_count}-node {node_count_scope} with no " +1494 | "topology data; the shape of the whole fabric is not known, so the advisory cannot be judged" + | ^^^^^^^^^ +1495 | ), +1496 | ) + | + +E501 Line too long (109 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:1514:101 + | +1512 | detail.update( +1513 | status="pass", +1514 | message=f"NCCL_IB_QPS_PER_CONNECTION={qps} from {source} on a multi-tier Clos fabric by {basis}", + | ^^^^^^^^^ +1515 | ) +1516 | return detail + | + +E501 Line too long (110 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:1520:101 + | +1518 | warning = ( +1519 | f"NCCL_IB_QPS_PER_CONNECTION={qps} (the default) on a multi-tier Clos fabric by {basis}; " +1520 | f"one queue pair gives ECMP a single hash input, so parallel flows collide on the same spine uplink. " + | ^^^^^^^^^^ +1521 | f"Set it to {RECOMMENDED_QPS_PER_CONNECTION} (up to 4) identically on every rank, for example in " +1522 | f"{NCCL_CONF_PATH}. Sweep the value at the message sizes of the real workload before you fix it, " + | + +E501 Line too long (106 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:1521:101 + | +1519 | f"NCCL_IB_QPS_PER_CONNECTION={qps} (the default) on a multi-tier Clos fabric by {basis}; " +1520 | f"one queue pair gives ECMP a single hash input, so parallel flows collide on the same spine uplink. " +1521 | f"Set it to {RECOMMENDED_QPS_PER_CONNECTION} (up to 4) identically on every rank, for example in " + | ^^^^^^ +1522 | f"{NCCL_CONF_PATH}. Sweep the value at the message sizes of the real workload before you fix it, " +1523 | "because the best value depends on the fabric and adds CPU cost" + | + +E501 Line too long (106 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:1522:101 + | +1520 | f"one queue pair gives ECMP a single hash input, so parallel flows collide on the same spine uplink. " +1521 | f"Set it to {RECOMMENDED_QPS_PER_CONNECTION} (up to 4) identically on every rank, for example in " +1522 | f"{NCCL_CONF_PATH}. Sweep the value at the message sizes of the real workload before you fix it, " + | ^^^^^^ +1523 | "because the best value depends on the fabric and adds CPU cost" +1524 | ) + | + +E501 Line too long (111 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:1603:101 + | +1601 | visible under ``hosts``. +1602 | """ +1603 | statuses = [str(report.get("summaries", {}).get(key, {}).get("status") or "unknown") for report in reports] + | ^^^^^^^^^^^ +1604 | if "fail" in statuses: +1605 | status = "fail" + | + +E501 Line too long (111 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:1674:101 + | +1672 | payload[key].update( +1673 | status="unknown", +1674 | message="only a stand-in host was checked; the platform of the compute nodes was not read", + | ^^^^^^^^^^^ +1675 | ) +1676 | payload["nccl_topo_file"] = summarize_nccl_topo_file(reports=reports, container=container, errors=errors) + | + +E501 Line too long (109 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:1676:101 + | +1674 | message="only a stand-in host was checked; the platform of the compute nodes was not read", +1675 | ) +1676 | payload["nccl_topo_file"] = summarize_nccl_topo_file(reports=reports, container=container, errors=errors) + | ^^^^^^^^^ +1677 | payload["nccl_ib_qps"] = summarize_nccl_ib_qps( +1678 | reports=reports, + | + +E501 Line too long (101 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:1868:101 + | +1866 | errors: list[str] = [] +1867 | if proc.returncode != 0: +1868 | errors.append(f"srun host check exited {proc.returncode}; parsing any completed host output") + | ^ +1869 | for value in load_fanout().parse_json_lines(proc.stdout, require_host=True): +1870 | reports.append(value) + | + +E501 Line too long (117 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:1908:101 + | +1906 | "name": "check", +1907 | "image": image, +1908 | "imagePullPolicy": os.environ.get("CLUSTERMAX_AUDIT_K8S_HOST_CHECK_PULL_POLICY", "IfNotPresent"), + | ^^^^^^^^^^^^^^^^^ +1909 | "command": ["sh", "-c", "sleep 600"], +1910 | "env": [{"name": "NODE_NAME", "value": node["name"]}], + | + +E501 Line too long (104 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:1947:101 + | +1945 | apply_proc = kubectl(["apply", "-f", "-"], timeout=45, input_text=manifest) +1946 | if apply_proc.returncode != 0: +1947 | return None, f"{node['name']}: failed to create host check pod: {apply_proc.stderr.strip()}" + | ^^^^ +1948 | +1949 | wait_proc = kubectl( + | + +E501 Line too long (107 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:1954:101 + | +1952 | ) +1953 | if wait_proc.returncode != 0: +1954 | return None, f"{node['name']}: host check pod did not become Ready: {wait_proc.stderr.strip()}" + | ^^^^^^^ +1955 | +1956 | check_args = [ + | + +SIM105 Use `contextlib.suppress(OSError, subprocess.SubprocessError)` instead of `try`-`except`-`pass` + --> cmax/scripts/1-audit/checks/platform_config.py:1993:9 + | +1991 | return None, f"{node['name']}: host check failed: {exc}" +1992 | finally: +1993 | / try: +1994 | | kubectl(["delete", "pod", pod_name, "-n", namespace, "--ignore-not-found=true", "--wait=false"], timeout=20) +1995 | | except (OSError, subprocess.SubprocessError): +1996 | | pass + | |________________^ +help: Replace `try`-`except`-`pass` with `with contextlib.suppress(OSError, subprocess.SubprocessError): ...` + +E501 Line too long (120 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:1994:101 + | +1992 | finally: +1993 | try: +1994 | kubectl(["delete", "pod", pod_name, "-n", namespace, "--ignore-not-found=true", "--wait=false"], timeout=20) + | ^^^^^^^^^^^^^^^^^^^^ +1995 | except (OSError, subprocess.SubprocessError): +1996 | pass + | + +E501 Line too long (103 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:2096:101 + | +2094 | report["check_scope"] = "local" +2095 | reports = [report] +2096 | errors = [*discovery_errors, "no GPU node was checked; the local host was checked instead"] + | ^^^ +2097 | else: +2098 | reports = [ + | + +E501 Line too long (109 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:2143:101 + | +2141 | "available": False, +2142 | "reason_code": "no_host_topo_file", +2143 | "reason": "no readable NCCL topology file on the checked hosts; the container check was skipped", + | ^^^^^^^^^ +2144 | } +2145 | threshold = as_int(os.environ.get("CLUSTERMAX_AUDIT_CLOS_NODE_THRESHOLD"), default=DEFAULT_CLOS_NODE_THRESHOLD) + | + +E501 Line too long (115 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:2145:101 + | +2143 | "reason": "no readable NCCL topology file on the checked hosts; the container check was skipped", +2144 | } +2145 | threshold = as_int(os.environ.get("CLUSTERMAX_AUDIT_CLOS_NODE_THRESHOLD"), default=DEFAULT_CLOS_NODE_THRESHOLD) + | ^^^^^^^^^^^^^^^ +2146 | payload = build_payload( +2147 | reports=reports, + | + +E501 Line too long (123 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:2204:101 + | +2202 | def main(argv: list[str]) -> int: +2203 | parser = argparse.ArgumentParser(description="Shared platform configuration collection") +2204 | parser.add_argument("--collect-host", action="store_true", help="emit one host report instead of aggregate check JSON") + | ^^^^^^^^^^^^^^^^^^^^^^^ +2205 | parser.add_argument("--root", default="/", help="host root path for proc/sys/etc reads") +2206 | parser.add_argument("--harness", default=os.environ.get("CLUSTERMAX_AUDIT_HARNESS", "standalone")) + | + +E501 Line too long (102 > 100) + --> cmax/scripts/1-audit/checks/platform_config.py:2206:101 + | +2204 | parser.add_argument("--collect-host", action="store_true", help="emit one host report instead of aggregate check JSON") +2205 | parser.add_argument("--root", default="/", help="host root path for proc/sys/etc reads") +2206 | parser.add_argument("--harness", default=os.environ.get("CLUSTERMAX_AUDIT_HARNESS", "standalone")) + | ^^ +2207 | parser.add_argument( +2208 | "--skip-scale-out", + | + +I001 [*] Import block is un-sorted or un-formatted + --> cmax/scripts/1-audit/checks/system/arm-smmu-virtualization-check.py:12:1 + | +11 | sys.path.insert(0, str(Path(__file__).resolve().parents[1])) +12 | from platform_config import run_named_check + | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +help: Organize imports + | +13 | + - +14 | if __name__ == "__main__": + | + +I001 [*] Import block is un-sorted or un-formatted + --> cmax/scripts/1-audit/checks/system/hbm_memory_exposure.py:21:1 + | +19 | """ +20 | +21 | / from __future__ import annotations +22 | | +23 | | import argparse +24 | | import json +25 | | import os +26 | | import re +27 | | import socket +28 | | import subprocess +29 | | import sys +30 | | import uuid +31 | | from pathlib import Path +32 | | from typing import Any + | |______________________^ +help: Organize imports + | +33 | + - +34 | GIB_KB = 1024 * 1024 + | + +E501 Line too long (125 > 100) + --> cmax/scripts/1-audit/checks/system/hbm_memory_exposure.py:145:101 + | +145 | def run_command(command: list[str], *, timeout: int = 30, input_text: str | None = None) -> subprocess.CompletedProcess[str]: + | ^^^^^^^^^^^^^^^^^^^^^^^^^ +146 | return subprocess.run( +147 | command, + | + +UP022 Prefer `capture_output` over sending `stdout` and `stderr` to `PIPE` + --> cmax/scripts/1-audit/checks/system/hbm_memory_exposure.py:146:12 + | +145 | def run_command(command: list[str], *, timeout: int = 30, input_text: str | None = None) -> subprocess.CompletedProcess[str]: +146 | return subprocess.run( + | ____________^ +147 | | command, +148 | | input=input_text, +149 | | stdout=subprocess.PIPE, +150 | | stderr=subprocess.PIPE, +151 | | text=True, +152 | | timeout=timeout, +153 | | ) + | |_____^ +help: Replace with `capture_output` keyword argument + +E501 Line too long (101 > 100) + --> cmax/scripts/1-audit/checks/system/hbm_memory_exposure.py:331:101 + | +329 | total_gb = round(sum(as_int(node.get("mem_total_kb")) for node in hbm_nodes) / GIB_KB, 1) +330 | failures.append( +331 | f"{len(hbm_nodes)} memory-only NUMA node(s) expose {total_gb} GB that looks like GPU HBM" + | ^ +332 | ) + | + +E501 Line too long (101 > 100) + --> cmax/scripts/1-audit/checks/system/hbm_memory_exposure.py:335:101 + | +334 | if coherent_candidate and cdmm_mode != "driver": +335 | msg = f"coherent GPU platform is not in CDMM driver mode (CoherentGPUMemoryMode={cdmm_mode})" + | ^ +336 | if harness == "k8s": +337 | failures.append(msg) + | + +E501 Line too long (101 > 100) + --> cmax/scripts/1-audit/checks/system/hbm_memory_exposure.py:342:101 + | +341 | if report.get("memory_only_numa_nodes") and not hbm_nodes: +342 | warnings.append("host has memory-only NUMA node(s), but they were not classified as GPU HBM") + | ^ +343 | +344 | status = "pass" + | + +E501 Line too long (110 > 100) + --> cmax/scripts/1-audit/checks/system/hbm_memory_exposure.py:410:101 + | +408 | gpu_count_hint: int = 0, +409 | ) -> dict[str, Any]: +410 | hostname = os.environ.get("NODE_NAME") or read_text(under_root(root, "/proc/sys/kernel/hostname")).strip() + | ^^^^^^^^^^ +411 | if not hostname: +412 | hostname = socket.gethostname() + | + +E501 Line too long (103 > 100) + --> cmax/scripts/1-audit/checks/system/hbm_memory_exposure.py:416:101 + | +414 | smi_gpus = query_nvidia_smi() if root == Path("/") else [] +415 | proc_gpus = parse_gpu_information(root) +416 | gpu_models = [str(gpu.get("model") or gpu.get("model_name") or "") for gpu in smi_gpus + proc_gpus] + | ^^^ +417 | if gpu_model_hint: +418 | gpu_models.append(gpu_model_hint) + | + +E501 Line too long (105 > 100) + --> cmax/scripts/1-audit/checks/system/hbm_memory_exposure.py:421:101 + | +419 | gpu_models = sorted({model for model in gpu_models if model}) +420 | +421 | gpu_memories = [as_int(gpu.get("memory_mb")) for gpu in smi_gpus if as_int(gpu.get("memory_mb")) > 0] + | ^^^^^ +422 | if gpu_memory_mb_hint > 0: +423 | gpu_memories.append(gpu_memory_mb_hint) + | + +E501 Line too long (106 > 100) + --> cmax/scripts/1-audit/checks/system/hbm_memory_exposure.py:436:101 + | +435 | meminfo_total_kb = parse_memtotal_kb(read_text(under_root(root, "/proc/meminfo"))) +436 | cpu_numa_mem_kb = sum(as_int(node.get("mem_total_kb")) for node in numa_nodes if node.get("has_cpus")) + | ^^^^^^ +437 | memory_only_mem_kb = sum(as_int(node.get("mem_total_kb")) for node in memory_only_nodes) +438 | hbm_like_mem_kb = sum(as_int(node.get("mem_total_kb")) for node in hbm_like_nodes) + | + +E501 Line too long (107 > 100) + --> cmax/scripts/1-audit/checks/system/hbm_memory_exposure.py:464:101 + | +462 | "meminfo_matches_node_total": ( +463 | node_mem_total_kb > 0 +464 | and abs(meminfo_total_kb - node_mem_total_kb) <= max(int(node_mem_total_kb * 0.02), GIB_KB) + | ^^^^^^^ +465 | ), +466 | "meminfo_includes_hbm_like_memory": ( + | + +E501 Line too long (104 > 100) + --> cmax/scripts/1-audit/checks/system/hbm_memory_exposure.py:467:101 + | +465 | ), +466 | "meminfo_includes_hbm_like_memory": ( +467 | hbm_like_mem_kb > 0 and meminfo_total_kb >= cpu_numa_mem_kb + int(hbm_like_mem_kb * 0.8) + | ^^^^ +468 | ), +469 | }, + | + +SIM114 [*] Combine `if` branches using logical `or` operator + --> cmax/scripts/1-audit/checks/system/hbm_memory_exposure.py:496:5 + | +494 | if any(status == "fail" for status in statuses): +495 | status = "fail" +496 | / elif any(status == "warning" for status in statuses): +497 | | status = "warning" +498 | | elif errors: +499 | | status = "warning" + | |__________________________^ +500 | elif any(status == "pass" for status in statuses): +501 | status = "pass" + | +help: Combine `if` branches + | +495 | status = "fail" + - elif any(status == "warning" for status in statuses): + - status = "warning" + - elif errors: +496 + elif any(status == "warning" for status in statuses) or errors: +497 | status = "warning" + | + +E501 Line too long (106 > 100) + --> cmax/scripts/1-audit/checks/system/hbm_memory_exposure.py:534:101 + | +532 | "warnings": warnings, +533 | "hosts_checked": len(reports), +534 | "hosts": {str(report.get("host") or f"host-{idx}"): report for idx, report in enumerate(reports)}, + | ^^^^^^ +535 | } + | + +E501 Line too long (101 > 100) + --> cmax/scripts/1-audit/checks/system/hbm_memory_exposure.py:594:101 + | +592 | errors: list[str] = [] +593 | if proc.returncode != 0: +594 | errors.append(f"srun host check exited {proc.returncode}; parsing any completed host output") + | ^ +595 | +596 | for line in proc.stdout.splitlines(): + | + +E501 Line too long (121 > 100) + --> cmax/scripts/1-audit/checks/system/hbm_memory_exposure.py:624:101 + | +624 | def kubectl(command: list[str], *, timeout: int = 60, input_text: str | None = None) -> subprocess.CompletedProcess[str]: + | ^^^^^^^^^^^^^^^^^^^^^ +625 | return run_command(["kubectl", *command], timeout=timeout, input_text=input_text) + | + +E501 Line too long (117 > 100) + --> cmax/scripts/1-audit/checks/system/hbm_memory_exposure.py:655:101 + | +653 | "name": "check", +654 | "image": image, +655 | "imagePullPolicy": os.environ.get("CLUSTERMAX_AUDIT_K8S_HOST_CHECK_PULL_POLICY", "IfNotPresent"), + | ^^^^^^^^^^^^^^^^^ +656 | "command": ["sh", "-c", "sleep 600"], +657 | "env": _k8s_check_env(node), + | + +E501 Line too long (105 > 100) + --> cmax/scripts/1-audit/checks/system/hbm_memory_exposure.py:662:101 + | +660 | {"name": "host-proc", "mountPath": "/host/proc", "readOnly": True}, +661 | {"name": "host-sys", "mountPath": "/host/sys", "readOnly": True}, +662 | {"name": "host-kubelet", "mountPath": "/host/var/lib/kubelet", "readOnly": True}, + | ^^^^^ +663 | ], +664 | } + | + +E501 Line too long (117 > 100) + --> cmax/scripts/1-audit/checks/system/hbm_memory_exposure.py:675:101 + | +675 | def run_k8s_host_check(namespace: str, node: dict[str, Any], image: str) -> tuple[dict[str, Any] | None, str | None]: + | ^^^^^^^^^^^^^^^^^ +676 | suffix = uuid.uuid4().hex[:8] +677 | pod_name = f"clustermax-hbm-{suffix}" + | + +E501 Line too long (104 > 100) + --> cmax/scripts/1-audit/checks/system/hbm_memory_exposure.py:682:101 + | +680 | apply_proc = kubectl(["apply", "-f", "-"], timeout=45, input_text=manifest) +681 | if apply_proc.returncode != 0: +682 | return None, f"{node['name']}: failed to create host check pod: {apply_proc.stderr.strip()}" + | ^^^^ +683 | +684 | wait_proc = kubectl( + | + +E501 Line too long (107 > 100) + --> cmax/scripts/1-audit/checks/system/hbm_memory_exposure.py:689:101 + | +687 | ) +688 | if wait_proc.returncode != 0: +689 | return None, f"{node['name']}: host check pod did not become Ready: {wait_proc.stderr.strip()}" + | ^^^^^^^ +690 | +691 | exec_proc = kubectl( + | + +E501 Line too long (133 > 100) + --> cmax/scripts/1-audit/checks/system/hbm_memory_exposure.py:692:101 + | +691 | exec_proc = kubectl( +692 | ["exec", "-i", "-n", namespace, pod_name, "--", "python3", "-", "--collect-host", "--root", "/host", "--harness", "k8s"], + | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +693 | timeout=60, +694 | input_text=Path(__file__).read_text(), + | + +SIM105 Use `contextlib.suppress(Exception)` instead of `try`-`except`-`pass` + --> cmax/scripts/1-audit/checks/system/hbm_memory_exposure.py:709:9 + | +707 | return None, f"{node['name']}: host check failed: {exc}" +708 | finally: +709 | / try: +710 | | kubectl(["delete", "pod", pod_name, "-n", namespace, "--ignore-not-found=true", "--wait=false"], timeout=20) +711 | | except Exception: +712 | | pass + | |________________^ +help: Replace `try`-`except`-`pass` with `with contextlib.suppress(Exception): ...` + +E501 Line too long (120 > 100) + --> cmax/scripts/1-audit/checks/system/hbm_memory_exposure.py:710:101 + | +708 | finally: +709 | try: +710 | kubectl(["delete", "pod", pod_name, "-n", namespace, "--ignore-not-found=true", "--wait=false"], timeout=20) + | ^^^^^^^^^^^^^^^^^^^^ +711 | except Exception: +712 | pass + | + +E501 Line too long (103 > 100) + --> cmax/scripts/1-audit/checks/system/hbm_memory_exposure.py:740:101 + | +738 | image = os.environ.get("CLUSTERMAX_AUDIT_K8S_HOST_CHECK_IMAGE", "python:3.12-alpine") +739 | max_nodes = as_int(os.environ.get("CLUSTERMAX_AUDIT_K8S_MAX_HOST_CHECKS"), default=len(nodes)) +740 | return load_fanout().fan_out_k8s(_hbm_per_node(namespace, image), nodes=nodes, max_nodes=max_nodes) + | ^^^ + +E501 Line too long (123 > 100) + --> cmax/scripts/1-audit/checks/system/hbm_memory_exposure.py:763:101 + | +761 | def main(argv: list[str]) -> int: +762 | parser = argparse.ArgumentParser(description=__doc__) +763 | parser.add_argument("--collect-host", action="store_true", help="emit one host report instead of aggregate check JSON") + | ^^^^^^^^^^^^^^^^^^^^^^^ +764 | parser.add_argument("--root", default="/", help="host root path for proc/sys/kubelet reads") +765 | parser.add_argument("--harness", default=os.environ.get("CLUSTERMAX_AUDIT_HARNESS", "standalone")) + | + +E501 Line too long (102 > 100) + --> cmax/scripts/1-audit/checks/system/hbm_memory_exposure.py:765:101 + | +763 | parser.add_argument("--collect-host", action="store_true", help="emit one host report instead of aggregate check JSON") +764 | parser.add_argument("--root", default="/", help="host root path for proc/sys/kubelet reads") +765 | parser.add_argument("--harness", default=os.environ.get("CLUSTERMAX_AUDIT_HARNESS", "standalone")) + | ^^ +766 | args = parser.parse_args(argv) + | + +I001 [*] Import block is un-sorted or un-formatted + --> cmax/scripts/1-audit/checks/system/vm-iommu-check.py:12:1 + | +11 | sys.path.insert(0, str(Path(__file__).resolve().parents[1])) +12 | from platform_config import run_named_check + | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +help: Organize imports + | +13 | + - +14 | if __name__ == "__main__": + | + +E402 Module level import not at top of file + --> cmax/scripts/1-audit/gpu_profiles.py:24:1 + | +22 | sys.path.insert(0, WORKLOAD_DIR) +23 | +24 | from kubernetes_quantities import kubernetes_memory_gib + | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +help: Move module level imports to top of file + +E402 Module level import not at top of file + --> cmax/scripts/1-audit/merge_audit.py:17:1 + | +15 | sys.path.insert(0, WORKLOAD_DIR) +16 | +17 | from kubernetes_quantities import kubernetes_memory_gib + | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +18 | +19 | # Vendor prefixes that nvidia-smi / rocm-smi / k8s gpu.product labels prepend. + | +help: Move module level imports to top of file + +E501 Line too long (102 > 100) + --> cmax/scripts/1-audit/merge_audit.py:434:101 + | +432 | _set_default(stack, "gpuMetricsJobAttribution", _truthy(gme.get("jobAttribution"))) +433 | if gme.get("jobAttributionMethod"): +434 | _set_default(stack, "gpuMetricsJobAttributionMethod", gme.get("jobAttributionMethod")) + | ^^ +435 | if "amdDeviceMetricsExporter" in gme: +436 | amd_exp = gme.get("amdDeviceMetricsExporter") + | + +E501 Line too long (101 > 100) + --> cmax/scripts/1-audit/merge_audit.py:443:101 + | +441 | amd_exp = comps.get("amdDeviceMetricsExporter") +442 | if "nodeProblemDetector" in comps: +443 | _set_default(stack, "nodeProblemDetector", _truthy(comps.get("nodeProblemDetector"))) + | ^ +444 | if amd_exp is not None: +445 | _set_default(stack, "amdDeviceMetricsExporter", _truthy(amd_exp)) + | + +E501 Line too long (103 > 100) + --> cmax/scripts/1-audit/merge_audit.py:457:101 + | +455 | module_inventory = audit.get("lmod") +456 | module_values = ( +457 | tuple(module_inventory.get(key) for key in ("hasCudaModule", "hasHpcxModule", "hasNcclModule")) + | ^^^ +458 | if isinstance(module_inventory, dict) +459 | else () + | + +E501 Line too long (104 > 100) + --> cmax/scripts/1-audit/merge_audit.py:574:101 + | +572 | inventory_nodes = as_int(nested_get(audit, "nodes", "total"), default=1) +573 | gpu_nodes = as_int( +574 | first_known(primary_profile.get("nodeCount"), nested_get(audit, "gpus", "nodeCount"), default=0) + | ^^^^ +575 | ) +576 | cluster_nodes = gpu_nodes if gpu_nodes > 0 else inventory_nodes + | + +E501 Line too long (108 > 100) + --> cmax/scripts/1-audit/merge_audit.py:591:101 + | +589 | if gpu_nodes > 0: +590 | total_cpus = as_int( +591 | first_known(primary_profile.get("totalCpus"), nested_get(audit, "gpus", "totalCpus"), default=0) + | ^^^^^^^^ +592 | ) +593 | total_memory_gb = as_int( + | + +E501 Line too long (108 > 100) + --> cmax/scripts/1-audit/merge_audit.py:618:101 + | +616 | # The raw nvidia-smi / label string stays under audit_data.gpus.model. +617 | "gpu_model": normalize_chip_name( +618 | first_known(primary_profile.get("model"), nested_get(audit, "gpus", "model"), default="unknown") + | ^^^^^^^^ +619 | ), +620 | "nodes": cluster_nodes, + | + +E501 Line too long (113 > 100) + --> cmax/scripts/1-audit/merge_audit.py:655:101 + | +655 | def print_summary(out_path: Path, values: dict[str, Any], nic_topology: dict[str, list[dict[str, Any]]]) -> None: + | ^^^^^^^^^^^^^ +656 | cluster = values["cluster"] +657 | audit = values["audit_data"] + | + +E501 Line too long (101 > 100) + --> cmax/scripts/1-audit/merge_audit.py:686:101 + | +684 | for nic in nics: +685 | by_role[nic["role"]] = by_role.get(nic["role"], 0) + 1 +686 | print(f" nic_topology = {len(nic_topology)} node(s), {total_nics} NIC(s) total: {by_role}") + | ^ + +E501 Line too long (104 > 100) + --> cmax/scripts/1-audit/merge_audit.py:692:101 + | +690 | if len(argv) != 6: +691 | print( +692 | "usage: merge_audit.py ", + | ^^^^ +693 | file=sys.stderr, +694 | ) + | + +UP035 [*] Import from `collections.abc` instead: `Iterable` + --> cmax/scripts/1-audit/minimum_versions.py:26:1 + | +24 | from datetime import date, datetime, timedelta, timezone +25 | from pathlib import Path +26 | from typing import Any, Iterable + | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +27 | +28 | MINIMUMS_ENV = "CLUSTERMAX_MINIMUM_VERSIONS" + | +help: Import from `collections.abc` + | +25 | from pathlib import Path + - from typing import Any, Iterable +26 + from typing import Any +27 + from collections.abc import Iterable +28 | + | + +E501 Line too long (102 > 100) + --> cmax/scripts/1-audit/minimum_versions.py:417:101 + | +415 | if value is None: +416 | return 1 +417 | print(value if not isinstance(value, (dict, list)) else json.dumps(value, sort_keys=True)) + | ^^ +418 | elif args.age_days: +419 | age = age_days(path) + | + +I001 [*] Import block is un-sorted or un-formatted + --> cmax/scripts/1-audit/run_checks.py:4:1 + | + 2 | """Run audit checks and merge their JSON objects.""" + 3 | + 4 | / from __future__ import annotations + 5 | | + 6 | | import json + 7 | | import os + 8 | | import re + 9 | | import subprocess +10 | | import sys +11 | | import tempfile +12 | | from pathlib import Path +13 | | from typing import Any + | |______________________^ +help: Organize imports + | +14 | + - +15 | CHECK_PROFILES = { + | + +UP022 Prefer `capture_output` over sending `stdout` and `stderr` to `PIPE` + --> cmax/scripts/1-audit/run_checks.py:230:16 + | +228 | else [str(check)] +229 | ) +230 | proc = subprocess.run( + | ________________^ +231 | | argv, +232 | | stdout=subprocess.PIPE, +233 | | stderr=subprocess.PIPE, +234 | | text=True, +235 | | env=env, +236 | | ) + | |_________^ +237 | if proc.stderr: +238 | sys.stderr.write(proc.stderr) + | +help: Replace with `capture_output` keyword argument + +E501 Line too long (104 > 100) + --> cmax/scripts/1-audit/run_checks.py:267:101 + | +265 | " vboost " +266 | f"{vboost.get('status', 'unknown')}: " +267 | f"{vboost.get('allowed_nodes', 0)}/{vboost.get('checked_nodes', 0)} node(s) allowed" + | ^^^^ +268 | ) + | + +E501 Line too long (103 > 100) + --> cmax/scripts/1-audit/run_legacy_audit.py:30:101 + | +28 | if len(argv) != 6: +29 | print( +30 | "usage: run_legacy_audit.py ", + | ^^^ +31 | file=sys.stderr, +32 | ) + | + +UP035 [*] Import from `collections.abc` instead: `Iterable` + --> cmax/scripts/1-audit/security_version_audit.py:23:1 + | +21 | from dataclasses import asdict, dataclass, replace +22 | from pathlib import Path +23 | from typing import Any, Iterable + | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +24 | +25 | WORKLOAD_DIR = str(Path(__file__).resolve().parent) + | +help: Import from `collections.abc` + | +22 | from pathlib import Path + - from typing import Any, Iterable +23 + from typing import Any +24 + from collections.abc import Iterable +25 | + | + +I001 [*] Import block is un-sorted or un-formatted + --> cmax/scripts/1-audit/security_version_audit.py:29:1 + | +27 | sys.path.insert(0, WORKLOAD_DIR) +28 | +29 | / import minimum_versions # noqa: E402 +30 | | from minimum_versions import MinimumDataError # noqa: E402 + | |_____________________________________________^ +help: Organize imports + | +31 | + - +32 | UNKNOWN_VALUES = {"", "none", "not-found", "unknown", "n/a"} + | + +E501 Line too long (105 > 100) + --> cmax/scripts/1-audit/security_version_audit.py:581:101 + | +579 | legacy[name] = retired +580 | if not fixed: +581 | raise MinimumDataError("minimum version table has no fixed release lines for virtioNetBluefield") + | ^^^^^ +582 | return fixed, legacy + | + +E501 Line too long (101 > 100) + --> cmax/scripts/1-audit/security_version_audit.py:1802:101 + | +1800 | except json.JSONDecodeError: +1801 | return [] +1802 | return [entry for entry in parsed if isinstance(entry, dict)] if isinstance(parsed, list) else [] + | ^ + +E501 Line too long (110 > 100) + --> cmax/scripts/1-audit/validate_audit.py:74:101 + | +73 | print("", file=sys.stderr) +74 | print(f"ERROR: audit incomplete (software.workerCheckOk={str(worker_check_ok).lower()})", file=sys.stderr) + | ^^^^^^^^^^ +75 | print(" GPU / driver / NCCL fields would be 'unknown' or missing.", file=sys.stderr) +76 | print(f" Audit raw JSON saved at: {audit_path}", file=sys.stderr) + | + +I001 [*] Import block is un-sorted or un-formatted + --> cmax/security.py:1:1 + | + 1 | / from __future__ import annotations + 2 | | + 3 | | import importlib.util + 4 | | import json + 5 | | import os + 6 | | import platform + 7 | | import re + 8 | | import shutil + 9 | | import subprocess +10 | | import sys +11 | | from dataclasses import asdict, dataclass +12 | | from datetime import datetime, timezone +13 | | from pathlib import Path +14 | | from types import ModuleType +15 | | from typing import Any, Callable +16 | | +17 | | from cmax import minimum_links, report_style, runtime_paths +18 | | from cmax.yaml_support import load_yaml_module + | |______________________________________________^ +help: Organize imports + | +19 | + - +20 | PASS = "pass" + | + +F401 [*] `re` imported but unused + --> cmax/security.py:7:8 + | +5 | import os +6 | import platform +7 | import re + | ^^ +8 | import shutil +9 | import subprocess + | +help: Remove unused import: `re` + | +6 | import platform + - import re +7 | import shutil + | + +F401 [*] `datetime.timezone` imported but unused + --> cmax/security.py:12:32 + | +10 | import sys +11 | from dataclasses import asdict, dataclass +12 | from datetime import datetime, timezone + | ^^^^^^^^ +13 | from pathlib import Path +14 | from types import ModuleType + | +help: Remove unused import: `datetime.timezone` + | +11 | from dataclasses import asdict, dataclass + - from datetime import datetime, timezone +12 + from datetime import datetime +13 | from pathlib import Path + | + +UP035 [*] Import from `collections.abc` instead: `Callable` + --> cmax/security.py:15:1 + | +13 | from pathlib import Path +14 | from types import ModuleType +15 | from typing import Any, Callable + | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +16 | +17 | from cmax import minimum_links, report_style, runtime_paths + | +help: Import from `collections.abc` + | +14 | from types import ModuleType + - from typing import Any, Callable +15 + from typing import Any +16 + from collections.abc import Callable +17 | + | + +SIM112 Use capitalized environment variable `CONTAINER` instead of `container` + --> cmax/security.py:115:32 + | +113 | ): +114 | return True +115 | return bool(os.environ.get("container")) + | ^^^^^^^^^^^ +help: Replace `container` with `CONTAINER` + +SIM105 Use `contextlib.suppress(RuntimeError)` instead of `try`-`except`-`pass` + --> cmax/security.py:238:5 + | +236 | cwd = Path.cwd().resolve() +237 | candidates.extend((cwd, *cwd.parents)) +238 | / try: +239 | | candidates.append(runtime_paths.package_runtime_root()) +240 | | except RuntimeError: +241 | | pass + | |____________^ +242 | +243 | seen: set[Path] = set() + | +help: Replace `try`-`except`-`pass` with `with contextlib.suppress(RuntimeError): ...` + +E501 Line too long (104 > 100) + --> cmax/security.py:476:101 + | +474 | PASS, +475 | f"observed versions for all {len(devices)} devices meet minimum version {minimum}", +476 | "Every detected ConnectX or BlueField device meets the published firmware minimum version.", + | ^^^^ +477 | ) + | + +E501 Line too long (112 > 100) + --> cmax/security.py:578:101 + | +576 | WARNING, +577 | observed, +578 | "The installed kernel inventory was unavailable, so a pending security reboot cannot be ruled out.", + | ^^^^^^^^^^^^ +579 | ) +580 | if newer is False or str(newer).lower() == "false": + | + +E501 Line too long (108 > 100) + --> cmax/security.py:585:101 + | +583 | WARNING, +584 | observed, +585 | "The installed kernel inventory was unavailable, so a pending security reboot cannot be ruled out.", + | ^^^^^^^^ +586 | ) + | + +E501 Line too long (111 > 100) + --> cmax/security.py:610:101 + | +608 | CRITICAL, +609 | observed, +610 | "The running kernel is below the distribution minimum for known local privilege-escalation fixes.", + | ^^^^^^^^^^^ +611 | ) +612 | if status in {"not-applicable", "not_applicable"}: + | + +E501 Line too long (105 > 100) + --> cmax/security.py:664:101 + | +662 | WARNING, +663 | observed, +664 | "The physical-host kernel patch state and nested virtualization boundary could not be verified.", + | ^^^^^ +665 | ) + | + +E501 Line too long (156 > 100) + --> cmax/security.py:761:101 + | +759 | … +760 | … not recorded", +761 | …s, but this older artifact does not show whether the check used a privileged host-root path.", + | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +762 | … +763 | … + | + +E501 Line too long (110 > 100) + --> cmax/security.py:785:101 + | +783 | WARNING, +784 | f"{label}=true", +785 | "This isolation boundary is controlled by the physical host and needs provider verification.", + | ^^^^^^^^^^ +786 | ) +787 | if value is False or str(value).lower() == "false": + | + +E501 Line too long (114 > 100) + --> cmax/security.py:1150:101 + | +1148 | "nvidia-driver", +1149 | "NVIDIA driver minimum version", +1150 | "GPU driver vulnerabilities can expose the host kernel through device interfaces available to workloads.", + | ^^^^^^^^^^^^^^ +1151 | # The bulletin link comes from the minimum table (see MINIMUM_BOUND_CHECKS). +1152 | # It was hardcoded here, so it kept naming May 2026 and pointing at that + | + +E501 Line too long (107 > 100) + --> cmax/security.py:1154:101 + | +1152 | # It was hardcoded here, so it kept naming May 2026 and pointing at that +1153 | # bulletin after a refresh moved a branch minimum to a later one. +1154 | "Install the fixed release for the deployed NVIDIA driver branch and reload the driver or reboot.", + | ^^^^^^^ +1155 | _references( +1156 | ("NVIDIA product security", "https://www.nvidia.com/en-us/security/"), + | + +E501 Line too long (108 > 100) + --> cmax/security.py:1163:101 + | +1161 | "nvidia-container-toolkit", +1162 | "NVIDIA Container Toolkit minimum version", +1163 | "Affected container hooks can allow a crafted image to escape its container during initialization.", + | ^^^^^^^^ +1164 | # The minimum version, its bulletin, and its CVEs come from the generated +1165 | # minimum table at evaluation time (see MINIMUM_BOUND_CHECKS), so this text + | + +E501 Line too long (113 > 100) + --> cmax/security.py:1167:101 + | +1165 | # minimum table at evaluation time (see MINIMUM_BOUND_CHECKS), so this text +1166 | # cannot drift from the minimum the audit graded against. +1167 | "Upgrade NVIDIA Container Toolkit to the published minimum version, then restart the container runtime.", + | ^^^^^^^^^^^^^ +1168 | _references( +1169 | ("NVIDIA product security", "https://www.nvidia.com/en-us/security/"), + | + +E501 Line too long (105 > 100) + --> cmax/security.py:1176:101 + | +1174 | "cuda-toolkit", +1175 | "CUDA Toolkit minimum version", +1176 | "CUDA Toolkit vulnerabilities can affect compiler and runtime components used by GPU workloads.", + | ^^^^^ +1177 | # The minimum version, its bulletin, and its CVEs come from the minimum +1178 | # table at evaluation time (see MINIMUM_BOUND_CHECKS). The superseded + | + +E501 Line too long (111 > 100) + --> cmax/security.py:1193:101 + | +1191 | "runc", +1192 | "runc minimum version", +1193 | "Affected OCI runtime releases contain container setup races that can bypass masked-path protections.", + | ^^^^^^^^^^^ +1194 | "Upgrade to the fixed release for the installed runc branch and restart the container runtime.", +1195 | # The advisory link comes from the minimum table (see MINIMUM_BOUND_CHECKS). + | + +E501 Line too long (104 > 100) + --> cmax/security.py:1194:101 + | +1192 | "runc minimum version", +1193 | "Affected OCI runtime releases contain container setup races that can bypass masked-path protections.", +1194 | "Upgrade to the fixed release for the installed runc branch and restart the container runtime.", + | ^^^^ +1195 | # The advisory link comes from the minimum table (see MINIMUM_BOUND_CHECKS). +1196 | # These CVEs are the named masked-path race and stay as history; the + | + +E501 Line too long (106 > 100) + --> cmax/security.py:1223:101 + | +1221 | "connectx-firmware", +1222 | "ConnectX and BlueField firmware minimum version", +1223 | "NIC firmware handles direct memory access and fabric traffic at a privileged hardware boundary.", + | ^^^^^^ +1224 | "Apply the fixed NVIDIA firmware release for every detected ConnectX or BlueField device.", +1225 | # The bulletin link comes from the minimum table (see MINIMUM_BOUND_CHECKS), + | + +E501 Line too long (107 > 100) + --> cmax/security.py:1270:101 + | +1268 | "januscape", +1269 | "Januscape nested virtualization boundary", +1270 | "Nested virtualization exposure can let an untrusted guest attack the physical-host KVM boundary.", + | ^^^^^^^ +1271 | "Disable nested virtualization and remove guest access to /dev/kvm until the physical-host vendor fix is installed.", +1272 | _references( + | + +E501 Line too long (125 > 100) + --> cmax/security.py:1271:101 + | +1269 | "Januscape nested virtualization boundary", +1270 | "Nested virtualization exposure can let an untrusted guest attack the physical-host KVM boundary.", +1271 | "Disable nested virtualization and remove guest access to /dev/kvm until the physical-host vendor fix is installed.", + | ^^^^^^^^^^^^^^^^^^^^^^^^^ +1272 | _references( +1273 | ( + | + +E501 Line too long (112 > 100) + --> cmax/security.py:1301:101 + | +1299 | "ufm-profile", +1300 | "InfiniBand UFM secured profile", +1301 | "Unauthenticated fabric management traffic can expose neighboring tenants or enable denial of service.", + | ^^^^^^^^^^^^ +1302 | "Enable and verify the UFM Secured Bare Metal Cloud controls, including randomized management keys and rate limiting.", +1303 | _references( + | + +E501 Line too long (127 > 100) + --> cmax/security.py:1302:101 + | +1300 | "InfiniBand UFM secured profile", +1301 | "Unauthenticated fabric management traffic can expose neighboring tenants or enable denial of service.", +1302 | "Enable and verify the UFM Secured Bare Metal Cloud controls, including randomized management keys and rate limiting.", + | ^^^^^^^^^^^^^^^^^^^^^^^^^^^ +1303 | _references( +1304 | ( + | + +E501 Line too long (107 > 100) + --> cmax/security.py:1318:101 + | +1316 | "pcie-passthrough", +1317 | "PCIe passthrough isolation", +1318 | "Passed-through accelerators depend on host IOMMU, ACS, reset isolation, and memory sanitization.", + | ^^^^^^^ +1319 | "Have the provider verify dedicated host IOMMU groups, DMA remapping, ACS, reset isolation, and VRAM clearing.", +1320 | _references( + | + +E501 Line too long (120 > 100) + --> cmax/security.py:1319:101 + | +1317 | "PCIe passthrough isolation", +1318 | "Passed-through accelerators depend on host IOMMU, ACS, reset isolation, and memory sanitization.", +1319 | "Have the provider verify dedicated host IOMMU groups, DMA remapping, ACS, reset isolation, and VRAM clearing.", + | ^^^^^^^^^^^^^^^^^^^^ +1320 | _references( +1321 | ( + | + +E501 Line too long (102 > 100) + --> cmax/security.py:1677:101 + | +1675 | if verbosity < 3: +1676 | hint = { +1677 | 1: "run with -vv for CVE and documentation links, -vvv for issue details and remediation", + | ^^ +1678 | 2: "run with -vvv for issue details and remediation", +1679 | }[verbosity] + | + +Found 224 errors. +[*] 41 fixable with the `--fix` option (13 hidden fixes can be enabled with the `--unsafe-fixes` option). diff --git a/reports/static-analysis/shellcheck.txt b/reports/static-analysis/shellcheck.txt new file mode 100644 index 0000000..226a6b1 --- /dev/null +++ b/reports/static-analysis/shellcheck.txt @@ -0,0 +1,1435 @@ + +In cmax/scripts/1-audit/audit-common.sh line 1: +# audit-common.sh - shared helpers sourced by the per-harness audit collectors +^-- SC2148 (error): Tips depend on target shell and yours is unknown. Add a shebang or a 'shell' directive. + + +In cmax/scripts/1-audit/audit-common.sh line 361: +' $VIRTIO_NET_SUMMARY_KEYS 2>/dev/null) || return 0 + ^----------------------^ SC2086 (info): Double quote to prevent globbing and word splitting. + +Did you mean: +' "$VIRTIO_NET_SUMMARY_KEYS" 2>/dev/null) || return 0 + + +In cmax/scripts/1-audit/audit-common.sh line 1041: + "model": "${GPU_MODEL}", + ^----------^ SC2153 (info): Possible misspelling: GPU_MODEL may not be assigned. Did you mean gpu_model? + + +In cmax/scripts/1-audit/audit-common.sh line 1223: + "rdmaType": "${RDMA_TYPE}", + ^----------^ SC2153 (info): Possible misspelling: RDMA_TYPE may not be assigned. Did you mean rdma_type? + + +In cmax/scripts/1-audit/cluster-audit-k8s.sh line 1118: +FIRST_GPU_NODE=$(head -n 1 <<< "$GPU_NODE_NAMES") +^------------^ SC2034 (warning): FIRST_GPU_NODE appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-k8s.sh line 1459: + NCU_TEST_SCRIPT=' + ^-- SC2016 (info): Expressions don't expand in single quotes, use double quotes for that. + + +In cmax/scripts/1-audit/cluster-audit-k8s.sh line 2071: + *"nginx"*|*"ingress-nginx"*) + ^-------^ SC2221 (warning): This pattern always overrides a later one on line 2071. + ^---------------^ SC2222 (warning): This pattern never matches because of a previous pattern on line 2071. + + +In cmax/scripts/1-audit/cluster-audit-k8s.sh line 2349: + STORAGE_CHECK_SCRIPT=' + ^-- SC2016 (info): Expressions don't expand in single quotes, use double quotes for that. + + +In cmax/scripts/1-audit/cluster-audit-k8s.sh line 2491: +[[ "$MPI_OPERATOR" == "true" ]] && print_info "MPI Operator: Installed" || print_warn "MPI Operator: Not installed" + ^-- SC2015 (info): Note that A && B || C is not if-then-else. C may run when A is true. + + +In cmax/scripts/1-audit/cluster-audit-k8s.sh line 2492: +[[ "$KUEUE" == "true" ]] && print_info "Kueue: Installed" || print_warn "Kueue: Not installed" + ^-- SC2015 (info): Note that A && B || C is not if-then-else. C may run when A is true. + + +In cmax/scripts/1-audit/cluster-audit-k8s.sh line 2493: +[[ "$VOLCANO" == "true" ]] && print_info "Volcano: Installed" || print_warn "Volcano: Not installed" + ^-- SC2015 (info): Note that A && B || C is not if-then-else. C may run when A is true. + + +In cmax/scripts/1-audit/cluster-audit-k8s.sh line 2494: +[[ "$TRAINING_OPERATOR" == "true" ]] && print_info "Training Operator: Installed" || print_warn "Training Operator: Not installed" + ^-- SC2015 (info): Note that A && B || C is not if-then-else. C may run when A is true. + + +In cmax/scripts/1-audit/cluster-audit-k8s.sh line 2631: + local script=' + ^-- SC2016 (info): Expressions don't expand in single quotes, use double quotes for that. + + +In cmax/scripts/1-audit/cluster-audit-k8s.sh line 2722: + outcome=$(check_deadline 30 kubectl exec -n "$ns" "$pod" -- chroot /host bash -c ' + ^-- SC2016 (info): Expressions don't expand in single quotes, use double quotes for that. + + +In cmax/scripts/1-audit/cluster-audit-k8s.sh line 3412: + DCGM_EXPORTER_IMAGE=$(printf '%s' "$MONITORING_JSON" \ + ^-----------------^ SC2034 (warning): DCGM_EXPORTER_IMAGE appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm-security.sh line 15: +RED='\033[0;31m' +^-^ SC2034 (warning): RED appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm-security.sh line 16: +GREEN='\033[0;32m' +^---^ SC2034 (warning): GREEN appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm-security.sh line 17: +YELLOW='\033[1;33m' +^----^ SC2034 (warning): YELLOW appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm-security.sh line 18: +BLUE='\033[0;34m' +^--^ SC2034 (warning): BLUE appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm-security.sh line 19: +CYAN='\033[0;36m' +^--^ SC2034 (warning): CYAN appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm-security.sh line 20: +BOLD='\033[1m' +^--^ SC2034 (warning): BOLD appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm-security.sh line 21: +NC='\033[0m' +^-- SC2034 (warning): NC appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm-security.sh line 107: + K8S_SECURITY_NODES_JSON="$nodes_json" + ^---------------------^ SC2034 (warning): K8S_SECURITY_NODES_JSON appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm-security.sh line 484: + DCGM_EXPORTER_PRESENT=unknown + ^-------------------^ SC2034 (warning): DCGM_EXPORTER_PRESENT appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 32: +RED='\033[0;31m' +^-^ SC2034 (warning): RED appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 33: +GREEN='\033[0;32m' +^---^ SC2034 (warning): GREEN appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 34: +YELLOW='\033[1;33m' +^----^ SC2034 (warning): YELLOW appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 35: +BLUE='\033[0;34m' +^--^ SC2034 (warning): BLUE appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 36: +CYAN='\033[0;36m' +^--^ SC2034 (warning): CYAN appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 37: +BOLD='\033[1m' +^--^ SC2034 (warning): BOLD appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 38: +NC='\033[0m' # No Color +^-- SC2034 (warning): NC appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 140: +AUDIT_TIMESTAMP=$(date -u +"%Y-%m-%dT%H:%M:%SZ") +^-------------^ SC2034 (warning): AUDIT_TIMESTAMP appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 190: +[[ "$SLURMD_RUNNING" == "true" ]] && print_info "slurmd: Running" || print_detail "slurmd: Not running (head node may not be compute)" + ^-- SC2015 (info): Note that A && B || C is not if-then-else. C may run when A is true. + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 276: + GPU_TOTAL_CPUS=$(echo "$GPU_NODE_SUMMARY" | jq -r '.totalCpus // 0') + ^------------^ SC2034 (warning): GPU_TOTAL_CPUS appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 277: + GPU_TOTAL_MEMORY_GB=$(echo "$GPU_NODE_SUMMARY" | jq -r '.totalMemoryGB // 0') + ^-----------------^ SC2034 (warning): GPU_TOTAL_MEMORY_GB appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 297: +[[ "$DOWN_NODES" -gt 0 ]] && print_warn "Down/Drained: ${DOWN_NODES}" || print_info "Down/Drained: ${DOWN_NODES}" + ^-- SC2015 (info): Note that A && B || C is not if-then-else. C may run when A is true. + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 320: + [[ "$default" == "YES" ]] && print_info "${name} (DEFAULT): ${nodes} nodes, max ${max_time}" || print_info "${name}: ${nodes} nodes, max ${max_time}" + ^-- SC2015 (info): Note that A && B || C is not if-then-else. C may run when A is true. + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 393: +WORKER_PEERMEM="false" +^------------^ SC2034 (warning): WORKER_PEERMEM appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 458: +WORKER_BOOT_SIZE="unknown" +^--------------^ SC2034 (warning): WORKER_BOOT_SIZE appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 825: +ACS_BRIDGES="${WORKER_ACS_BRIDGES:-0}" +^---------^ SC2034 (warning): ACS_BRIDGES appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 826: +ACS_ENABLED_COUNT="${WORKER_ACS_ENABLED_COUNT:-0}" +^---------------^ SC2034 (warning): ACS_ENABLED_COUNT appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 827: +ACS_TOTAL_BRIDGES="${WORKER_ACS_TOTAL_BRIDGES:-0}" +^---------------^ SC2034 (warning): ACS_TOTAL_BRIDGES appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 829: +ACS_METHOD="${WORKER_ACS_METHOD:-config}" +^--------^ SC2034 (warning): ACS_METHOD appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 835: + ACS_SCOPED="true"; ACS_SUPPORTED="true" + ^--------^ SC2034 (warning): ACS_SCOPED appears unused. Verify use (or export if used externally). + ^-----------^ SC2034 (warning): ACS_SUPPORTED appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 891: + ACS_ENABLED="unknown" + ^---------^ SC2034 (warning): ACS_ENABLED appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 916: + [[ "$WORKER_GDRCOPY_DEV" == "true" ]] && print_info "/dev/gdrdrv: present" || print_detail "/dev/gdrdrv: missing (gdrdrv not loaded or no permissions)" + ^-- SC2015 (info): Note that A && B || C is not if-then-else. C may run when A is true. + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 943: + ROCM_SMI_AVAILABLE="true" + ^----------------^ SC2034 (warning): ROCM_SMI_AVAILABLE appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 949: + AMD_SMI_AVAILABLE="true" + ^---------------^ SC2034 (warning): AMD_SMI_AVAILABLE appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 981: + AMD_PEERMEM_LOADED="false" + ^----------------^ SC2034 (warning): AMD_PEERMEM_LOADED appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 985: + ROCM_CONTAINER_TOOLKIT="true" + ^--------------------^ SC2034 (warning): ROCM_CONTAINER_TOOLKIT appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 991: + RDC_INSTALLED="true" + ^-----------^ SC2034 (warning): RDC_INSTALLED appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 1002: + ROCM_BANDWIDTH_TEST_INSTALLED="true" + ^---------------------------^ SC2034 (warning): ROCM_BANDWIDTH_TEST_INSTALLED appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 1010: + ROCPROF_INSTALLED="true" + ^---------------^ SC2034 (warning): ROCPROF_INSTALLED appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 1033: + RVS_INSTALLED="true" + ^-----------^ SC2034 (warning): RVS_INSTALLED appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 1039: + TRANSFERBENCH_INSTALLED="true" + ^---------------------^ SC2034 (warning): TRANSFERBENCH_INSTALLED appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 1198: + NCU_PROFILING_CONF_FOUND="true" + ^----------------------^ SC2034 (warning): NCU_PROFILING_CONF_FOUND appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 1396: + NVCC_IN_PATH="true" + ^----------^ SC2034 (warning): NVCC_IN_PATH appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 1451: + NVHPC_STATUS="fail" + ^----------^ SC2034 (warning): NVHPC_STATUS appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 1507: + NCCL_INSTALLED="true" + ^------------^ SC2034 (warning): NCCL_INSTALLED appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 1532: + NCCL_CONF_OVERRIDES="true" + ^-----------------^ SC2034 (warning): NCCL_CONF_OVERRIDES appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 1575: + MPI_INSTALLED="true" + ^-----------^ SC2034 (warning): MPI_INSTALLED appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 1631: + SRUN_MPI_PMIX="true" + ^-----------^ SC2034 (warning): SRUN_MPI_PMIX appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 1674: + HAS_CUDA_MODULE="true" + ^-------------^ SC2034 (warning): HAS_CUDA_MODULE appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 1681: + HAS_HPCX_MODULE="true" + ^-------------^ SC2034 (warning): HAS_HPCX_MODULE appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 1688: + HAS_NCCL_MODULE="true" + ^-------------^ SC2034 (warning): HAS_NCCL_MODULE appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 1751: + PYXIS_CLI_AVAILABLE="true" + ^-----------------^ SC2034 (warning): PYXIS_CLI_AVAILABLE appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 1753: + PYXIS_RUNTIME_WORKS="true" + ^-----------------^ SC2034 (warning): PYXIS_RUNTIME_WORKS appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 1843: + DOCKER_NVIDIA_RUNTIME_CONFIGURED="${WORKER_CONTAINER_NVIDIA_RUNTIME_CONFIGURED:-false}" + ^-- SC2034 (warning): DOCKER_NVIDIA_RUNTIME_CONFIGURED appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 1844: + RUNC_INSTALLED="${WORKER_CONTAINER_RUNC_INSTALLED:-false}" + ^------------^ SC2034 (warning): RUNC_INSTALLED appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 1845: + RUNC_VERSION="${WORKER_CONTAINER_RUNC_VERSION:-unknown}" + ^----------^ SC2034 (warning): RUNC_VERSION appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 1855: + version_meets_minimum "$DOCKER_VERSION" "$DOCKER_RECOMMENDED_MIN" && DOCKER_VERSION_OK="true" + ^---------------^ SC2034 (warning): DOCKER_VERSION_OK appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 1856: + version_meets_minimum "$NVIDIA_CT_VERSION" "$NVIDIA_CT_RECOMMENDED_MIN" && NVIDIA_CT_VERSION_OK="true" + ^------------------^ SC2034 (warning): NVIDIA_CT_VERSION_OK appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 1858: + [[ "${WORKER_CONTAINER_ENROOT_IMPORT:-}" == "pass" ]] && ENROOT_IMPORT_WORKS="true" + ^-----------------^ SC2034 (warning): ENROOT_IMPORT_WORKS appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 1868: + [[ "$NVIDIA_CONTAINER_TOOLKIT" == "true" ]] && print_info "NVIDIA Container Toolkit: ${NVIDIA_CT_VERSION}" || print_warn "NVIDIA Container Toolkit: not found on worker" + ^-- SC2015 (info): Note that A && B || C is not if-then-else. C may run when A is true. + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 1869: + [[ "$ENROOT_INSTALLED" == "true" ]] && print_info "Enroot: ${ENROOT_VERSION}; import=${WORKER_CONTAINER_ENROOT_IMPORT}" || print_warn "Enroot: not found on worker" + ^-- SC2015 (info): Note that A && B || C is not if-then-else. C may run when A is true. + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 1870: + [[ "$SINGULARITY_INSTALLED" == "true" ]] && print_info "Singularity/Apptainer: ${SINGULARITY_VERSION}" || print_warn "Singularity/Apptainer: not found on worker" + ^-- SC2015 (info): Note that A && B || C is not if-then-else. C may run when A is true. + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 1945: + MOFED_FLAVOR="${WORKER_MOFED_FLAVOR:-none}" + ^----------^ SC2034 (warning): MOFED_FLAVOR appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 1987: + IB_INSTALLED="true" + ^----------^ SC2034 (warning): IB_INSTALLED appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 1994: + HCA_DEVICES_JSON=$(printf '%s\n' "${HCA_DEVICES_LIST[@]}" | jq -R . | jq -s .) + ^--------------^ SC2034 (warning): HCA_DEVICES_JSON appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 2068: + [[ "$IB_PKEYS_CONFIGURED" == "true" ]] && print_info "IB PKeys: Configured" || print_warn "IB PKeys: No valid partition keys found" + ^-- SC2015 (info): Note that A && B || C is not if-then-else. C may run when A is true. + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 2187: + NIC_HAS_OTHER="true" + ^-----------^ SC2034 (warning): NIC_HAS_OTHER appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 2258: + NIC_FABRIC_JSON="[]" + ^-------------^ SC2034 (warning): NIC_FABRIC_JSON appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 2270: +NETWORK_UTILITIES_JSON="{$(IFS=,; echo "${NETUTILS_JSON_ENTRIES[*]}")}" +^--------------------^ SC2034 (warning): NETWORK_UTILITIES_JSON appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 2310: + IB_TENANT_ISOLATION="pass" + ^-----------------^ SC2034 (warning): IB_TENANT_ISOLATION appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 2317: +SAQUERY_OUT=$(sudo -n saquery NodeRecord 2>/dev/null | grep NodeDescription | wc -l | tr -d '[:space:]' || echo "0") + ^------------------^ SC2126 (style): Consider using 'grep -c' instead of 'grep|wc -l'. + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 2346: + IB_SM_KEY_CONFIGURED="true" + ^------------------^ SC2034 (warning): IB_SM_KEY_CONFIGURED appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 2366: + SHARP_AVAILABLE="true" + ^-------------^ SC2034 (warning): SHARP_AVAILABLE appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 2373: + SHARP_AM_KEY_CONFIGURED="true" + ^---------------------^ SC2034 (warning): SHARP_AM_KEY_CONFIGURED appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 2739: + [[ "$DCGM_INSTALLED" == "true" ]] && print_warn "DCGM binary found but service NOT active on compute node" \ + ^-- SC2015 (info): Note that A && B || C is not if-then-else. C may run when A is true. + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 2745: + DCGM_HEALTH_WATCHES_ENABLED="true" + ^-------------------------^ SC2034 (warning): DCGM_HEALTH_WATCHES_ENABLED appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 2773: + && print_info "DCGM service: Active on head node" \ + ^-- SC2015 (info): Note that A && B || C is not if-then-else. C may run when A is true. + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 2805: +[[ -n "$PROLOG" && "$PROLOG" != "(null)" ]] && print_info "Prolog: ${PROLOG}" || print_detail "Prolog: Not configured" + ^-- SC2015 (info): Note that A && B || C is not if-then-else. C may run when A is true. + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 2806: +[[ -n "$EPILOG" && "$EPILOG" != "(null)" ]] && print_info "Epilog: ${EPILOG}" || print_detail "Epilog: Not configured" + ^-- SC2015 (info): Note that A && B || C is not if-then-else. C may run when A is true. + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 2867: + NHC_PATH=$(ls /usr/sbin/nhc /usr/local/sbin/nhc 2>/dev/null | head -1) + ^-- SC2012 (info): Use find instead of ls to better handle non-alphanumeric filenames. + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 2916: +[[ "$PROMETHEUS_DETECTED" == "true" ]] && print_info "Prometheus: detected (port 9090 or systemd active)" || print_detail "Prometheus: not detected" + ^-- SC2015 (info): Note that A && B || C is not if-then-else. C may run when A is true. + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 2917: +[[ "$DCGM_EXPORTER_DETECTED" == "true" ]] && print_info "dcgm-exporter: detected (port 9400 or systemd active)" || print_detail "dcgm-exporter: not detected" + ^-- SC2015 (info): Note that A && B || C is not if-then-else. C may run when A is true. + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 2918: +[[ "$NODE_EXPORTER_DETECTED" == "true" ]] && print_info "node-exporter: detected (port 9100 or systemd active)" || print_detail "node-exporter: not detected" + ^-- SC2015 (info): Note that A && B || C is not if-then-else. C may run when A is true. + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 2919: +[[ "$GRAFANA_DETECTED" == "true" ]] && print_info "Grafana: detected (port 3000 or systemd active)" || print_detail "Grafana: not detected" + ^-- SC2015 (info): Note that A && B || C is not if-then-else. C may run when A is true. + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 2945: +[[ -n "$UNKILLABLE_STEP" && "$UNKILLABLE_STEP" != "(null)" ]] && \ + ^-- SC2015 (info): Note that A && B || C is not if-then-else. C may run when A is true. + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 2961: + PASSWORDLESS_SSH="enabled" + ^--------------^ SC2034 (warning): PASSWORDLESS_SSH appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 3007: + SACCT_AVAILABLE="true" + ^-------------^ SC2034 (warning): SACCT_AVAILABLE appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 3041: +ESSENTIAL_TOOLS_JSON="{$(IFS=,; echo "${TOOLS_JSON_ENTRIES[*]}")}" +^------------------^ SC2034 (warning): ESSENTIAL_TOOLS_JSON appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 3049: + IPMITOOL_INSTALLED="true" + ^----------------^ SC2034 (warning): IPMITOOL_INSTALLED appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 3071: + IPMI_EXPOSED="true" + ^----------^ SC2034 (warning): IPMI_EXPOSED appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 3089: + SUDO_AVAILABLE="true" + ^------------^ SC2034 (warning): SUDO_AVAILABLE appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 3103: + SLURM_CMD_AVAIL[$cmd]="false" + ^-------------------^ SC2034 (warning): SLURM_CMD_AVAIL appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 3104: + SLURM_CMDS_OK="false" + ^-----------^ SC2034 (warning): SLURM_CMDS_OK appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 3116: +[[ "$USERADD_AVAILABLE" == "true" ]] && print_info "useradd: $(command -v useradd)" || print_warn "useradd: not found" + ^-- SC2015 (info): Note that A && B || C is not if-then-else. C may run when A is true. + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 3117: +[[ "$GROUPADD_AVAILABLE" == "true" ]] && print_info "groupadd: $(command -v groupadd)" || print_warn "groupadd: not found" + ^-- SC2015 (info): Note that A && B || C is not if-then-else. C may run when A is true. + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 3283: + [[ -n "$PART_DEF_MEM" ]] && print_info " DefMemPerNode: ${PART_DEF_MEM} MB" || print_detail " DefMemPerNode: Not set (using cluster default)" + ^-- SC2015 (info): Note that A && B || C is not if-then-else. C may run when A is true. + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 3285: + [[ -n "$PART_MAX_CPUS" ]] && print_info " MaxCPUsPerNode: ${PART_MAX_CPUS}" || print_detail " MaxCPUsPerNode: Unlimited" + ^-- SC2015 (info): Note that A && B || C is not if-then-else. C may run when A is true. + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 3375: +SECURITY_VERSION_AUDIT_JSON=$(build_security_version_audit \ +^-------------------------^ SC2034 (warning): SECURITY_VERSION_AUDIT_JSON appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-slurm.sh line 3387: +AUDIT_TYPE="slurm" +^--------^ SC2034 (warning): AUDIT_TYPE appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 42: +RED='\033[0;31m' +^-^ SC2034 (warning): RED appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 43: +GREEN='\033[0;32m' +^---^ SC2034 (warning): GREEN appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 44: +YELLOW='\033[1;33m' +^----^ SC2034 (warning): YELLOW appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 45: +BLUE='\033[0;34m' +^--^ SC2034 (warning): BLUE appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 46: +CYAN='\033[0;36m' +^--^ SC2034 (warning): CYAN appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 47: +BOLD='\033[1m' +^--^ SC2034 (warning): BOLD appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 48: +NC='\033[0m' # No Color +^-- SC2034 (warning): NC appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 89: +AUDIT_TIMESTAMP=$(date -u +"%Y-%m-%dT%H:%M:%SZ") +^-------------^ SC2034 (warning): AUDIT_TIMESTAMP appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 100: +SLURM_VERSION="n/a (standalone)" +^-----------^ SC2034 (warning): SLURM_VERSION appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 101: +SLURM_VERSION_NUM="0.0.0" +^---------------^ SC2034 (warning): SLURM_VERSION_NUM appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 103: +CONTROL_MACHINE="n/a" +^-------------^ SC2034 (warning): CONTROL_MACHINE appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 104: +SLURM_USER="n/a" +^--------^ SC2034 (warning): SLURM_USER appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 105: +SLURMCTLD_RUNNING="false" +^---------------^ SC2034 (warning): SLURMCTLD_RUNNING appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 106: +SLURMD_RUNNING="false" +^------------^ SC2034 (warning): SLURMD_RUNNING appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 107: +SLURMDBD_RUNNING="false" +^--------------^ SC2034 (warning): SLURMDBD_RUNNING appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 150: +IDLE_NODES=1 +^--------^ SC2034 (warning): IDLE_NODES appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 151: +ALLOCATED_NODES=0 +^-------------^ SC2034 (warning): ALLOCATED_NODES appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 152: +DOWN_NODES=0 +^--------^ SC2034 (warning): DOWN_NODES appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 157: +NODES_JSON=$(cat </dev/null | tr '\n' ',' | sed 's/,$//' || echo "unknown") + ^-- SC2012 (info): Use find instead of ls to better handle non-alphanumeric filenames. + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 1145: + NVHPC_INSTALLED="true" + ^-------------^ SC2034 (warning): NVHPC_INSTALLED appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 1186: + NCCL_INSTALLED="true" + ^------------^ SC2034 (warning): NCCL_INSTALLED appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 1196: +NCCL_CONF="/etc/nccl.conf" +^-------^ SC2034 (warning): NCCL_CONF appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 1197: +NCCL_CONF_OVERRIDES="false" +^-----------------^ SC2034 (warning): NCCL_CONF_OVERRIDES appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 1198: +MPIRUN_PATH="" +^---------^ SC2034 (warning): MPIRUN_PATH appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 1199: +MPI_INSTALLED="false" +^-----------^ SC2034 (warning): MPI_INSTALLED appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 1200: +HPCX_IN_PATH="false" +^----------^ SC2034 (warning): HPCX_IN_PATH appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 1201: +HPCX_PATH="" +^-------^ SC2034 (warning): HPCX_PATH appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 1202: +SRUN_MPI_PMIX="false" +^-----------^ SC2034 (warning): SRUN_MPI_PMIX appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 1209: +LMOD_INSTALLED="false" +^------------^ SC2034 (warning): LMOD_INSTALLED appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 1210: +LMOD_VERSION="n/a" +^----------^ SC2034 (warning): LMOD_VERSION appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 1211: +HAS_CUDA_MODULE="false" +^-------------^ SC2034 (warning): HAS_CUDA_MODULE appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 1212: +HAS_HPCX_MODULE="false" +^-------------^ SC2034 (warning): HAS_HPCX_MODULE appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 1213: +HAS_NCCL_MODULE="false" +^-------------^ SC2034 (warning): HAS_NCCL_MODULE appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 1222: +PYXIS_INSTALLED="false" +^-------------^ SC2034 (warning): PYXIS_INSTALLED appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 1255: + ENROOT_IMPORT_WORKS="true" + ^-----------------^ SC2034 (warning): ENROOT_IMPORT_WORKS appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 1280: +CONTAINER_WORKER_CHECK_OK="true" +^-----------------------^ SC2034 (warning): CONTAINER_WORKER_CHECK_OK appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 1281: +CONTAINER_WORKER_NODE=$(hostname 2>/dev/null || echo "localhost") +^-------------------^ SC2034 (warning): CONTAINER_WORKER_NODE appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 1312: + DOCKER_VERSION_OK="true" + ^---------------^ SC2034 (warning): DOCKER_VERSION_OK appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 1323: + DOCKER_NVIDIA_RUNTIME_CONFIGURED="true" + ^-- SC2034 (warning): DOCKER_NVIDIA_RUNTIME_CONFIGURED appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 1331: +DOCKER_ON_HEAD="$DOCKER_INSTALLED" +^------------^ SC2034 (warning): DOCKER_ON_HEAD appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 1360: + NVIDIA_CT_VERSION_OK="true" + ^------------------^ SC2034 (warning): NVIDIA_CT_VERSION_OK appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 1389: + CONTAINER_RUNTIME_SCOPE="host" + ^---------------------^ SC2034 (warning): CONTAINER_RUNTIME_SCOPE appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 1404: +DOCKER_ON_WORKERS="$DOCKER_INSTALLED" +^---------------^ SC2034 (warning): DOCKER_ON_WORKERS appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 1415: + SINGULARITY_INSTALLED="true" + ^-------------------^ SC2034 (warning): SINGULARITY_INSTALLED appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 1556: + [[ "$IB_PKEYS_CONFIGURED" == "true" ]] && print_info "IB PKeys: Configured" || print_warn "IB PKeys: No valid partition keys found" + ^-- SC2015 (info): Note that A && B || C is not if-then-else. C may run when A is true. + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 1569: + print_detail " $(basename $pf): ${pv} (default partition)" + ^-^ SC2086 (info): Double quote to prevent globbing and word splitting. + +Did you mean: + print_detail " $(basename "$pf"): ${pv} (default partition)" + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 1573: + print_detail " $(basename $pf): ${pv}" + ^-^ SC2086 (info): Double quote to prevent globbing and word splitting. + +Did you mean: + print_detail " $(basename "$pf"): ${pv}" + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 1801: +SAQUERY_OUT=$(sudo -n saquery NodeRecord 2>/dev/null | grep NodeDescription | wc -l | tr -d '[:space:]' || echo "0") + ^------------------^ SC2126 (style): Consider using 'grep -c' instead of 'grep|wc -l'. + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 1872: + IB_INSTALLED=false + ^----------^ SC2034 (warning): IB_INSTALLED appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 1875: + HCA_DEVICES_JSON="[]" + ^--------------^ SC2034 (warning): HCA_DEVICES_JSON appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 1878: + HCA_NAMING_VALID=false + ^--------------^ SC2034 (warning): HCA_NAMING_VALID appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 1882: + NIC_FABRIC_JSON="[]" + ^-------------^ SC2034 (warning): NIC_FABRIC_JSON appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 1890: + NIC_HAS_OTHER=false + ^-----------^ SC2034 (warning): NIC_HAS_OTHER appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 1891: + NETWORK_UTILITIES_JSON="{}" + ^--------------------^ SC2034 (warning): NETWORK_UTILITIES_JSON appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 1892: + NCCL_GID_INDEX_VALUE=unset + ^------------------^ SC2209 (warning): Use var=$(command) to assign output (or quote to assign string). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 1893: + IB_TENANT_ISOLATION=not_applicable + ^-----------------^ SC2034 (warning): IB_TENANT_ISOLATION appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 1894: + IB_SM_KEY_CONFIGURED=false + ^------------------^ SC2034 (warning): IB_SM_KEY_CONFIGURED appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 1895: + SHARP_AVAILABLE=false + ^-------------^ SC2034 (warning): SHARP_AVAILABLE appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 1896: + SHARP_AM_KEY_CONFIGURED=false + ^---------------------^ SC2034 (warning): SHARP_AM_KEY_CONFIGURED appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 1897: + UFM_SECURED_PROFILE_JSON='{"applicable":false,"status":"not_applicable","profile":"Secured Bare Metal Cloud","verification":"not applicable to standalone","requiredControls":[]}' + ^----------------------^ SC2034 (warning): UFM_SECURED_PROFILE_JSON appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 1903: +TOPOLOGY_CONF="" +^-----------^ SC2034 (warning): TOPOLOGY_CONF appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 2070: +HEALTH_CHECK_PROGRAM="" +^------------------^ SC2034 (warning): HEALTH_CHECK_PROGRAM appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 2071: +HEALTH_CHECK_INTERVAL="" +^-------------------^ SC2034 (warning): HEALTH_CHECK_INTERVAL appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 2072: +HEALTH_CHECK_CONFIGURED="false" +^---------------------^ SC2034 (warning): HEALTH_CHECK_CONFIGURED appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 2073: +DCGM_SLURM="false" +^--------^ SC2034 (warning): DCGM_SLURM appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 2074: +PROLOG_RUNTIME_SEC="n/a" +^----------------^ SC2034 (warning): PROLOG_RUNTIME_SEC appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 2075: +PROLOG_FAST="n/a" +^---------^ SC2034 (warning): PROLOG_FAST appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 2076: +NHC_INSTALLED="false" +^-----------^ SC2034 (warning): NHC_INSTALLED appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 2077: +AUTO_REMEDIATION_CONFIGURED="false" +^-------------------------^ SC2034 (warning): AUTO_REMEDIATION_CONFIGURED appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 2101: + [[ "$DCGM_INSTALLED" == "true" ]] && print_warn "DCGM binary found but service NOT active on compute node" \ + ^-- SC2015 (info): Note that A && B || C is not if-then-else. C may run when A is true. + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 2107: + DCGM_HEALTH_WATCHES_ENABLED="true" + ^-------------------------^ SC2034 (warning): DCGM_HEALTH_WATCHES_ENABLED appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 2135: + && print_info "DCGM service: Active on head node" \ + ^-- SC2015 (info): Note that A && B || C is not if-then-else. C may run when A is true. + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 2179: +[[ "$PROMETHEUS_DETECTED" == "true" ]] && print_info "Prometheus: detected (port 9090 or systemd active)" || print_detail "Prometheus: not detected" + ^-- SC2015 (info): Note that A && B || C is not if-then-else. C may run when A is true. + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 2180: +[[ "$DCGM_EXPORTER_DETECTED" == "true" ]] && print_info "dcgm-exporter: detected (port 9400 or systemd active)" || print_detail "dcgm-exporter: not detected" + ^-- SC2015 (info): Note that A && B || C is not if-then-else. C may run when A is true. + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 2181: +[[ "$NODE_EXPORTER_DETECTED" == "true" ]] && print_info "node-exporter: detected (port 9100 or systemd active)" || print_detail "node-exporter: not detected" + ^-- SC2015 (info): Note that A && B || C is not if-then-else. C may run when A is true. + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 2182: +[[ "$GRAFANA_DETECTED" == "true" ]] && print_info "Grafana: detected (port 3000 or systemd active)" || print_detail "Grafana: not detected" + ^-- SC2015 (info): Note that A && B || C is not if-then-else. C may run when A is true. + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 2195: + PASSWORDLESS_SSH="enabled" + ^--------------^ SC2034 (warning): PASSWORDLESS_SSH appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 2205: +SSH_TO_COMPUTE="false" +^------------^ SC2034 (warning): SSH_TO_COMPUTE appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 2206: +FIRST_COMPUTE_NODE="$(hostname)" +^----------------^ SC2034 (warning): FIRST_COMPUTE_NODE appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 2209: +ACCOUNTING_STORAGE="none" +^----------------^ SC2034 (warning): ACCOUNTING_STORAGE appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 2210: +SACCT_AVAILABLE="false" +^-------------^ SC2034 (warning): SACCT_AVAILABLE appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 2238: +ESSENTIAL_TOOLS_JSON="{$(IFS=,; echo "${TOOLS_JSON_ENTRIES[*]}")}" +^------------------^ SC2034 (warning): ESSENTIAL_TOOLS_JSON appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 2246: + IPMITOOL_INSTALLED="true" + ^----------------^ SC2034 (warning): IPMITOOL_INSTALLED appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 2268: + IPMI_EXPOSED="true" + ^----------^ SC2034 (warning): IPMI_EXPOSED appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 2286: + SUDO_AVAILABLE="true" + ^------------^ SC2034 (warning): SUDO_AVAILABLE appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 2292: +SLURM_CMDS_OK="false" +^-----------^ SC2034 (warning): SLURM_CMDS_OK appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 2293: +declare -A SLURM_CMD_AVAIL=() + ^-------------^ SC2034 (warning): SLURM_CMD_AVAIL appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 2302: +[[ "$USERADD_AVAILABLE" == "true" ]] && print_info "useradd: $(command -v useradd)" || print_warn "useradd: not found" + ^-- SC2015 (info): Note that A && B || C is not if-then-else. C may run when A is true. + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 2303: +[[ "$GROUPADD_AVAILABLE" == "true" ]] && print_info "groupadd: $(command -v groupadd)" || print_warn "groupadd: not found" + ^-- SC2015 (info): Note that A && B || C is not if-then-else. C may run when A is true. + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 2332: +DEF_CPUS_PER_TASK="1" +^---------------^ SC2034 (warning): DEF_CPUS_PER_TASK appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 2333: +DEF_MEM_PER_CPU=0 +^-------------^ SC2034 (warning): DEF_MEM_PER_CPU appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 2334: +DEF_MEM_PER_GPU=0 +^-------------^ SC2034 (warning): DEF_MEM_PER_GPU appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 2335: +DEF_MEM_PER_NODE=0 +^--------------^ SC2034 (warning): DEF_MEM_PER_NODE appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 2336: +MAX_MEM_PER_CPU=0 +^-------------^ SC2034 (warning): MAX_MEM_PER_CPU appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 2337: +MAX_MEM_PER_NODE=0 +^--------------^ SC2034 (warning): MAX_MEM_PER_NODE appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 2338: +CPU_FREQ_DEF="" +^----------^ SC2034 (warning): CPU_FREQ_DEF appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 2339: +CPU_FREQ_GOV="" +^----------^ SC2034 (warning): CPU_FREQ_GOV appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 2340: +TASK_PLUGIN="n/a" +^---------^ SC2034 (warning): TASK_PLUGIN appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 2341: +PROCTRACK_TYPE="n/a" +^------------^ SC2034 (warning): PROCTRACK_TYPE appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 2426: +DCGM_EXPORTER_PRESENT="${DCGM_EXPORTER_DETECTED:-unknown}" +^-------------------^ SC2034 (warning): DCGM_EXPORTER_PRESENT appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 2427: +SECURITY_VERSION_AUDIT_JSON=$(build_security_version_audit \ +^-------------------------^ SC2034 (warning): SECURITY_VERSION_AUDIT_JSON appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/cluster-audit-standalone.sh line 2440: +AUDIT_TYPE="standalone" +^--------^ SC2034 (warning): AUDIT_TYPE appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/container-check.sh line 43: + if [[ $? -eq 0 ]]; then + ^-- SC2181 (style): Check exit code directly with e.g. 'if mycmd;', not indirectly with $?. + + +In cmax/scripts/1-audit/host-check.sh line 16: + . /etc/os-release + ^-------------^ SC1091 (info): Not following: /etc/os-release: openBinaryFile: does not exist (No such file or directory) + + +In cmax/scripts/1-audit/host-check.sh line 461: + WORKER_VIRT_DETECTION=cpuinfo-hypervisor + ^----------------^ SC2100 (warning): Use $((..)) for arithmetics, e.g. i=$((i - 2)) + + +In cmax/scripts/1-audit/host-check.sh line 933: + acs_nicmap[${BASH_REMATCH[1]}]="${BASH_REMATCH[2]}" + ^----------------^ SC2004 (style): $/${} is unnecessary on arithmetic variables. + + +In cmax/scripts/1-audit/host-check.sh line 971: + $acs_tmo ib_write_bw -d "$acs_dev" -p 18796 $1 -s 65536 -n 100 -F >"$sl" 2>&1 & + ^-- SC2086 (info): Double quote to prevent globbing and word splitting. + +Did you mean: + $acs_tmo ib_write_bw -d "$acs_dev" -p 18796 "$1" -s 65536 -n 100 -F >"$sl" 2>&1 & + + +In cmax/scripts/1-audit/host-check.sh line 973: + $acs_tmo ib_write_bw -d "$acs_dev" -p 18796 $1 -s 65536 -n 100 -F 127.0.0.1 >"$cl" 2>&1 + ^-- SC2086 (info): Double quote to prevent globbing and word splitting. + +Did you mean: + $acs_tmo ib_write_bw -d "$acs_dev" -p 18796 "$1" -s 65536 -n 100 -F 127.0.0.1 >"$cl" 2>&1 + + +In cmax/scripts/1-audit/host-check.sh line 1092: + echo "WORKER_RDMA_DRIVERS=$(echo $_rdma_drivers | tr ' ' '\n' | grep . | sort -u | paste -sd, -)" + ^------------^ SC2086 (info): Double quote to prevent globbing and word splitting. + +Did you mean: + echo "WORKER_RDMA_DRIVERS=$(echo "$_rdma_drivers" | tr ' ' '\n' | grep . | sort -u | paste -sd, -)" + + +In cmax/scripts/1-audit/host-check.sh line 1093: + echo "WORKER_RDMA_LINK_LAYERS=$(echo $_rdma_layers | tr ' ' '\n' | grep . | sort -u | paste -sd, -)" + ^-----------^ SC2086 (info): Double quote to prevent globbing and word splitting. + +Did you mean: + echo "WORKER_RDMA_LINK_LAYERS=$(echo "$_rdma_layers" | tr ' ' '\n' | grep . | sort -u | paste -sd, -)" + + +In cmax/scripts/1-audit/host-check.sh line 1190: + ROCM_DIR=$(ls -d /opt/rocm-* /opt/rocm 2>/dev/null | head -1) + ^-- SC2012 (info): Use find instead of ls to better handle non-alphanumeric filenames. + + +In cmax/scripts/1-audit/host-check.sh line 1304: +_render_node=$(ls /dev/dri/renderD* 2>/dev/null | head -1) + ^-- SC2012 (info): Use find instead of ls to better handle non-alphanumeric filenames. + + +In cmax/scripts/1-audit/host-check.sh line 1324: + _rvs_conf=$(ls -d /opt/rocm*/share/rocm-validation-suite/conf 2>/dev/null | head -1) + ^-- SC2012 (info): Use find instead of ls to better handle non-alphanumeric filenames. + + +In cmax/scripts/1-audit/host-check.sh line 1878: + WORKER_NHC_PATH=$(ls /usr/sbin/nhc /usr/local/sbin/nhc 2>/dev/null | head -1) + ^-- SC2012 (info): Use find instead of ls to better handle non-alphanumeric filenames. + + +In cmax/scripts/1-audit/host-check.sh line 1983: +IB_DEVS=$(ls /sys/class/infiniband/ 2>/dev/null | grep -v bond | tr '\n' ',' | sed 's/,$//') + ^-- SC2010 (warning): Don't use ls | grep. Use a glob or a for loop with a condition to allow non-alphanumeric filenames. + + +In cmax/scripts/1-audit/host-check.sh line 1986: +for dev in $(ls /sys/class/infiniband/ 2>/dev/null | grep -v bond); do + ^-- SC2010 (warning): Don't use ls | grep. Use a glob or a for loop with a condition to allow non-alphanumeric filenames. + + +In cmax/scripts/1-audit/host-check.sh line 1987: + rate=$(cat /sys/class/infiniband/${dev}/ports/1/rate 2>/dev/null || echo unknown) + ^----^ SC2086 (info): Double quote to prevent globbing and word splitting. + +Did you mean: + rate=$(cat /sys/class/infiniband/"${dev}"/ports/1/rate 2>/dev/null || echo unknown) + + +In cmax/scripts/1-audit/host-check.sh line 1988: + state=$(cat /sys/class/infiniband/${dev}/ports/1/state 2>/dev/null | grep -oP '\d+: \K\w+' || echo unknown) + ^----^ SC2086 (info): Double quote to prevent globbing and word splitting. + +Did you mean: + state=$(cat /sys/class/infiniband/"${dev}"/ports/1/state 2>/dev/null | grep -oP '\d+: \K\w+' || echo unknown) + + +In cmax/scripts/1-audit/host-check.sh line 1989: + link_layer=$(cat /sys/class/infiniband/${dev}/ports/1/link_layer 2>/dev/null || echo unknown) + ^----^ SC2086 (info): Double quote to prevent globbing and word splitting. + +Did you mean: + link_layer=$(cat /sys/class/infiniband/"${dev}"/ports/1/link_layer 2>/dev/null || echo unknown) + + +In cmax/scripts/1-audit/host-check.sh line 1990: + pci_vendor=$(cat /sys/class/infiniband/${dev}/device/vendor 2>/dev/null || echo unknown) + ^----^ SC2086 (info): Double quote to prevent globbing and word splitting. + +Did you mean: + pci_vendor=$(cat /sys/class/infiniband/"${dev}"/device/vendor 2>/dev/null || echo unknown) + + +In cmax/scripts/1-audit/host-check.sh line 1991: + pci_device=$(cat /sys/class/infiniband/${dev}/device/device 2>/dev/null || echo unknown) + ^----^ SC2086 (info): Double quote to prevent globbing and word splitting. + +Did you mean: + pci_device=$(cat /sys/class/infiniband/"${dev}"/device/device 2>/dev/null || echo unknown) + + +In cmax/scripts/1-audit/host-check.sh line 1992: + hca_type=$(cat /sys/class/infiniband/${dev}/hca_type 2>/dev/null || echo unknown) + ^----^ SC2086 (info): Double quote to prevent globbing and word splitting. + +Did you mean: + hca_type=$(cat /sys/class/infiniband/"${dev}"/hca_type 2>/dev/null || echo unknown) + + +In cmax/scripts/1-audit/host-check.sh line 1993: + fw_ver=$(cat /sys/class/infiniband/${dev}/fw_ver 2>/dev/null || echo unknown) + ^----^ SC2086 (info): Double quote to prevent globbing and word splitting. + +Did you mean: + fw_ver=$(cat /sys/class/infiniband/"${dev}"/fw_ver 2>/dev/null || echo unknown) + + +In cmax/scripts/1-audit/host-check.sh line 2009: + for pf in /sys/class/infiniband/${dev}/ports/1/pkeys/*; do + ^----^ SC2231 (info): Quote expansions in this for loop glob to prevent wordsplitting, e.g. "$dir"/*.txt . + + +In cmax/scripts/1-audit/host-check.sh line 2022: + if grep -rl "RoCE v2\|roce_v2\|RoCEv2" /sys/class/infiniband/${dev}/ports/1/gid_attrs/types/ 2>/dev/null | grep -q .; then + ^----^ SC2086 (info): Double quote to prevent globbing and word splitting. + +Did you mean: + if grep -rl "RoCE v2\|roce_v2\|RoCEv2" /sys/class/infiniband/"${dev}"/ports/1/gid_attrs/types/ 2>/dev/null | grep -q .; then + + +In cmax/scripts/1-audit/host-check.sh line 2167: + echo "WORKER_BLKDEV_${NAME}=${TYPE}|${SIZE}|${MOUNTPOINT}|${FSTYPE}|${TRAN}" + ^-----^ SC2153 (info): Possible misspelling: NAME may not be assigned. Did you mean name? + ^-------^ SC2153 (info): Possible misspelling: FSTYPE may not be assigned. Did you mean fstype? + + +In cmax/scripts/1-audit/k8s-control.sh line 9: + RUNNER_NAME="${1:?runner name required}" + ^---------^ SC2034 (warning): RUNNER_NAME appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/k8s-control.sh line 1806: + local rdma_claim_template="$(k8s_detect_dra_rdma_claim_template)" + ^-----------------^ SC2155 (warning): Declare and assign separately to avoid masking return values. + + +In cmax/scripts/1-audit/monitoring-k8s.sh line 395: + PROM_PODS=$(kubectl get pods -n "$ns" --no-headers 2>/dev/null | grep -E "prometheus-[^k]" | wc -l | tr -d ' ') + ^-----------------------^ SC2126 (style): Consider using 'grep -c' instead of 'grep|wc -l'. + + +In cmax/scripts/1-audit/monitoring-k8s.sh line 723: + GFD_LABELS="true" + ^--------^ SC2034 (warning): GFD_LABELS appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/monitoring-k8s.sh line 778: + DCGM_HEALTH_ENABLED="true" + ^-----------------^ SC2034 (warning): DCGM_HEALTH_ENABLED appears unused. Verify use (or export if used externally). + + +In cmax/scripts/1-audit/run.sh line 43: +echo "Audit script: ${AUDIT_SCRIPT#$REPO_ROOT/}" + ^--------^ SC2295 (info): Expansions inside ${..} need to be quoted separately, otherwise they match as patterns. + +Did you mean: +echo "Audit script: ${AUDIT_SCRIPT#"$REPO_ROOT"/}" + +For more information: + https://www.shellcheck.net/wiki/SC2148 -- Tips depend on target shell and y... + https://www.shellcheck.net/wiki/SC2010 -- Don't use ls | grep. Use a glob o... + https://www.shellcheck.net/wiki/SC2034 -- ACCOUNTING_STORAGE appears unused... diff --git a/reports/static-analysis/summary.txt b/reports/static-analysis/summary.txt new file mode 100644 index 0000000..b02af78 --- /dev/null +++ b/reports/static-analysis/summary.txt @@ -0,0 +1,8 @@ +ClusterMAX static-analysis summary (report-only) + +analysis-bandit exit=1 lines=499 +analysis-mypy exit=1 lines=81 +analysis-ruff exit=1 lines=2515 +analysis-ruff-format exit=1 lines=3932 +analysis-shellcheck exit=123 lines=867 +