Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Captured static-analysis reports are verbatim tool output. Keep them exactly
# as produced (aligned columns, diff blank lines) — exempt from whitespace checks.
reports/static-analysis/*.txt -whitespace
27 changes: 27 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -316,13 +316,40 @@ evidence. Use `-vvv` to include recommended remediation and labeled reference
URLs. Use `--command raw` to print the saved collector log. Use
`--no-interactive` to print the review and exit.

## Develop with Nix

The repository ships a [Nix](https://nixos.org) flake for a reproducible
development environment, a hermetic build of `cmax`, OCI container images, and
report-only static analysers. It is optional: it does not change the pip install
above. If you are new to Nix, [`nix/README.md`](nix/README.md) has a short
introduction, install steps, and video walkthroughs.

Quickstart, from the repo root (flakes must be enabled — see
[`nix/README.md`](nix/README.md)):

```
nix develop # dev shell (Python, ruff, mypy, bandit, shellcheck); type 'cmax-help'
nix build .#cmax # build the CLI -> ./result/bin/cmax
nix run .#test # run the pytest suite
nix build .#analysis # run all static analysers; cat result/summary.txt
nix build .#oci-cmax # OCI image for the host arch; docker load < result
nix flake check # package build + CLI smoke check + nix formatting
```

The full target list is in the header comment of `flake.nix` and in
[`nix/README.md`](nix/README.md).

> Flakes only see git-tracked files. After adding or editing files under `nix/`,
> `git add` them before `nix build` / `nix develop`.

## Repository contents

| Path | Contents |
|---|---|
| `cmax/` | This directory contains the command code and `cmax.yaml` configuration. |
| `cmax/scripts/1-audit/` | This directory contains the scripts that run an audit. |
| `tests/audit/` | This directory contains all audit tests, fixtures, and test helpers. |
| `flake.nix`, `nix/` | The Nix flake and its modules. See [`nix/README.md`](nix/README.md). |

This release excludes provider results, internal notes, the private dashboard,
benchmark implementations, bundled data, and database code.
Expand Down
185 changes: 185 additions & 0 deletions STATIC_ANALYSIS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,185 @@
# ClusterMAX static-analysis review

First pass over the `cmax` codebase with the flake's report-only analysers.
This document triages the findings by priority so a maintainer can act on the
signal and skip the noise. **No code is changed by this review** — it is a
catalogue and a recommended order of work.

- **Generated by**: `nix build .#analysis` (branch `static-analysis-review`, off `nix`).
- **Date**: 2026-08-31.
- **Tool versions** (pinned by the flake's `nixpkgs`): ruff 0.16.4, mypy 2.1.0,
bandit 1.9.4, shellcheck 0.11.0. Python target 3.10 (interpreter 3.12.14).
- **Raw reports**: [`reports/static-analysis/`](reports/static-analysis/) — one
`*.txt` per tool plus `summary.txt`.
- **Tooling docs**: [`nix/README.md`](nix/README.md).

Analysis is report-only: the analysers always succeed and never gate a build.
See "Reproduce & fix" at the end.

## Summary

| Tool | Findings | P1 security | P2 correctness | P3 maintainability | P4 style |
|---|---:|---:|---:|---:|---:|
| bandit | 41 | 3 | – | – | – (38 accepted) |
| ruff (lint) | 224 | – | 9 | 58 | 157 |
| mypy | 75 | – | 75 | – | – |
| ruff-format | 11 files | – | – | – | 11 files |
| shellcheck | 279 | – | ~24 | – | ~255 |

Recommended order: **P1 → P2 → P3 → P4**. P4 (line length + formatting) is the
largest bucket but is mechanical and auto-fixable; do it last, or first as a
one-shot so later diffs stay clean.

---

## P1 — Security (bandit)

41 bandit findings: **2 Medium**, 39 Low. Only **3** need action; the rest are
false positives or accepted by design (documented below so they are not
re-triaged every run).

### Act on these

| Test | Sev | Location | Issue | Recommended fix |
|---|---|---|---|---|
| `B314` | Medium | `cmax/scripts/1-audit/checks/platform_config.py:864` | `xml.etree.ElementTree.fromstring` on tool output | Parse with `defusedxml`, or call `defusedxml.defuse_stdlib()` once at startup. |
| `B310` | Medium | `cmax/minimum_sync.py:214` | `urllib` `urlopen` allows `file:`/custom schemes | Validate the URL scheme is `https` before opening. |
| `B405` | Low | `cmax/scripts/1-audit/checks/platform_config.py:66` | `import xml.etree.ElementTree` | Same as `B314` — switch to `defusedxml`. |

The two XML findings are the same root cause: XML from cluster tools is parsed
with the stdlib parser, which is vulnerable to entity-expansion attacks. Fixing
the import and the parse call clears all three.

### Accepted — no action

| Test | Count | Why accepted |
|---|---:|---|
| `B603` subprocess-without-shell / `B404` import-subprocess / `B607` partial-path | 28 | By design: `cmax` is an audit tool that shells out to cluster commands (`kubectl`, `srun`, `nvidia-smi`, …). Calls use argument lists, not `shell=True`. |
| `B105`/`B106` hardcoded-password | 5 | All false positives: the flagged strings are status/message values, not secrets — e.g. `'pass'`, `'0'`, `'kubelet CPU Manager policy check passed'` (`cmax/security.py:21`, `cmax/audit_report.py:23`, `cmax/scripts/1-audit/security_version_audit.py:814,827`). |
| `B101` assert-used | 4 | Non-critical asserts in `cmax/progress.py`. Harmless; convert to explicit checks only if these paths run under `python -O`. |
| `B110` try-except-pass | 1 | `cmax/scripts/1-audit/checks/system/hbm_memory_exposure.py:711` — a best-effort probe. Fine; add a comment if you want the intent recorded. |

---

## P2 — Correctness (ruff `F`/`B` + mypy)

Real defects or latent bugs — code that is wrong or can fail at runtime.

### ruff bug-class rules (9)

| Rule | Count | Meaning | Example |
|---|---:|---|---|
| `F401` | 5 | Unused import | `cmax/banner.py:43`, `cmax/security.py:7,12` |
| `B023` | 1 | Function uses a loop variable not bound in the loop (classic closure bug) | `cmax/progress.py:155` |
| `B905` | 1 | `zip()` without `strict=` — silently drops items on length mismatch | — |
| `B007` | 1 | Loop control variable not used in the body | — |
| `F841` | 1 | Local variable assigned but never used | — |

`B023` and `B905` are the two worth reading closely — both can produce silently
wrong results. `F401`/`F841` are safe deletions (`ruff check --fix` handles them).

### mypy (75 errors)

Non-strict, `ignore_missing_imports`, so these are genuine local type problems —
mostly attribute/index access on values typed `Any | dict | None`, i.e. **missing
`None` handling** that can raise at runtime.

| Category | Count | Category | Count |
|---|---:|---|---:|
| `assignment` | 25 | `attr-defined` | 4 |
| `arg-type` | 18 | `var-annotated` | 2 |
| `index` | 10 | `return-value` | 2 |
| `union-attr` | 6 | other | 3 |
| `operator` | 5 | | |

Most affected: `cmax/minimum_refresh.py` (13), `cmax/progress.py` (11),
`cmax/security.py` (4). Representative: `cmax/scripts/1-audit/gpu_profiles.py:74`
— `Item "None" of "... | None" has no attribute "get"` (a `.get()` on a value
that can be `None`). Fixing the `union-attr`/`index` cases first removes the
runtime-crash risk; `assignment`/`arg-type` are mostly annotations to tighten.

---

## P3 — Maintainability (ruff `SIM`/`UP`/`I001`/`E741`/`E402`/`C4`)

58 findings. Not bugs — clarity, modern syntax, and import hygiene. Many are
auto-fixable.

| Group | Count | What | Auto-fix |
|---|---:|---|---|
| `I001` | 13 | Unsorted / unformatted imports | `ruff check --fix` |
| `UP035`/`UP045`/`UP037`/`UP022` | 21 | Deprecated typing imports and old-style syntax (pyupgrade) | mostly `--fix` |
| `SIM105`/`SIM115`/`SIM114`/… | 17 | Simplifiable code (`contextlib.suppress`, context managers, merged branches) | some `--fix` |
| `E741` | 3 | Ambiguous names (`l`, `I`, `O`) | manual |
| `E402` | 2 | Module import not at top of file | manual |
| `C408`/`C420` | 2 | Unnecessary `dict()` / dict-comprehension rewrite | `--fix` |

`SIM115` (open a file without a context manager) is worth a manual look — it can
leak file handles. The rest are cosmetic-to-minor.

---

## P4 — Style (ruff `E501` + ruff-format)

Largest bucket, lowest priority, fully mechanical.

- **`E501` line-too-long — 157**. All are length-only. Either raise the limit or
reflow. Concentrated in `cmax/security.py` (28) and `cmax/progress.py` (7).
- **ruff-format — 11 files** would be reformatted (3955-line diff): `audit_report.py`,
`audit_review.py`, `banner.py`, `cli.py`, `criteria_links.py`, `minimum_refresh.py`,
`minimum_sync.py`, `progress.py`, `report_style.py`, `security.py`,
`target_selection.py`.

One command fixes both: `ruff format cmax`. Doing this **once, in its own commit**
keeps every later review diff readable. `[tool.ruff] line-length = 100` in
`pyproject.toml` sets the width.

---

## Shell scripts (shellcheck)

279 findings across 10 audit scripts: **1 error, 206 warning, 67 info, 5 style**.
Concentrated in `cluster-audit-standalone.sh` (133), `cluster-audit-slurm.sh` (89),
`host-check.sh` (23).

### Act on these

| Code | Count | Meaning | Action |
|---|---:|---|---|
| `SC2148` | 1 (error) | No shebang / shell directive | `cmax/scripts/1-audit/audit-common.sh` is `source`d, so it has no shebang; add `# shellcheck shell=bash` at the top to set the dialect. |
| `SC2086` | 15 | Unquoted expansion — word-splitting / globbing | Quote the variables (real correctness risk with spaces/globs). |
| `SC2015` | 35 | `A && B || C` is not if/then/else | Review each; `C` runs when `B` fails, which is often not intended. |
| `SC2153`/`SC2010`/`SC2012` | ~13 | Possible misspelled var / parsing `ls` | Check the var names; prefer globs over parsing `ls`. |

### Mostly noise

- **`SC2034` unused-variable — 200** (72% of all findings). Many are collected
keys or documentation values referenced indirectly. Triage per script; silence
intentional ones with `# shellcheck disable=SC2034` rather than deleting.

Fixing the single error and the `SC2086` quoting issues first gives the best
return; `SC2034` can be swept later, script by script.

---

## Reproduce & fix

```
# Regenerate every report (writes result/summary.txt + per-tool report.txt):
nix build .#analysis && cat result/summary.txt

# A single tool:
nix build .#analysis-bandit # or -ruff, -ruff-format, -mypy, -shellcheck

# In the dev shell, iterate and apply the safe auto-fixes:
nix develop
cmax-lint # ruff check cmax
cmax-fmt # ruff format cmax (fixes all of P4)
cmax-types # mypy cmax
cmax-sec # bandit -r cmax
cmax-shellcheck # shellcheck the audit .sh scripts
```

Suggested sequence: land the three P1 security fixes, then the P2 `ruff --fix`
deletions and the `B023`/`B905`/`SIM115` reads, then a single `ruff format`
commit for P4, and finally sweep P3 and the shell warnings per file.
61 changes: 61 additions & 0 deletions flake.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

71 changes: 71 additions & 0 deletions flake.nix
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
#
# flake.nix — ClusterMAX
#
# Thin orchestrator. Every concern lives under ./nix/ and is wired up here.
# See ./nix/default.nix for the per-system aggregator.
#
# Targets:
# nix develop # dev shell (ruff, mypy, bandit, shellcheck, pytest)
# nix build .#cmax # build the cmax CLI
# nix run .#cmax -- --help # run the CLI
# nix build .#oci-cmax # OCI image for the host arch (amd64 or aarch64)
# nix build .#analysis # run ALL static-analysis reports (report-only)
# nix build .#analysis-ruff # ruff lint report
# nix build .#analysis-ruff-format # ruff format --check report
# nix build .#analysis-mypy # mypy type report
# nix build .#analysis-bandit # bandit security report
# nix build .#analysis-shellcheck # shellcheck report for the audit .sh scripts
# nix run .#test # run the pytest suite (uses the host toolchain)
# nix flake check # package build + CLI smoke + nix formatting (gates)
# nix fmt # format the .nix files
#
# Static analysis is report-only: the analysis-* targets always succeed and write
# their findings to $out/report.txt.
#
# The pytest suite runs via `nix run .#test`, not `nix flake check`: its command
# stubs hard-code /bin/bash and /bin/cat, which do not exist in the hermetic Nix
# build sandbox. The gates that DO run under `nix flake check` are the package
# build (its installCheck smoke-tests the CLI + resources) and nix formatting.
#
# Containers are native per-system: build .#oci-cmax on an x86_64 host for the
# amd64 image, and on an aarch64 host (or through binfmt/qemu) for the arm64 image.
#
{
description = "ClusterMAX — GPU cluster audit and security CLI";

inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
flake-utils.url = "github:numtide/flake-utils";
};

outputs =
{
self,
nixpkgs,
flake-utils,
}:
flake-utils.lib.eachSystem [ "x86_64-linux" "aarch64-linux" ] (
system:
let
pkgs = import nixpkgs { inherit system; };
lib = nixpkgs.lib;

aggregator = import ./nix {
inherit pkgs lib;
src = ./.;
};
in
{
inherit (aggregator)
packages
devShells
checks
apps
formatter
;
}
)
// {
overlays.default = import ./nix/overlays.nix { inherit self; };
};
}
Loading