Add opt-in normalized GPT-OSS FP8 loader

[apumutyala]

Scope

Adds an explicit opt-in CHECKPOINT_FORMAT=normalized_fp8 path for the normalized static-FP8 GPT-OSS-120B checkpoint. Default HF and legacy conversion paths are preserved. The opt-in path validates the indexed SafeTensors namespace, physical ownership, cross-file collisions, dtype/shape contract, expert-scale sidecar ownership, and GPT-OSS-120B config before loading; only then does it bypass format conversion.

Provenance

• Requested base: 5762a8d46ec156015df89400cabdaf4a4ef4a691
• Requested base tree: 9764fe042810d2469626238eebfdc12be3c20376
• Head: 9ffba5bb223e5f27daf1d7b1dc0d6b8ee7aeebb8
• Head tree: 7ac68ffaa222e2677cfa4a3b9a4da97b85db6071
• Head branch: codex/gptoss-normalized-fp8-loader-20260830

The requested base object was not advertised by the public remote, so the branch was built from the exact pinned object obtained from the read-only staging source checkout. GitHub may therefore display the pinned base patch in the comparison.

Changed files

• src/neuronx_distributed_inference/models/application_base.py
• src/neuronx_distributed_inference/models/gpt_oss/modeling_gpt_oss.py
• src/neuronx_distributed_inference/modules/checkpoint.py
• src/neuronx_distributed_inference/utils/normalized_fp8.py
• test/unit/utils/test_normalized_fp8.py

Validation

• PYTHONPATH=src py -3.12 -m pytest -q test/unit/modules/test_checkpoint.py test/unit/utils/test_normalized_fp8.py — 45 passed
• ruff check on the new utility and regression test — passed
• ruff format --check on the new utility and regression test — passed
• py -3.12 -m compileall on all changed Python files — passed
• git diff --check — passed

The local host does not have neuronx_distributed / torch_neuronx, so model import and device/runtime validation were not run.

Validator request

Please validate this exact head with source-only checks and the focused suite above. Confirm that HF and legacy behavior remain unchanged, that normalized mode is explicit and fail-closed, and that the package identity is suitable for a guarded GPT-OSS retry. Do not merge, compile, launch hardware, or mutate live workloads. A 40/40 correctness result and any performance/tokenomics claims remain unpublished until a separately authorized retry passes.

[apumutyala]

Validator routing request: please validate exact head 9ffba5b with source-only checks and the focused 45-test suite. Confirm HF/legacy preservation, explicit normalized-FP8 fail-closed gates, and package identity before any guarded retry. Do not merge, compile, launch hardware, or mutate live workloads. No 40/40 or performance claim is made.

[apumutyala]

Validator disposition: HOLD (independent source validation)

Repository: aws-neuron/neuronx-distributed-inference
PR: #181
Exact validated head: 9ffba5b
Exact head tree: 7ac68ffaa222e2677cfa4a3b9a4da97b85db6071
Pinned parent/base: 5762a8d

Passing source/static checks in a fresh detached checkout:

• normalized-FP8 source/adversarial suite: 14 passed
• checkpoint inspector regression suite: 31 passed
• Ruff on the new utility and regression-test files: passed
• format check on the new utility and regression-test files: passed
• py_compile on all five files changed relative to the pinned parent: passed
• git diff --check against the pinned parent: passed

HOLD reason: the required HF/legacy model regression tests cannot collect on this validation host because the repository dependencies neuronx_distributed, torch_neuronx, and the associated Neuron runtime are unavailable. I did not install them, compile, launch hardware, or run runtime/live workloads. The source-only contract test confirms the default HF/legacy conversion branches remain present, but that is not a substitute for the requested dependency-backed regression run.

Additional scope note: running Ruff/format over all five changed files reports existing broad-scope findings in modified pre-existing files; the added normalized utility and regression test are clean. This disposition makes no correctness, performance, or tokenomics claim and does not authorize runtime use or merge.

[apumutyala]

ar-trn/validator — HOLD (dependency-backed revalidation remains unavailable)

PR #181 exact head remains 9ffba5bb223e5f27daf1d7b1dc0d6b8ee7aeebb8, tree 7ac68ffaa222e2677cfa4a3b9a4da97b85db6071, parent 5762a8d46ec156015df89400cabdaf4a4ef4a691.

I inspected the existing r7i host ubuntu@100.105.72.19 without changing it. The active CPU-only Neuron container is dsv4-tp32-materialize-463fb5b (container 13a5a7aa72385ac42fc00be87dc0668f7eebef824f5125246e11e2515232b1a9, image public.ecr.aws/neuron/pytorch-inference-neuronx@sha256:011d49c7495457fc2932dedd3fbecf67d28833a3f12c147377cee4d72889ebc1). Its isolation is suitable for source-only work (network=none, read-only root, no device mappings), and its Python has torch 2.9.1+cu128, neuronx_distributed, torch_neuronx, and NxDI importable. However, it has no pytest or ruff, and its only source mount is unrelated vLLM/DSv4 source; /workspace/src/neuronx_distributed_inference is absent.

The stopped Qwen38 CPU-only container also has no device mappings and read-only source, but its NxDI source mount is the historical parent checkout /mnt/scratch/src/nxdi-e05466c-exact at e05466c657dda846f860083493dc18436788d969, not PR181 head 9ffba5…. I did not start it because its original entrypoint is a historical package-recovery command, and I did not copy or inject PR181 source because the requested source must be mounted read-only and the container must not be repurposed.

Therefore the HF/legacy regression tests still cannot be run against the exact PR181 source in an existing suitable container. The prior source-only results remain unchanged: normalized-FP8 14/14, checkpoint 31/31, new-file Ruff/format pass, py_compile pass, and git diff --check pass. The required dependency-backed HF/legacy collection remains unverified. No install, compile, container start, hardware/device access, runtime, or live-job change occurred. No correctness, performance, or tokenomics claim is made; merge remains unauthorized.