Skip to content

feat(model): reject mismatched HF smoke runtime environments - #179

Merged
sanskarIN merged 9 commits into
mainfrom
feat/model-hf-runtime-parity-gate-20261030
Oct 30, 2026
Merged

sanskarIN merged 9 commits into
mainfrom
feat/model-hf-runtime-parity-gate-20261030

Conversation

@sanskarIN

Copy link
Copy Markdown
Owner

Motivation

Matching synthetic fixture hashes and base checkpoint bytes do not establish that base and LoRA generation benchmarks used comparable environments. CPU/CUDA, BF16/FP32, operating system or PyTorch/Transformers version changes can distort apparent latency deltas.

Changes

  • Bump local optional Hugging Face FIM smoke protocol to lineloom-hf-synthetic-smoke-v6 and the offline comparison protocol to v5.
  • Attach a narrowly allowlisted eight-field runtime profile to opt-in local real smoke summaries: device, precision, operating-system family, Python/PyTorch/Transformers versions, CUDA runtime version and HIP runtime version.
  • Reject missing, extra, malformed or source-bearing metadata fields. Require byte-for-byte identical validated profiles on base and adapter runs; no comparison output on mismatches.
  • Keep hostnames, full environment variables, serial numbers, checkpoint/model names, paths, raw code and generated completions out of public reports. Output only runtimeProfileComparable and checked field count in source-free comparison summaries.
  • Add deterministic offline tests for missing fields, types, changed runtimes, CPU-vs-BF16 rejection and matching reviewed CUDA metadata.
  • Update model/train/README.md and model/EVALUATION_PROTOCOL.md with v6 compatibility, privacy and measured-performance limitations.

Validation and boundaries

  • Run locked Windows/macOS/Linux CI, independent Linux extension-host test, dependency-security audit, and VSIX install/upgrade checks before merge.
  • No GPU experiment, checkpoint download, training corpus collection, new model, live inference, new extension release or marketplace upload is performed.
  • Matching profile is not verification of identical physical GPU, drivers, temperatures, process load, execution order, code correctness or usable editor latency. All real-model evidence remains outstanding.

@sanskarIN
sanskarIN merged commit a18bea2 into main Oct 30, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant