From 29f401a859ba70e04a44810702fecec13a0f23a3 Mon Sep 17 00:00:00 2001 From: Amit Bahree Date: Sun, 13 Sep 2026 17:11:18 -0700 Subject: [PATCH 1/4] MEAP review feedback - Ch1 and 2 --- ACCELERATORS.md | 41 +++++++++ code/scripts/measure_peak_vram.py | 72 ++++++++++++++++ code/tests/test_import_smoke.py | 52 ++++++++++++ .../Ch1_sidebar__base-only_.log | 36 ++++++++ .../Ch1_sidebar__base_plus_lora_.log | 62 ++++++++++++++ .../cuda_dgx_spark/Ch2_adapter_preview.log | 48 +++++++++++ .../cuda_dgx_spark/Ch2_quickstart__LoRA_.log | 50 +++++++++++ .../cuda_dgx_spark/Ch3_dataset_manifest.log | 65 ++++++++++++++ .../cuda_dgx_spark/Ch3_synthetic_pipeline.log | 58 +++++++++++++ .../cuda_dgx_spark/Ch4_few_shot_hf.log | 9 ++ .../cuda_dgx_spark/Ch4_few_shot_mock.log | 6 ++ .../Ch4_prompt_validator_mock.log | 12 +++ .../cuda_dgx_spark/Ch4_rag_eval_hash.log | 26 ++++++ .../cuda_dgx_spark/Ch5_eval_report.md | 45 ++++++++++ .../Ch5_evaluate__base_vs_smoke_adapter_.log | 30 +++++++ .../Ch5_generate__load_adapter_.log | 19 +++++ code/validation/cuda_dgx_spark/README.md | 85 +++++++++++++++++++ code/validation/cuda_dgx_spark/_env.log | 6 ++ .../cuda_dgx_spark/dgx_spark_recipe.sh | 67 +++++++++++++++ 19 files changed, 789 insertions(+) create mode 100644 code/scripts/measure_peak_vram.py create mode 100644 code/tests/test_import_smoke.py create mode 100644 code/validation/cuda_dgx_spark/Ch1_sidebar__base-only_.log create mode 100644 code/validation/cuda_dgx_spark/Ch1_sidebar__base_plus_lora_.log create mode 100644 code/validation/cuda_dgx_spark/Ch2_adapter_preview.log create mode 100644 code/validation/cuda_dgx_spark/Ch2_quickstart__LoRA_.log create mode 100644 code/validation/cuda_dgx_spark/Ch3_dataset_manifest.log create mode 100644 code/validation/cuda_dgx_spark/Ch3_synthetic_pipeline.log create mode 100644 code/validation/cuda_dgx_spark/Ch4_few_shot_hf.log create mode 100644 code/validation/cuda_dgx_spark/Ch4_few_shot_mock.log create mode 100644 code/validation/cuda_dgx_spark/Ch4_prompt_validator_mock.log create mode 100644 code/validation/cuda_dgx_spark/Ch4_rag_eval_hash.log create mode 100644 code/validation/cuda_dgx_spark/Ch5_eval_report.md create mode 100644 code/validation/cuda_dgx_spark/Ch5_evaluate__base_vs_smoke_adapter_.log create mode 100644 code/validation/cuda_dgx_spark/Ch5_generate__load_adapter_.log create mode 100644 code/validation/cuda_dgx_spark/README.md create mode 100644 code/validation/cuda_dgx_spark/_env.log create mode 100644 code/validation/cuda_dgx_spark/dgx_spark_recipe.sh diff --git a/ACCELERATORS.md b/ACCELERATORS.md index abfa43f..c7aa5ad 100644 --- a/ACCELERATORS.md +++ b/ACCELERATORS.md @@ -33,6 +33,7 @@ chapters but is impractical for training. ## Which accelerator do I need? - **Any NVIDIA GPU with enough VRAM** runs everything; this is the reference path. +- **DGX Spark (GB10)** is validated for chapters 1 to 5 with the chapter recipe in this repo. Use a `cu130` (or newer) PyTorch wheel on GB10; older CUDA wheel indices may detect CUDA but lack `sm_121` kernels. - **An AMD GPU on Linux (ROCm)** also runs everything, including QLoRA and the full-parameter chapters. Best on datacenter (MI-series) cards; consumer RDNA support varies by GPU generation. - **A Mac (Apple Silicon)** is great for chapters 1 through 5's LoRA path, chapter 7's LoRA student, and chapter 9, but cannot *train* QLoRA or the full-parameter chapters (6 and 8). For training those, use a cloud GPU. - **No GPU, or a Mac/small card?** You can still follow chapters 5 through 8 by pulling the trained model from Hugging Face and running inference or evaluation, without training it. See [Running without training](#running-without-training-pull-the-model-from-hugging-face). @@ -108,6 +109,7 @@ The code is not pinned to one accelerator. The table records the exact machines | Accelerator | Machine | OS | Driver / runtime | PyTorch | Coverage | |---|---|---|---|---|---| +| NVIDIA CUDA | DGX Spark (GB10) | Linux (aarch64) | driver 580.159.03, CUDA 13.0 | 2.13.0+cu130 | Chapters 1 to 5 validated end-to-end with chapter-oriented recipe (`smoke`, `ch1` ... `ch5`, `all`), including chapter 2 quickstart training, chapter 3 synthetic pipeline, chapter 4 HF backend, and chapter 5 LoRA smoke + eval | | NVIDIA CUDA | A30 (24 GB) | Linux | CUDA 12.x | 2.11+cu126 | All chapters (reference platform for the book's published numbers) | | NVIDIA CUDA | H200 (140 GB) | Ubuntu (Nebius) | driver 580.159.04 | 2.12.1+cu126 | Full book end-to-end via validate_all.sh (25/25, incl. all training) | | NVIDIA CUDA | B200 (179 GB, Blackwell) | Ubuntu (Nebius) | driver 580.159.04 | 2.11.0+cu128 | Full book end-to-end via validate_all.sh (25/25); needs the cu128 wheel for sm_100, and 4-bit QLoRA is slow on current Blackwell kernels (see below) | @@ -115,10 +117,35 @@ The code is not pinned to one accelerator. The table records the exact machines | AMD ROCm | Instinct MI300X (192 GB) | Ubuntu 24.04 | ROCm 7.x, HIP 7.0.51831 | 2.10.0+rocm7.0 | Full book end-to-end, including Ch5 QLoRA (4-bit) and full-parameter Ch6/7/8 | | AMD ROCm | Instinct MI300X (192 GB) | Ubuntu (RunPod) | ROCm 6.1, HIP 6.1.40091 | 2.6.0+rocm6.1 | Full book except Ch5 QLoRA via validate_all.sh (24/25); QLoRA needs ROCm 6.2+ (no `bitsandbytes` 6.1 kernel, see AMD notes) | +## DGX Spark (GB10) run details (chapters 1 to 5) + +We validated the DGX Spark chapter recipe end-to-end, including each target +(`smoke`, `ch1`, `ch2`, `ch3`, `ch4`, `ch5`) and the umbrella `all` target. +Run artifacts are in `code/validation/cuda_dgx_spark/`. + +Key points from the run: + +- **Use cu130 wheels on GB10.** A cu126 wheel detects CUDA but warns that + kernels are not built for `sm_121` (compute capability 12.1). The tested-good + setup is PyTorch `2.13.0+cu130`. +- **Recipe fix landed during validation.** The chapter 2 data prep path + originally called `reformat_it_answers.py` without required flags. The recipe + now passes: + `python scripts/reformat_it_answers.py --in data/it_support/train.jsonl --out data/it_support_fmt/train.jsonl`. + (Since 2026-09-10 the script also has defaults, so the bare command processes both splits; the explicit flags still work.) +- **Optional OpenRouter warnings are non-fatal.** If `OPENROUTER_API_KEY` is + unset, the reformat step logs warnings and keeps passthrough rows; the run + still completes. +- **Known warning noise remains non-blocking.** Some scripts emit + `torch_dtype` deprecation and generation-config warnings (`top_p`/`top_k`); + execution and outputs are unaffected. + ## Dependency versions All accelerators run the same code on **Python 3.12** with `transformers` 4.57.6, `peft` 0.17.1, `trl` 1.5.x, `bitsandbytes` 0.49.2 (CUDA and ROCm only). The package pins `transformers>=4.47.0,<5.0`: transformers 5.x removes a symbol that `peft` imports, so a fresh install without the upper bound resolves to 5.x and breaks `import peft`. The per-accelerator PyTorch install command is in the [README Quick start](README.md#quick-start). +For **DGX Spark (GB10)** specifically, prefer a `cu130` (or newer) PyTorch wheel. The `cu126` wheel can report CUDA available but still miss `sm_121` kernels. + ## Performance across GPUs All three GPUs do a full end-to-end pass (every chapter, real 1-epoch training) with the same code and Python 3.12. Per-step wall time in seconds: @@ -144,10 +171,24 @@ All three GPUs do a full end-to-end pass (every chapter, real 1-epoch training) The A30 (Ampere, 2020) is roughly 2x the datacenter cards on training and ~2.5-3x on the generation-heavy eval steps. The H200 and MI300X are close to each other; a 4B model on small datasets does not stress either, so the gap between them would only open up with larger models, longer context, or bigger batches. +**DGX Spark note:** our DGX Spark validation was chapter-recipe scoped (chapters 1 to 5), not a full-book benchmark like the table above. So we do not include DGX Spark in this full-book timing matrix to avoid apples-to-oranges comparisons. + +Representative recipe-scope timings observed on DGX Spark GB10: + +| DGX Spark recipe step | Observed wall time | +|---|---:| +| Ch2 quickstart training | ~74 s | +| Ch5 tiny LoRA smoke training (`validate_chapter05.py`) | ~1.4 s (training runtime, after model load) | +| Typical Qwen3-4B checkpoint load (ch1-ch5 runs) | ~44 to 46 s | + +Raw logs for these timings are in `code/validation/cuda_dgx_spark/Recipe_ch2.log`, `code/validation/cuda_dgx_spark/Recipe_ch5.log`, and `code/validation/cuda_dgx_spark/Recipe_all.log`. + ## Cross-accelerator validation pass (validate_all.sh) We ran the full code path on every accelerator with one harness, `docs/overview/validate_all.sh --full` (raw scrubbed logs are in `code/validation//`). Every box produced **identical functional results**: drift baseline 0.1859 (YELLOW) and a deliberately topic-shifted 0.6855 (RED) with `kubernetes` as the top drift term, the same bias_fairness safety alert, and a clean LoRA adapter load with no offload error. So the code reproduces across Ampere, Hopper, and Blackwell (and on Apple Silicon for the inference paths). Per-step training wall time, in seconds: +DGX Spark results in this repo are tracked separately under `code/validation/cuda_dgx_spark/` because the validation scope there is chapters 1 to 5 via recipe targets rather than the full `validate_all.sh --full` matrix. Those artifacts include per-target logs (`Recipe_smoke.log`, `Recipe_ch1.log` ... `Recipe_ch5.log`, `Recipe_all.log`) and a run summary (`code/validation/cuda_dgx_spark/README.md`). + | Step | A30 (Ampere) | H200 (Hopper) | B200 (Blackwell) | |---|---:|---:|---:| | Ch5 LoRA train | 763 | 485 | 993 | diff --git a/code/scripts/measure_peak_vram.py b/code/scripts/measure_peak_vram.py new file mode 100644 index 0000000..12b195d --- /dev/null +++ b/code/scripts/measure_peak_vram.py @@ -0,0 +1,72 @@ +"""Measure the peak GPU memory a chapter's training script needs on YOUR hardware. + +Runs any training module in-process for a few steps and prints per-device peak +memory, so you can check a chapter against your card before committing to a run. +This is how the numbers in ACCELERATORS.md ("GPU requirements at a glance") were +produced on the book's reference A30s. + +Examples (run from code/ with the venv active; pick the GPUs with CUDA_VISIBLE_DEVICES): + + CUDA_VISIBLE_DEVICES=0 python -m scripts.measure_peak_vram chapter05.train_lora \ + --train data/it_support_fmt/train.jsonl --valid data/it_support_fmt/valid.jsonl \ + --out /tmp/probe_lora --max_steps 3 --report_to none + + CUDA_VISIBLE_DEVICES=0,1 python -m scripts.measure_peak_vram chapter06.train_sft \ + --train data/it_support_fmt/train.jsonl --valid data/it_support_fmt/valid.jsonl \ + --out /tmp/probe_sft --max_steps 3 --report_to none + +Everything after the module name is passed to that module unchanged. An +out-of-memory error is reported as a result, not a crash, with the peak reached +before the failure. Delete the --out directory afterwards (full SFT writes ~8 GB). +""" +from __future__ import annotations + +import json +import runpy +import sys +import time +import traceback + +import torch + +from common.gpu import peak_gpu_memory + + +def main() -> None: + if len(sys.argv) < 2 or sys.argv[1] in ("-h", "--help"): + print(__doc__) + sys.exit(0) + module = sys.argv[1] + sys.argv = [module] + sys.argv[2:] + t0 = time.time() + status, error = "ok", "" + try: + runpy.run_module(module, run_name="__main__") + except SystemExit: + pass + except BaseException as exc: # noqa: BLE001 - we want OOM reported as a result + status = "ERROR" + error = f"{type(exc).__name__}: {str(exc)[:300]}" + traceback.print_exc() + result = { + "module": module, + "status": status, + "error": error, + "seconds": round(time.time() - t0), + "torch": torch.__version__, + "devices": peak_gpu_memory(), + } + total = sum(float(d["peak_allocated_gib"]) for d in result["devices"]) + print("\n=== Peak GPU memory ===") + for d in result["devices"]: + print(f" cuda:{d['device']} ({d['name']}): {d['peak_allocated_gib']:.2f} GiB allocated, " + f"{d['peak_reserved_gib']:.2f} GiB reserved") + if len(result["devices"]) > 1: + print(f" total across GPUs: {total:.2f} GiB (what a single card would need)") + if status != "ok": + print(f" run ended with {error}") + print("PEAK_VRAM_RESULT " + json.dumps(result)) + + +if __name__ == "__main__": + main() diff --git a/code/tests/test_import_smoke.py b/code/tests/test_import_smoke.py new file mode 100644 index 0000000..bc595f4 --- /dev/null +++ b/code/tests/test_import_smoke.py @@ -0,0 +1,52 @@ +"""Import-smoke test. + +Imports every module that pulls the heavy ML dependency chain (peft, trl, +transformers) so CI catches dependency-resolution breaks on a fresh install -- +the kind the chapter test suites miss because they only import lightweight data +and metrics modules. + +This exists because a fresh install once resolved transformers to 5.x, which +removed ``HybridCache``; ``peft<0.18`` imports that at load time, so +``import peft`` (and the chapter 2 quickstart) failed everywhere -- yet CI +stayed green because no test imported peft. These imports are GPU-free and do +no training; they only exercise the import graph. + +Deliberately excluded: + - chapter05.train_qlora (imports bitsandbytes, a CUDA-only extra) + - chapter03.ch03_data_quality_explore (a script with no __main__ guard, so + importing it would run the full experiment) +""" +from __future__ import annotations + +import importlib +import sys +from pathlib import Path + +import pytest + +# Make the code/ root importable when running pytest without an editable install. +_code_root = Path(__file__).resolve().parent.parent +if str(_code_root) not in sys.path: + sys.path.insert(0, str(_code_root)) + +MODULES = [ + "chapter02.quickstart", + "chapter05.modeling", + "chapter05.train_lora", + "chapter06.train_sft", + "chapter07.train_student", + "chapter08.train_dpo", + "chapter09.safety_monitor", +] + + +@pytest.mark.parametrize("module", MODULES) +def test_module_imports(module: str) -> None: + """Importing the module must not raise (catches dependency-resolution breaks). + + Chapters not present in this checkout are skipped, not failed: the public repo + publishes chapters as the MEAP releases them, so a chapter's package may be absent. + """ + if not (_code_root / module.split(".")[0]).is_dir(): + pytest.skip(f"{module.split('.')[0]} is not in this checkout") + importlib.import_module(module) diff --git a/code/validation/cuda_dgx_spark/Ch1_sidebar__base-only_.log b/code/validation/cuda_dgx_spark/Ch1_sidebar__base-only_.log new file mode 100644 index 0000000..e87fbc5 --- /dev/null +++ b/code/validation/cuda_dgx_spark/Ch1_sidebar__base-only_.log @@ -0,0 +1,36 @@ +`torch_dtype` is deprecated! Use `dtype` instead! +[load] base model Qwen/Qwen3-4B-Instruct-2507 + Loading checkpoint shards: 0%| | 0/3 [00:00 ✅ This works for both Outlook desktop and web versions. + +--- + +### 📱 2. **If You’re Using Outlook on the Web (OWA)** +- You can reset your password directly in the browser. +- Just go to [https://outlook.office.com](https://outlook.office + +[skip] --base_only set; skipping LoRA run. + +[skip] --base_only set; skipping SFT run. + +[saved] /home/weehyong/experimentations/ModelAadptationBook/ModelAdaptationBook/code/chapter01/sidebar_outputs.json diff --git a/code/validation/cuda_dgx_spark/Ch1_sidebar__base_plus_lora_.log b/code/validation/cuda_dgx_spark/Ch1_sidebar__base_plus_lora_.log new file mode 100644 index 0000000..89092c1 --- /dev/null +++ b/code/validation/cuda_dgx_spark/Ch1_sidebar__base_plus_lora_.log @@ -0,0 +1,62 @@ +`torch_dtype` is deprecated! Use `dtype` instead! +[load] base model Qwen/Qwen3-4B-Instruct-2507 + Loading checkpoint shards: 0%| | 0/3 [00:00 ✅ This works for both Outlook desktop and web versions. + +--- + +### 📱 2. **If You’re Using Outlook on the Web (OWA)** +- You can reset your password directly in the browser. +- Just go to [https://outlook.office.com](https://outlook.office + +[load] LoRA adapter on top of base: chapter05/runs/validate_lora_smoke + +=== BASE + LoRA adapter (validate_lora_smoke) === +If you've forgotten your Outlook password, here’s what you should do — depending on your situation: + +### 1. **Check if You Can Reset the Password via Microsoft 365 (Office 365)** +Most Outlook accounts are tied to a Microsoft 365 (formerly Office 365) account. To reset your password: + +- Go to [https://account.microsoft.com](https://account.microsoft.com) +- Sign in with your email address or phone number associated with your Microsoft account. +- Click on **"Password"** under your profile. +- Follow the prompts to reset your password using: + - A trusted device (if you have one), + - A security question, + - Or a phone number linked to your account. + +> ✅ This works for personal Outlook.com, Hotmail, or work/school accounts using Microsoft 365. + +--- + +### 2. **If You're Using a Work or School Account (Exchange/Office 365)** +Your organization may have a policy for password resets. In this case: + +- Contact your **IT department** or **HR support team**. +- They can help you reset your password through their internal systems. +- You may need to verify your identity (e.g., via email + +[skip] SFT model not found at /home/weehyong/experimentations/ModelAadptationBook/ModelAdaptationBook/code/chapter06/runs/sft_run1. + Run Chapter 6 first to produce sft_run1, or pass --sft_dir. + +[saved] /home/weehyong/experimentations/ModelAadptationBook/ModelAdaptationBook/code/chapter01/sidebar_outputs.json diff --git a/code/validation/cuda_dgx_spark/Ch2_adapter_preview.log b/code/validation/cuda_dgx_spark/Ch2_adapter_preview.log new file mode 100644 index 0000000..546b8e7 --- /dev/null +++ b/code/validation/cuda_dgx_spark/Ch2_adapter_preview.log @@ -0,0 +1,48 @@ +Loading base model: Qwen/Qwen3-4B-Instruct-2507 + Loading checkpoint shards: 0%| | 0/3 [00:00system +You are a helpful assistant.<|im_end|> +<|im_start|>user +My laptop won't connect to office Wi-Fi after update. What should I do?<|im_end|> +<|im_start|>assistant +If your laptop won't connect to your office Wi-Fi after a recent update, here are several steps you can take to troubleshoot and resolve the issue: + +--- + +### 🔍 1. **Check the Wi-Fi Connection Basics** +- **Ensure the Wi-Fi is turned on** (check the Wi-Fi icon in the system tray). +- **Verify the office Wi-Fi network is active** — ask your IT department if the network is down or if there are any outages. +- **Try connecting to a different network** (e.g., home Wi-Fi) to confirm your laptop is functioning properly. + +--- + +### 🔄 2. diff --git a/code/validation/cuda_dgx_spark/README.md b/code/validation/cuda_dgx_spark/README.md new file mode 100644 index 0000000..75681da --- /dev/null +++ b/code/validation/cuda_dgx_spark/README.md @@ -0,0 +1,85 @@ +# DGX Spark Validation (Chapters 1-5) + +This folder captures a real reader-style validation run on DGX Spark class hardware (NVIDIA GB10). + +## Environment + +See [./_env.log](./_env.log). + +Summary: +- torch 2.13.0+cu130 +- CUDA available: True +- GPU: NVIDIA GB10 +- Platform: Linux aarch64 + +## What Was Run (Chapter by Chapter) + +### Chapter 1 +- PASS: base-only sidebar run + - [Ch1_sidebar__base-only_.log](./Ch1_sidebar__base-only_.log) +- PASS: base + LoRA adapter run (using Chapter 5 smoke adapter) + - [Ch1_sidebar__base_plus_lora_.log](./Ch1_sidebar__base_plus_lora_.log) +- Expected skip: SFT path skipped because Chapter 6 artifact was not present in this chapter-1-to-5 scope. + +### Chapter 2 +- PASS: adapter preview script (resolved Hub adapter and generated outputs) + - [Ch2_adapter_preview.log](./Ch2_adapter_preview.log) +- PASS: full quickstart LoRA training run + - [Ch2_quickstart__LoRA_.log](./Ch2_quickstart__LoRA_.log) + +### Chapter 3 +- PASS: dataset manifest runnable module + - [Ch3_dataset_manifest.log](./Ch3_dataset_manifest.log) +- PASS: end-to-end synthetic data generation pipeline (teacher calls + quality gate + save) + - [Ch3_synthetic_pipeline.log](./Ch3_synthetic_pipeline.log) +- Note: the heavier data-quality experiment script (4 training conditions) was not executed in this run because it is a long benchmark-style workload; dependencies import correctly. + +### Chapter 4 +- PASS: few-shot mock backend + - [Ch4_few_shot_mock.log](./Ch4_few_shot_mock.log) +- PASS: few-shot HF backend on GPU + - [Ch4_few_shot_hf.log](./Ch4_few_shot_hf.log) +- PASS: prompt validator mock backend + - [Ch4_prompt_validator_mock.log](./Ch4_prompt_validator_mock.log) +- PASS: RAG retrieval evaluation (hash backend) + - [Ch4_rag_eval_hash.log](./Ch4_rag_eval_hash.log) + +### Chapter 5 +- PASS: chapter validator with tiny LoRA smoke training on CUDA + - Captured in terminal output during validation run +- PASS: inference generation using smoke adapter + - [Ch5_generate__load_adapter_.log](./Ch5_generate__load_adapter_.log) +- PASS: base vs adapter smoke evaluation pipeline + - [Ch5_evaluate__base_vs_smoke_adapter_.log](./Ch5_evaluate__base_vs_smoke_adapter_.log) + - [Ch5_eval_report.md](./Ch5_eval_report.md) + +## Issues Encountered and Fix/Recommendation + +1. High severity: wrong CUDA wheel for GB10 (sm_121) +- Symptom: torch 2.13.0+cu126 detected CUDA but warned kernels were not built for compute capability 12.1. +- Impact: training/inference can fail or silently use unsupported paths. +- Fix used in this validation: install torch wheels from cu130 index. +- Recommendation for docs/recipes: on DGX Spark/GB10, default to cu130 or newer wheels. + +2. Medium severity: deprecation warning from transformers keyword +- Symptom: torch_dtype warning asking to use dtype. +- Where observed: Chapter 1 and Chapter 4 HF run logs. +- Impact: no runtime failure, but noisy and future compatibility risk. +- Recommendation: migrate from torch_dtype to dtype in model loading calls. + +3. Medium severity: generation config warnings (temperature/top_p/top_k) +- Symptom: warning that generation flags may be ignored. +- Where observed: Chapter 1, Chapter 2 adapter preview, Chapter 4 HF, Chapter 5 generate/eval. +- Impact: behavior still runs, but warning can confuse readers. +- Recommendation: align generation config fields with current transformers version and remove stale flags. + +4. Medium severity: chapter scope coupling +- Symptom: Chapter 1 script optionally expects Chapter 6 SFT artifact. +- Impact: readers doing chapters 1-5 only will see expected skip message. +- Recommendation: keep current behavior but document this explicitly in chapter 1 quick-start notes. + +## Overall Verdict (Reader on DGX Spark) + +- Chapters 1-5 are runnable on DGX Spark with GPU acceleration when torch is installed from a GB10-compatible CUDA index (cu130 tested here). +- All executed chapter paths in this validation passed. +- The primary blocker was environment setup (GPU architecture wheel mismatch), not code logic. diff --git a/code/validation/cuda_dgx_spark/_env.log b/code/validation/cuda_dgx_spark/_env.log new file mode 100644 index 0000000..f610ae5 --- /dev/null +++ b/code/validation/cuda_dgx_spark/_env.log @@ -0,0 +1,6 @@ +torch 2.13.0+cu130 +platform Linux-6.17.0-1026-nvidia-aarch64-with-glibc2.39 +cuda True +gpu_count 1 +gpu_name NVIDIA GB10 +GPU 0: NVIDIA GB10 (UUID: GPU-82b9b9c3-29dd-ef5f-2312-33340dc5de6c) diff --git a/code/validation/cuda_dgx_spark/dgx_spark_recipe.sh b/code/validation/cuda_dgx_spark/dgx_spark_recipe.sh new file mode 100644 index 0000000..3ae653d --- /dev/null +++ b/code/validation/cuda_dgx_spark/dgx_spark_recipe.sh @@ -0,0 +1,67 @@ +#!/usr/bin/env bash +set -euo pipefail + +# DGX Spark recipe for ModelAdaptationBook chapters 1-5 +# Run from the repository root or from code/. + +if [[ -d code ]]; then + cd code +fi + +if [[ ! -f pyproject.toml ]]; then + echo "Expected to run in repository code/ directory containing pyproject.toml" + exit 1 +fi + +echo "[1/8] Python version" +python3 --version + +echo "[2/8] Create virtual environment" +python3 -m venv .venv +source .venv/bin/activate +python -m pip install -U pip + +echo "[3/8] Install GB10-compatible GPU PyTorch" +python -m pip install --upgrade --index-url https://download.pytorch.org/whl/cu130 torch torchvision torchaudio + +echo "[4/8] Verify CUDA" +python - <<'PY' +import torch +print('torch', torch.__version__) +print('cuda', torch.cuda.is_available()) +print('gpu_count', torch.cuda.device_count()) +if torch.cuda.is_available(): + print('gpu_name', torch.cuda.get_device_name(0)) + x = torch.randn(1024, 1024, device='cuda') + y = x @ x + torch.cuda.synchronize() + print('cuda_matmul_ok', y.shape) +PY + +echo "[5/8] Install book package" +python -m pip install -e ".[dev,chapter03]" + +echo "[6/8] Build dataset once for chapter 2/5" +python scripts/build_it_support_dataset.py +python scripts/reformat_it_answers.py + +echo "[7/8] Run chapter validation path" +python -m chapter01.run_sidebar_example --base_only +python -m chapter02.quickstart +python -m chapter03.ch03_datasetmanifest +python -m chapter04.few_shot_demo --shots 8 --backend hf --output chapter04/runs/few_shot_hf_dgx.json +python -m chapter04.prompt_validator --shots 8 --runs 3 --backend mock +python chapter05/scripts/validate_chapter05.py +python -m chapter05.generate --base Qwen/Qwen3-4B-Instruct-2507 --adapter chapter05/runs/validate_lora_smoke --prompt "My laptop won't connect to office Wi-Fi after update. What should I do?" +python -m chapter05.scripts.listing_5_3_evaluate \ + --base Qwen/Qwen3-4B-Instruct-2507 \ + --adapter chapter05/runs/validate_lora_smoke \ + --dolly_test chapter05/data/smoke/valid.jsonl \ + --toy_golden chapter05/data/golden/toy_test.jsonl \ + --safety_prompts chapter05/data/golden/safety_regression_prompts.jsonl \ + --out chapter05/runs/eval_report_dgx_smoke + +echo "[8/8] Quick test sweep" +python -m pytest -q chapter04/tests chapter05/tests + +echo "DGX Spark chapter 1-5 recipe completed successfully." From 9bb1c54901ee6f21aacb4e79d21c73e1f785ddf6 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 14 Sep 2026 00:21:54 +0000 Subject: [PATCH 2/4] Fix DGX Spark README filename and command args Co-authored-by: bahree <3529468+bahree@users.noreply.github.com> --- DGX-SPARK-READM.MD => DGX-SPARK-README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) rename DGX-SPARK-READM.MD => DGX-SPARK-README.md (98%) diff --git a/DGX-SPARK-READM.MD b/DGX-SPARK-README.md similarity index 98% rename from DGX-SPARK-READM.MD rename to DGX-SPARK-README.md index 57175f5..9a249f1 100644 --- a/DGX-SPARK-READM.MD +++ b/DGX-SPARK-README.md @@ -71,7 +71,7 @@ Expected: ```bash python scripts/build_it_support_dataset.py -python scripts/reformat_it_answers.py +python scripts/reformat_it_answers.py --in data/it_support/train.jsonl --out data/it_support_fmt/train.jsonl ``` ## 6. Chapter-by-chapter run sequence (1 to 5) From 2fcfc6113659b587d1fa40030d7fdc99f886524c Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 14 Sep 2026 00:22:59 +0000 Subject: [PATCH 3/4] Add --input/--output aliases and update DGX docs Co-authored-by: bahree <3529468+bahree@users.noreply.github.com> --- ACCELERATORS.md | 3 ++- DGX-SPARK-README.md | 4 ++-- DGX-SPARK-RECIPE.sh | 2 +- code/scripts/reformat_it_answers.py | 10 +++++----- 4 files changed, 10 insertions(+), 9 deletions(-) diff --git a/ACCELERATORS.md b/ACCELERATORS.md index 3e84f62..b3ec5af 100644 --- a/ACCELERATORS.md +++ b/ACCELERATORS.md @@ -131,7 +131,8 @@ Key points from the run: - **Recipe fix landed during validation.** The chapter 2 data prep path originally called `reformat_it_answers.py` without required flags. The recipe now passes: - `python scripts/reformat_it_answers.py --in data/it_support/train.jsonl --out data/it_support_fmt/train.jsonl`. + `python scripts/reformat_it_answers.py --input data/it_support/train.jsonl --output data/it_support_fmt/train.jsonl`. + (Since 2026-09-10 the script also has defaults, so the bare command processes both splits; the explicit flags still work.) - **Optional OpenRouter warnings are non-fatal.** If `OPENROUTER_API_KEY` is unset, the reformat step logs warnings and keeps passthrough rows; the run still completes. diff --git a/DGX-SPARK-README.md b/DGX-SPARK-README.md index 9a249f1..55f0261 100644 --- a/DGX-SPARK-README.md +++ b/DGX-SPARK-README.md @@ -71,7 +71,7 @@ Expected: ```bash python scripts/build_it_support_dataset.py -python scripts/reformat_it_answers.py --in data/it_support/train.jsonl --out data/it_support_fmt/train.jsonl +python scripts/reformat_it_answers.py --input data/it_support/train.jsonl --output data/it_support_fmt/train.jsonl ``` ## 6. Chapter-by-chapter run sequence (1 to 5) @@ -240,7 +240,7 @@ The new chapter-oriented recipe was exercised on DGX Spark in this environment: Recipe fix applied during validation: - Updated data reformat step to pass required args: - `python scripts/reformat_it_answers.py --in data/it_support/train.jsonl --out data/it_support_fmt/train.jsonl` + `python scripts/reformat_it_answers.py --input data/it_support/train.jsonl --output data/it_support_fmt/train.jsonl` If you are trying this on a fresh DGX Spark machine, run `setup` first, then `smoke`, then chapter commands (`ch1` ... `ch5`) or `all`. diff --git a/DGX-SPARK-RECIPE.sh b/DGX-SPARK-RECIPE.sh index d09b3b7..6a712f2 100644 --- a/DGX-SPARK-RECIPE.sh +++ b/DGX-SPARK-RECIPE.sh @@ -86,7 +86,7 @@ do_data() { ensure_venv echo "[data] Building IT-support dataset" python scripts/build_it_support_dataset.py - python scripts/reformat_it_answers.py --in data/it_support/train.jsonl --out data/it_support_fmt/train.jsonl + python scripts/reformat_it_answers.py --input data/it_support/train.jsonl --output data/it_support_fmt/train.jsonl } do_ch1() { diff --git a/code/scripts/reformat_it_answers.py b/code/scripts/reformat_it_answers.py index a843452..028b858 100644 --- a/code/scripts/reformat_it_answers.py +++ b/code/scripts/reformat_it_answers.py @@ -20,7 +20,7 @@ while its token-F1 is still scored against the original human answers. python scripts/reformat_it_answers.py # both splits - python scripts/reformat_it_answers.py --in X --out Y # one file + python scripts/reformat_it_answers.py --input X --output Y # one file Needs OPENROUTER_API_KEY (see code/README.md). Both output files are committed to the repo, so you only need to run this if you rebuild the dataset from source. @@ -142,16 +142,16 @@ def qa(row): def main(): ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) - ap.add_argument("--in", dest="inp", default=None, - help="input split from build_it_support_dataset.py; with no --in/--out both the train and " + ap.add_argument("--in", "--input", dest="inp", default=None, + help="input split from build_it_support_dataset.py; with no --input/--output both the train and " "valid splits are processed (the default the READMEs rely on)") - ap.add_argument("--out", default=None, help="output path (required if --in is given)") + ap.add_argument("--out", "--output", dest="out", default=None, help="output path (required if --input is given)") ap.add_argument("--limit", type=int, default=0, help="0 = all") ap.add_argument("--dry", action="store_true", help="print, do not write") ap.add_argument("--workers", type=int, default=8, help="concurrent API calls") args = ap.parse_args() if (args.inp is None) != (args.out is None): - ap.error("--in and --out must be given together (or neither, to process both default splits)") + ap.error("--input and --output must be given together (or neither, to process both default splits)") pairs = [(args.inp, args.out)] if args.inp else SPLITS for inp, out in pairs: process(inp, out, args.limit, args.dry, args.workers) From 2f506459bcbb95f86d6931151ba73c2b8229b770 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 14 Sep 2026 00:25:04 +0000 Subject: [PATCH 4/4] Align DGX data prep docs with default dual-split behavior Co-authored-by: bahree <3529468+bahree@users.noreply.github.com> --- ACCELERATORS.md | 8 ++++---- DGX-SPARK-README.md | 6 +++--- DGX-SPARK-RECIPE.sh | 2 +- 3 files changed, 8 insertions(+), 8 deletions(-) diff --git a/ACCELERATORS.md b/ACCELERATORS.md index b3ec5af..e831ba7 100644 --- a/ACCELERATORS.md +++ b/ACCELERATORS.md @@ -129,10 +129,10 @@ Key points from the run: kernels are not built for `sm_121` (compute capability 12.1). The tested-good setup is PyTorch `2.13.0+cu130`. - **Recipe fix landed during validation.** The chapter 2 data prep path - originally called `reformat_it_answers.py` without required flags. The recipe - now passes: - `python scripts/reformat_it_answers.py --input data/it_support/train.jsonl --output data/it_support_fmt/train.jsonl`. - (Since 2026-09-10 the script also has defaults, so the bare command processes both splits; the explicit flags still work.) + originally called `reformat_it_answers.py` in a way that failed on older script versions. The recipe + now uses: + `python scripts/reformat_it_answers.py`. + (Since 2026-09-10 the script defaults to processing both train and valid splits; explicit file flags still work.) - **Optional OpenRouter warnings are non-fatal.** If `OPENROUTER_API_KEY` is unset, the reformat step logs warnings and keeps passthrough rows; the run still completes. diff --git a/DGX-SPARK-README.md b/DGX-SPARK-README.md index 55f0261..4f50eaf 100644 --- a/DGX-SPARK-README.md +++ b/DGX-SPARK-README.md @@ -71,7 +71,7 @@ Expected: ```bash python scripts/build_it_support_dataset.py -python scripts/reformat_it_answers.py --input data/it_support/train.jsonl --output data/it_support_fmt/train.jsonl +python scripts/reformat_it_answers.py ``` ## 6. Chapter-by-chapter run sequence (1 to 5) @@ -239,8 +239,8 @@ The new chapter-oriented recipe was exercised on DGX Spark in this environment: Recipe fix applied during validation: -- Updated data reformat step to pass required args: - `python scripts/reformat_it_answers.py --input data/it_support/train.jsonl --output data/it_support_fmt/train.jsonl` +- Updated data reformat step to process both train and valid splits by default: + `python scripts/reformat_it_answers.py` If you are trying this on a fresh DGX Spark machine, run `setup` first, then `smoke`, then chapter commands (`ch1` ... `ch5`) or `all`. diff --git a/DGX-SPARK-RECIPE.sh b/DGX-SPARK-RECIPE.sh index 6a712f2..7a7a405 100644 --- a/DGX-SPARK-RECIPE.sh +++ b/DGX-SPARK-RECIPE.sh @@ -86,7 +86,7 @@ do_data() { ensure_venv echo "[data] Building IT-support dataset" python scripts/build_it_support_dataset.py - python scripts/reformat_it_answers.py --input data/it_support/train.jsonl --output data/it_support_fmt/train.jsonl + python scripts/reformat_it_answers.py } do_ch1() {