fix(scripts): create portable A/B output directories

TECHNICAL_REPORTS/2276-portable-ab-output-dirs-20261012.en.md

@@ -0,0 +1,36 @@
+# Technical Report: PR #2276 - Portable A/B Output Directories
+
+**Date**: 2026-10-12
+**Status**: Ready for review
+**Languages**: Bash
+**Risk Level**: Low
+
+## Executive Summary
+
+The A/B output-equality and paged-decode gates now create their default artifact directories with templates accepted by both GNU and BSD `mktemp`. If a default or requested output directory cannot be created, each gate prints a clear error and exits 2 before running a model or writing artifacts.
+
+## 1. Problem Statement
+
+GNU `mktemp -d -t <prefix>` rejects a prefix without `X` placeholders. The previous scripts left `OUT_DIR` empty after that failure and continued, which could misreport the setup failure as a model error and form output paths at the filesystem root.
+
+## 3. Technical Decisions
+
+### Use path templates and fail before execution
+
+Both scripts use `${TMPDIR:-/tmp}/<gate>.XXXXXX`, which works on GNU/Linux and macOS. The guards then reject an empty directory value or a failed `mkdir -p` with the scripts' inconclusive exit status, so all later artifact paths remain rooted under a validated directory.
+
+The regression fixture uses stub binaries and HTTP tools to exercise the full shell flows without a checkpoint or server. Its fake curl consumes piped request data, preserving the production pipeline's `pipefail` behavior without an intermittent SIGPIPE from the fake jq producer.
+
+## 7. Change Summary
+
+| Category | Count | Summary |
+|---|---:|---|
+| Production scripts | 2 | Portable default directories and explicit exit-2 setup failures |
+| Regression tests | 1 | Default path, invalid explicit path, direct `mktemp` failure, and no-model-invocation coverage |
+
+## Appendix: Test Results
+
+- `bash scripts/ci/ab_scripts_outdir_test.sh` passed, including the direct `TMPDIR` failure paths for both gates.
+- Twelve concurrent copies of the focused test passed after the fixture consumed the piped request body.
+- The focused test failed against the pre-fix `origin/main` scripts with eight expected assertions.
+- `bash -n scripts/ab_output_equality.sh scripts/paged_decode_counter_ab.sh scripts/ci/ab_scripts_outdir_test.sh` and `git diff --check` passed.

TECHNICAL_REPORTS/2276-portable-ab-output-dirs-20261012.ko.md

@@ -0,0 +1,36 @@
+# 기술 보고서: PR #2276 - 이식 가능한 A/B 출력 디렉터리
+
+**작성일**: 2026-10-12
+**상태**: 리뷰 준비 완료
+**언어**: Bash
+**위험도**: 낮음
+
+## 요약
+
+A/B 출력 동일성 게이트와 paged decode 게이트가 이제 GNU와 BSD `mktemp` 모두에서 허용되는 템플릿으로 기본 산출물 디렉터리를 만듭니다. 기본 또는 요청한 출력 디렉터리를 만들 수 없으면 각 게이트는 모델을 실행하거나 산출물을 쓰기 전에 명확한 오류를 출력하고 종료 상태 2를 반환합니다.
+
+## 1. 문제 정의
+
+GNU의 `mktemp -d -t <prefix>`는 `X` 자리표시자가 없는 접두사를 거부합니다. 이전 스크립트는 이 실패 뒤에도 빈 `OUT_DIR`로 계속 실행되어, 설정 실패를 모델 오류로 잘못 보고하거나 파일 시스템 루트에 출력 경로를 만들 수 있었습니다.
+
+## 3. 기술적 선택과 그 이유
+
+### 경로 템플릿 사용 및 실행 전 실패 처리
+
+두 스크립트는 GNU/Linux와 macOS에서 모두 동작하는 `${TMPDIR:-/tmp}/<gate>.XXXXXX`를 사용합니다. 이후 가드는 빈 디렉터리 값 또는 실패한 `mkdir -p`를 스크립트의 판정 불가 종료 상태로 처리하므로, 이후의 모든 산출물 경로는 검증된 디렉터리 아래에 남습니다.
+
+회귀 테스트 픽스처는 실제 체크포인트나 서버 없이 전체 셸 흐름을 실행하기 위해 stub 바이너리와 HTTP 도구를 사용합니다. 가짜 curl은 파이프된 요청 본문을 소비하여 가짜 jq 생산자에서 발생하는 간헐적인 SIGPIPE 없이 실제 파이프라인의 `pipefail` 동작을 보존합니다.
+
+## 7. 변경 요약
+
+| 분류 | 수 | 요약 |
+|---|---:|---|
+| 운영 스크립트 | 2 | 이식 가능한 기본 디렉터리와 명시적인 종료 상태 2 설정 실패 |
+| 회귀 테스트 | 1 | 기본 경로, 잘못된 명시 경로, 직접 `mktemp` 실패, 모델 미실행 검증 |
+
+## 부록: 테스트 결과
+
+- `bash scripts/ci/ab_scripts_outdir_test.sh`가 두 게이트의 직접 `TMPDIR` 실패 경로를 포함해 통과했습니다.
+- 가짜 curl이 파이프된 요청 본문을 소비하도록 한 뒤 집중 테스트 12개 동시 실행이 모두 통과했습니다.
+- 집중 테스트는 수정 전 `origin/main` 스크립트에서 예상한 8개 검증 실패를 보였습니다.
+- `bash -n scripts/ab_output_equality.sh scripts/paged_decode_counter_ab.sh scripts/ci/ab_scripts_outdir_test.sh` 및 `git diff --check`가 통과했습니다.

scripts/ab_output_equality.sh

@@ -49,8 +49,8 @@
 #   /bin/cp target/release/mlxcel target/release/mlxcel.before
 #   git stash pop && cargo build --release --features metal,accelerate --bin mlxcel
 #
-# Exit status: 0 all pairs equal, 1 at least one pair differs, 2 at least one
-# pair inconclusive (unstable baseline) and none differ.
+# Exit status: 0 all pairs equal, 1 at least one pair differs, 2 at least one pair
+# inconclusive (unstable baseline) and none differ, or the output directory cannot be created.
 
 set -uo pipefail
 
@@ -126,9 +126,15 @@ if [[ ${#PROMPTS[@]} -eq 0 ]]; then
 fi
 
 if [[ -z "$OUT_DIR" ]]; then
-  OUT_DIR="$(mktemp -d -t ab_output_equality)"
+  OUT_DIR="$(mktemp -d "${TMPDIR:-/tmp}/ab_output_equality.XXXXXX")" || {
+    echo "error: could not create output directory" >&2
+    exit 2
+  }
+fi
+if [[ -z "$OUT_DIR" ]] || ! mkdir -p "$OUT_DIR"; then
+  echo "error: could not create output directory" >&2
+  exit 2
 fi
-mkdir -p "$OUT_DIR"
 
 echo "baseline: $BASELINE_BIN"
 echo "arm:      $ARM_BIN"

scripts/ci/ab_scripts_outdir_test.sh

@@ -0,0 +1,174 @@
+#!/usr/bin/env bash
+# Regression coverage for the A/B script default output directories.
+
+set -euo pipefail
+
+script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+repo_root="$(cd "$script_dir/../.." && pwd)"
+output_equality="$repo_root/scripts/ab_output_equality.sh"
+paged_counter="$repo_root/scripts/paged_decode_counter_ab.sh"
+
+work="$(mktemp -d "${TMPDIR:-/tmp}/ab-scripts-outdir-test.XXXXXX")"
+trap 'rm -rf "$work"' EXIT
+
+failures=0
+
+fail() {
+  echo "FAIL [$1]: $2" >&2
+  failures=$((failures + 1))
+}
+
+write_fake_mlxcel() {
+  local path="$1" marker="$2"
+  printf '%s\n' \
+    '#!/usr/bin/env bash' \
+    "# $marker" \
+    'if [[ -n "${AB_SCRIPTS_OUTDIR_TEST_MODEL_MARKER:-}" ]]; then' \
+    '  : > "$AB_SCRIPTS_OUTDIR_TEST_MODEL_MARKER"' \
+    'fi' \
+    'case "${1:-}" in' \
+    '  generate)' \
+    '    printf "Generating...\\n"' \
+    '    printf "same generated text\\n"' \
+    '    printf "[Generated 1 tokens in 0.01s]\\n"' \
+    '    ;;' \
+    '  serve)' \
+    '    while :; do sleep 1; done' \
+    '    ;;' \
+    '  *)' \
+    '    exit 1' \
+    '    ;;' \
+    'esac' > "$path"
+  chmod +x "$path"
+}
+
+write_fake_curl() {
+  local path="$1"
+  printf '%s\n' \
+    '#!/usr/bin/env bash' \
+    'set -euo pipefail' \
+    'output=""' \
+    'read_stdin=0' \
+    'url=""' \
+    'while [[ $# -gt 0 ]]; do' \
+    '  case "$1" in' \
+    '    -o) output="$2"; shift 2 ;;' \
+    '    -H) shift 2 ;;' \
+    '    --data-binary) [[ "$2" == "@-" ]] && read_stdin=1; shift 2 ;;' \
+    '    -*) shift ;;' \
+    '    *) url="$1"; shift ;;' \
+    '  esac' \
+    'done' \
+    'case "$url" in' \
+    '  */health) body="{}" ;;' \
+    '  */metrics) body=$(printf "mlxcel_paged_decode_launches_total{path=\\\"fused_v2\\\"} 1\\nmlxcel_paged_decode_launches_total{path=\\\"gather\\\"} 0\\nmlxcel_paged_decode_launches_total{path=\\\"cascade\\\"} 0") ;;' \
+    '  */v1/completions) body="{\\\"choices\\\":[{\\\"text\\\":\\\"same\\\"}]}" ;;' \
+    '  *) printf "unexpected curl URL: %s\\n" "$url" >&2; exit 1 ;;' \
+    'esac' \
+    'if [[ "$read_stdin" -eq 1 ]]; then cat >/dev/null; fi' \
+    'if [[ -n "$output" ]]; then' \
+    '  printf "%s\\n" "$body" > "$output"' \
+    'else' \
+    '  printf "%s\\n" "$body"' \
+    'fi' > "$path"
+  chmod +x "$path"
+}
+
+write_fake_jq() {
+  local path="$1"
+  printf '%s\n' \
+    '#!/usr/bin/env bash' \
+    'if [[ "${1:-}" == "-n" ]]; then' \
+    '  printf "{}\\n"' \
+    '  exit 0' \
+    'fi' \
+    'if [[ "${1:-}" == "-r" ]]; then' \
+    '  printf "same\\n"' \
+    '  exit 0' \
+    'fi' \
+    'exit 1' > "$path"
+  chmod +x "$path"
+}
+
+assert_default_dir() {
+  local name="$1" prefix="$2" output="$3"
+  local out_dir
+  out_dir="$(sed -n 's/^outputs:[[:space:]]*//p' <<< "$output")"
+  case "$out_dir" in
+    "$work/tmp/$prefix".??????) ;;
+    *)
+      fail "$name" "expected an output directory below $work/tmp, got '${out_dir:-<empty>}'"
+      return
+      ;;
+  esac
+  [[ -d "$out_dir" ]] || fail "$name" "output directory was not created: $out_dir"
+}
+
+assert_create_failure() {
+  local name="$1" script="$2"
+  shift 2
+  local output="$work/$name.failure.out"
+  local status
+  set +e
+  "$script" "$@" --out-dir "$work/not-a-directory/child" > "$output" 2>&1
+  status=$?
+  set -e
+  [[ "$status" -eq 2 ]] || fail "$name create failure" "expected exit 2, got $status"
+  grep -Fq 'error: could not create output directory' "$output" || fail "$name create failure" "missing output-directory error"
+}
+
+assert_mktemp_failure() {
+  local name="$1" script="$2"
+  shift 2
+  local marker="$work/$name.model-invoked"
+  local output="$work/$name.mktemp-failure.out"
+  local status
+  rm -f "$marker"
+  set +e
+  AB_SCRIPTS_OUTDIR_TEST_MODEL_MARKER="$marker" TMPDIR="$work/not-a-directory/missing" "$script" "$@" > "$output" 2>&1
+  status=$?
+  set -e
+  [[ "$status" -eq 2 ]] || fail "$name mktemp failure" "expected exit 2, got $status"
+  grep -Fq 'error: could not create output directory' "$output" || fail "$name mktemp failure" "missing output-directory error"
+  [[ ! -e "$marker" ]] || fail "$name mktemp failure" "model binary was invoked after output-directory creation failed"
+}
+
+mkdir -p "$work/tmp" "$work/bin"
+write_fake_mlxcel "$work/baseline" baseline
+write_fake_mlxcel "$work/arm" arm
+write_fake_curl "$work/bin/curl"
+write_fake_jq "$work/bin/jq"
+printf 'not a directory\n' > "$work/not-a-directory"
+
+set +e
+output="$(TMPDIR="$work/tmp" "$output_equality" --baseline "$work/baseline" --arm "$work/arm" --model models/fake --prompt test -n 1 2>&1)"
+status=$?
+set -e
+if [[ "$status" -eq 0 ]]; then
+  assert_default_dir "output equality default" ab_output_equality "$output"
+else
+  fail "output equality default" "expected exit 0, got $status"
+  sed 's/^/    /' <<< "$output" >&2
+fi
+assert_create_failure "output equality" "$output_equality" --baseline "$work/baseline" --arm "$work/arm" --model models/fake --prompt test -n 1
+assert_mktemp_failure "output equality" "$output_equality" --baseline "$work/baseline" --arm "$work/arm" --model models/fake --prompt test -n 1
+
+set +e
+output="$(PATH="$work/bin:$PATH" TMPDIR="$work/tmp" "$paged_counter" --baseline "$work/baseline" --arm "$work/arm" --model models/fake --requests 1 -n 1 2>&1)"
+status=$?
+set -e
+if [[ "$status" -eq 0 ]]; then
+  assert_default_dir "paged counter default" paged_decode_counter_ab "$output"
+else
+  fail "paged counter default" "expected exit 0, got $status"
+  sed 's/^/    /' <<< "$output" >&2
+fi
+assert_create_failure "paged counter" "$paged_counter" --baseline "$work/baseline" --arm "$work/arm" --model models/fake --requests 1 -n 1
+assert_mktemp_failure "paged counter" "$paged_counter" --baseline "$work/baseline" --arm "$work/arm" --model models/fake --requests 1 -n 1
+
+if [[ "$failures" -ne 0 ]]; then
+  echo "A/B output-directory regression test: $failures case(s) failed" >&2
+  exit 1
+fi
+
+echo "A/B output-directory regression test: all cases passed"

scripts/paged_decode_counter_ab.sh

@@ -61,7 +61,7 @@
 # Exit status: 0 both sides launched the fused path the same number of times
 # with no fallbacks and produced the same text; 1 the counters disagree, a side
 # never launched the fused path, or the text differs; 2 the run could not be
-# made meaningful (server never came up, metrics unavailable).
+# made meaningful (server never came up, metrics unavailable, output directory unavailable).
 set -uo pipefail
 
 BASELINE_BIN=""
@@ -107,8 +107,16 @@ if cmp -s "$BASELINE_BIN" "$ARM_BIN"; then
   exit 1
 fi
 
-[[ -n "$OUT_DIR" ]] || OUT_DIR="$(mktemp -d -t paged_decode_counter_ab)"
-mkdir -p "$OUT_DIR"
+if [[ -z "$OUT_DIR" ]]; then
+  OUT_DIR="$(mktemp -d "${TMPDIR:-/tmp}/paged_decode_counter_ab.XXXXXX")" || {
+    echo "error: could not create output directory" >&2
+    exit 2
+  }
+fi
+if [[ -z "$OUT_DIR" ]] || ! mkdir -p "$OUT_DIR"; then
+  echo "error: could not create output directory" >&2
+  exit 2
+fi
 
 echo "baseline: $BASELINE_BIN"
 echo "arm:      $ARM_BIN"