Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions docs/fm-test-portable-shards.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,8 @@ Local timings are not interchangeable with CI timings: platform and machine load
Both hint tables were refreshed on 2026-09-30 from five Ubuntu CI runs: [36583881812](https://github.com/kunchenguid/firstmate/actions/runs/36583881812), [36658498535](https://github.com/kunchenguid/firstmate/actions/runs/36658498535), [36663947738](https://github.com/kunchenguid/firstmate/actions/runs/36663947738), [36664663190](https://github.com/kunchenguid/firstmate/actions/runs/36664663190), and [36669175457](https://github.com/kunchenguid/firstmate/actions/runs/36669175457).
Use the slowest successful `duration_ms` per script across their uploaded portable timing artifacts and completed `FM_TEST_END` log markers, with the two version/platform exceptions below.
All artifact records were cross-checked against the corresponding job's markers.
This covers all 24 parallel and 201 serial members; an existing live-capability skip is a portable-runner measurement, not a timing claim for the unavailable live integration.
Those runs are upstream's, and their records cover all 24 parallel members and the 201 serial members the lane held upstream; an existing live-capability skip is a portable-runner measurement, not a timing claim for the unavailable live integration.
This fork's serial lane carries additional fork-only members that no upstream run measured, so each of them packs on the `PORTABLE_SERIAL_DEFAULT_WEIGHT_MS` default until the fork refreshes its own hints from its own green CI runs; read the current lane size and unmeasured share from `bin/fm-test-run.sh --check-coverage` rather than from a count copied here.
Observed maxima provide conservative packing weights, not an upper bound on future durations.

Two serial-5 jobs were cancelled at their 30-minute cap and uploaded no artifact.
Expand Down Expand Up @@ -78,7 +79,7 @@ Refresh the CI-derived hints by downloading the per-shard timing artifacts from

```sh
for run in <run-id> <run-id> <run-id>; do
gh-axi run download "$run" -R kunchenguid/firstmate --dir "/tmp/fm-serial/$run"
gh-axi run download "$run" -R <owner>/firstmate --dir "/tmp/fm-serial/$run"
done
jq -r '.scripts[] | select(.exit == 0) | [.path, .duration_ms] | @tsv' /tmp/fm-serial/*/fm-test-timing-portable-serial-*/*.json \
| awk -F'\t' '$2 > m[$1] { m[$1] = $2 } END { for (p in m) print p, m[p] }' \
Expand Down
14 changes: 10 additions & 4 deletions tests/fm-session-lock-ancestry.test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -765,15 +765,21 @@ test_e2e_background_session_keeps_its_lock_across_a_recycled_chain() {
grep -qx "$frontend" "$dir/state/phase-1/ancestry" || fail "the healthy chain did not reach the front-end"
expect_phase_owned "$dir" 1 2 "$frontend" "healthy chain"

# Recycle the bridge: the daemon ends, the pty-host is reparented to init, and
# the front-end that holds the lock stays alive.
# Recycle the bridge: the daemon ends, the pty-host is orphaned to whatever
# reaper the host uses (init, or a user-session subreaper such as systemd
# --user), and the front-end that holds the lock stays alive.
kill -TERM "$daemon"
i=0
while [ "$i" -lt 200 ] && { kill -0 "$daemon" 2>/dev/null || [ "$(ps -o ppid= -p "$ptyhost" 2>/dev/null | tr -d ' ')" != 1 ]; }; do
ptyhost_ppid=$(ps -o ppid= -p "$ptyhost" 2>/dev/null | tr -d ' ')
while [ "$i" -lt 200 ] && [ "$ptyhost_ppid" = "$daemon" ]; do
sleep 0.05
i=$((i + 1))
ptyhost_ppid=$(ps -o ppid= -p "$ptyhost" 2>/dev/null | tr -d ' ')
done
[ "$(ps -o ppid= -p "$ptyhost" 2>/dev/null | tr -d ' ')" = 1 ] || fail "the pty-host was not reparented to init after the daemon ended"
kill -0 "$ptyhost" 2>/dev/null \
|| fail "the pty-host died with the daemon, so the recycled case cannot be exercised"
[ "$ptyhost_ppid" != "$daemon" ] \
|| fail "the pty-host is still parented by the daemon after it ended"
kill -0 "$frontend" 2>/dev/null || fail "the front-end died with the daemon, so the recycled case cannot be exercised"

# Phase 2: the same session id over the broken chain - the reported drift.
Expand Down
24 changes: 20 additions & 4 deletions tests/fm-test-run.test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -1754,10 +1754,11 @@ test_herdr_ci_family_run_has_a_step_timeout() {
# The required Herdr lane's hang tripwire is the family-run *step* bound, not
# the 75-minute job cap. Parse the workflow as YAML so nested `with.name`
# artifact keys cannot masquerade as the step contract.
command -v ruby >/dev/null 2>&1 \
|| fail "ruby is required to parse .github/workflows/ci.yml as YAML"
local json job_timeout step_timeout
json=$(ruby -ryaml -rjson -e '
# Ruby's YAML is the CI runner's parser; fall back to PyYAML on hosts that
# ship python3 without ruby so the contract still runs there.
if command -v ruby >/dev/null 2>&1; then
json=$(ruby -ryaml -rjson -e '
doc = YAML.load_file(ARGV[0])
job = doc.fetch("jobs").fetch("tests-herdr")
step = job.fetch("steps").find { |s|
Expand All @@ -1770,7 +1771,22 @@ puts JSON.generate(
"step_timeout" => step.fetch("timeout-minutes")
)
' "$ROOT/.github/workflows/ci.yml") \
|| fail "could not parse tests-herdr timeouts from ci.yml"
|| fail "could not parse tests-herdr timeouts from ci.yml"
elif python3 -c 'import yaml' >/dev/null 2>&1; then
json=$(python3 - "$ROOT/.github/workflows/ci.yml" <<'PY'
import json, sys, yaml
doc = yaml.safe_load(open(sys.argv[1], encoding="utf-8"))
job = doc["jobs"]["tests-herdr"]
step = next(s for s in job["steps"]
if isinstance(s, dict)
and s.get("name") == "Run real-Herdr family (serial, required)")
print(json.dumps({"job_timeout": job["timeout-minutes"],
"step_timeout": step["timeout-minutes"]}))
PY
) || fail "could not parse tests-herdr timeouts from ci.yml"
else
fail "ruby or python3 with PyYAML is required to parse .github/workflows/ci.yml as YAML"
fi
job_timeout=$(python3 -c 'import json,sys; print(json.load(sys.stdin)["job_timeout"])' <<<"$json") \
|| fail "could not read job timeout from parsed workflow"
step_timeout=$(python3 -c 'import json,sys; print(json.load(sys.stdin)["step_timeout"])' <<<"$json") \
Expand Down
Loading