Repository navigation
Jobs wait for memory that nobody will use #24
Description
Follow-up from #22 / #23. Goal: no job waits for a reason that is not real.
Measured on one MacBook (32 GB, DALP), from the local taskguard.db: 71,786 runs over 17 days, and the last 24 h of per-run memory samples. Memory was the main blocker for 44 job-hours of waiting, the largest of all blockers.
1. Wrapper jobs book their children's peak (largest)
Two commands wrap whole pipelines in one taskguard job. Jobs inside them do not queue (TASKGUARD_HELD), so the wrapper's peak is whatever ran inside it this time:
| key | runs | peak range | median |
|---|---|---|---|
root:sh -c git fetch -q origin main && git merge --no-edit o… |
40 | 0.1–20.2 GB | 3.0 GB |
root:bun run ci:local |
35 | 0.0–21.3 GB | 2.0 GB |
The need is the max of the last 10 peaks, so a tiny run still books up to 20 GB. Booked above the run's own peak: about 44 GB-hours, nearly all of the real excess.
Ideas: a wrapper books only its own processes and the inner jobs queue on their own; or a warning in taskguard doctor / the run output when a key's peaks span more than 10x; or advice to mark such commands passthrough.
2. The +25% boost after a starved run
385 runs carried the boost (mem_boosted). Booked above their own peak: 43.9 GB-hours; under it: 19.6 GB-hours. Most of it comes from the two keys above. Check how long one starved run keeps the boost.
3. First runs book too little (risk, not a wait)
2,074 first runs: 64.2 GB-hours under their real peak, 16.8 over. Not a wait, but it matters for admission at memory warning (#23 holds new jobs while a first run still grows).
Not worth it (measured)
- Booking only "what is still left" of a run, from past curves: only 3% of the booked-but-unused memory was for spikes that had already passed. Most big jobs peak near their end (
web/dappoxlint at 91–96% of the run). - Normal learned jobs are accurate: need / real peak has a median of 1.06, p90 1.64, and 14% of runs go above their need.
Not waste (on purpose)
min_memrules (e2e stacks, Canton): 22% of the excess. They book Docker memory that the job's own process tree does not show.- Long-lived stacks: my measure compared their start-up need with the whole run. After start-up their needs follow what they use.
🤖 Generated with Claude Code
Activity