feat(top): the Queue view as a tree, and --wraps

CHANGELOG.md

@@ -10,6 +10,19 @@ and the versions follow [semver](https://semver.org).
 
 ## [Unreleased]
 
+### Added
+
+- `taskguard top`: the Queue view is a tree. The jobs of one task-runner
+  run sit under one row, named by the command that started it, with their
+  counts, CPU and memory. The taskguard calls running inside a job, such as a
+  dev stack's start-up builds, show under that job with their own CPU and
+  memory. `←` folds a row, `→` unfolds it, and `Enter` on a run's row does
+  both. Running jobs keep their place,
+  oldest first.
+- `--wraps`: a command that runs other taskguard jobs runs straight through
+  from its first run, and the jobs inside it queue on their own. A queued run
+  that had jobs inside it prints advice to pass it.
+
 ### Changed
 
 - A command that wraps other taskguard jobs, such as `taskguard bun run

README.md

@@ -322,7 +322,9 @@ taskguard jobs inside them runs straight through, with one line saying so. The
 jobs inside it then queue on their own, each by its own need. A long-lived job,
 a job in a pool and a job with a minimum stay queued: their claim is on
 purpose. `taskguard prune 'KEY' --apply` forgets the history, so the command
-queues again.
+queues again. When a queued run had jobs inside it, taskguard prints advice to
+pass `--wraps`: `taskguard --wraps bun run ci:local` runs straight through
+from the first run, with nothing to learn first.
 
 Long-running commands are never queued: watch modes, dev servers, `--version`
 and `--help` run straight through (the list is in `src/builtin.rs`). Add your
@@ -420,7 +422,7 @@ run, and the minimum to pin when a job starves again and again.
 | View | What it shows |
 | --- | --- |
 | 1 Overview | Machine CPU and memory over time. The load of jobs taskguard started is stacked in colour per namespace. The load it did not start is split into groups: agents (with everything they started), browsers, editors, dev services such as databases, containers, and chat and other apps. The legend names the biggest programs in each group, so you can see what to close to make room. A strip shows how many jobs waited, coloured by reason, and `!` marks a starved run. |
-| 2 Queue | Running jobs, then waiting ones. The bars show in red the load that running jobs have above their estimates. The Why panel checks every rule for the selected job, with numbers, and says what will unblock it and when. `g` starts a waiting job now, and `e` sets a job's needs for this run only (for example `2 4G`), so it can fit into room you know it will not outgrow. |
+| 2 Queue | Running jobs, then waiting ones, as a tree: the jobs of one task-runner run sit under one row named by the command that started it (`bun run ci:local`), and the taskguard calls running inside a job (a dev stack's start-up builds) under that job, with their CPU and memory. `←` folds a row, `→` unfolds it, and `Enter` on a run's row does both. The bars show in red the load that running jobs have above their estimates. The Why panel checks every rule for the selected job, with numbers, and says what will unblock it and when. `g` starts a waiting job now, and `e` sets a job's needs for this run only (for example `2 4G`), so it can fit into room you know it will not outgrow. |
 | 3 Runs | Past runs, sortable and filterable, with the reasons each one waited. |
 | 4 Tasks | Every task taskguard knows, with its namespace, runs, last run, and what it learned. `n` (namespace), `/` (text) and `a` (not run for 1, 7, 30 or 90 days) filter the list. `x` prunes every run of the selected task; `X` prunes every task the filters leave, and needs a filter. Both ask first. Pruned runs move aside, as with `taskguard prune`, and `taskguard prune --undo --apply` puts them back. In a job's view, `x` prunes the selected run. |
 | 5 Trends | Commands whose memory, CPU or duration grows: the last 10 runs against the 10 before. |

src/config.rs

@@ -265,7 +265,7 @@ pub enum SettingKind {
     Size(u64, u64, u64),
 }
 
-pub const SETTINGS: [Setting; 13] = [
+pub const SETTINGS: [Setting; 14] = [
     Setting {
         key: "enabled",
         what: "off: every command runs straight through, unqueued and unmeasured (as TASKGUARD_DISABLE=1)",
@@ -283,9 +283,14 @@ pub const SETTINGS: [Setting; 13] = [
     },
     Setting {
         key: "pressure_max",
-        what: "Linux: no new job while memory pressure (PSI) is above this",
+        what: "no new job at this memory pressure (Linux: PSI share; macOS: 50 warning, 100 critical)",
         kind: SettingKind::Number(5.0, 5.0, 90.0, "%"),
     },
+    Setting {
+        key: "warn_admit",
+        what: "macOS warning level: jobs still start while memory fits and no first run still grows",
+        kind: SettingKind::Bool,
+    },
     Setting {
         key: "auto_pause",
         what: "pause the newest running job when memory reaches pause_at; resume it under resume_at",
@@ -335,6 +340,7 @@ impl Config {
             "enabled" => (if self.enabled { "on" } else { "off" }).into(),
             "hints" => (if self.hints { "on" } else { "off" }).into(),
             "auto_pause" => (if self.auto_pause { "on" } else { "off" }).into(),
+            "warn_admit" => (if self.warn_admit { "on" } else { "off" }).into(),
             "pause_at" => format!("{:.0}%", self.pause_at),
             "resume_at" => format!("{:.0}%", self.resume_at),
             "max_bypass" => format!("{}s", self.max_bypass),
@@ -351,6 +357,7 @@ impl Config {
             "enabled" => self.enabled,
             "hints" => self.hints,
             "auto_pause" => self.auto_pause,
+            "warn_admit" => self.warn_admit,
             _ => false,
         }
     }

src/help.rs

@@ -13,6 +13,8 @@ options (sem style; the default is to run in the foreground):
   --min-mem SIZE          never start with less than SIZE free (6G, 512M)
   --priority N            higher starts first (default 0; negative is allowed)
   --now                   skip the queue, but still measure and learn
+  --wraps                 COMMAND runs other taskguard jobs (turbo, a package
+                          script): run it straight through and let them queue
   --bg                    wait for room, then return and let the job run on
   --fg                    run in the foreground (the default)
   --pipe                  with --bg: pass stdin to the command
@@ -51,8 +53,12 @@ taskguard top [--view NAME] [--range RANGE] [--job KEY] [--print [--width N] [--
 
 The dashboard: the machine, the queue, past runs, every known task, trends
 and warnings. Inside it, ? lists the keys of the view you are in. In the
-Tasks view (tab 4), filter tasks and prune their history: x one task, X every
-task shown.
+Queue view (tab 2), the jobs of one task-runner run (a `turbo run`) sit under
+one row named by the command that started it, and the taskguard calls running
+inside a job sit under that job: ← folds a row, → unfolds it, and Enter on a
+run's row folds or unfolds it. In the Tasks
+view (tab 4), filter tasks and prune their history: x one task, X every task
+shown.
 
 options:
   --view NAME     open on a view: overview, queue, runs, tasks, trends,

src/main.rs

@@ -122,6 +122,7 @@ pub fn parse_opts(args: &[String]) -> Result<Opts> {
                 o.priority = Some(v.parse().map_err(|_| anyhow::anyhow!("--priority needs a whole number"))?);
             }
             "--now" => o.now = true,
+            "--wraps" => o.wraps = true,
             "--bg" => o.bg = true,
             "--fg" => o.bg = false,
             "--pipe" => o.pipe = true,

src/queue.rs

@@ -12,6 +12,15 @@ use serde::{Deserialize, Serialize};
 use std::fs::{self, File, OpenOptions};
 use std::path::{Path, PathBuf};
 
+/// A taskguard call that runs inside a running job, under its claim.
+#[derive(Debug, Clone, PartialEq)]
+pub struct Nested {
+    pub outer: i32,
+    pub pid: i32,
+    pub since: f64,
+    pub key: String,
+}
+
 /// One waiting or running job, as a JSON file in `wait/` or `run/`.
 ///
 /// Every taskguard version on the machine reads these files, and DALP pins a
@@ -287,9 +296,29 @@ impl Queue {
 
     /// A taskguard call nested inside the running job of `outer` (a pid)
     /// leaves `nested/<outer>.<pid>`, so that job learns that it wraps other
-    /// taskguard jobs.
-    pub fn note_nested(&self, outer: i32, pid: i32) {
-        let _ = fs::write(self.dir.join("nested").join(format!("{outer}.{pid}")), "");
+    /// taskguard jobs. The note holds the call's job key, for the tree in
+    /// `taskguard top`; when it was written is when the call started.
+    pub fn note_nested(&self, outer: i32, pid: i32, key: &str) {
+        let _ = fs::write(self.dir.join("nested").join(format!("{outer}.{pid}")), key);
+    }
+
+    /// Every nested call with a note: the outer job's pid, the call's pid,
+    /// when it started, and its key (empty from a note without one).
+    pub fn nested(&self) -> Vec<Nested> {
+        let Ok(rd) = fs::read_dir(self.dir.join("nested")) else { return Vec::new() };
+        rd.filter_map(|e| e.ok())
+            .filter_map(|e| {
+                let name = e.file_name().into_string().ok()?;
+                let (outer, pid) = name.split_once('.')?;
+                let since = e.metadata().ok()?.modified().ok()?.duration_since(std::time::UNIX_EPOCH).ok()?.as_secs_f64();
+                Some(Nested {
+                    outer: outer.parse().ok()?,
+                    pid: pid.parse().ok()?,
+                    since,
+                    key: fs::read_to_string(e.path()).unwrap_or_default(),
+                })
+            })
+            .collect()
     }
 
     /// How many nested calls left a note for the job of `owner`. The notes
@@ -2288,9 +2317,13 @@ mod tests {
     fn nested_calls_leave_notes_for_their_outer_job() {
         let tmp = tempfile::tempdir().unwrap();
         let q = Queue::open(tmp.path()).unwrap();
-        q.note_nested(100, 201);
-        q.note_nested(100, 202);
-        q.note_nested(1000, 203);
+        q.note_nested(100, 201, "a:tsc");
+        q.note_nested(100, 202, "b:tsc");
+        q.note_nested(1000, 203, "c:lint");
+        let mut seen: Vec<(i32, i32, String)> = q.nested().into_iter().map(|n| (n.outer, n.pid, n.key)).collect();
+        seen.sort();
+        assert_eq!(seen[0], (100, 201, "a:tsc".to_string()));
+        assert_eq!(seen.len(), 3);
         assert_eq!(q.take_nested(100), 2, "only its own notes, not those of pid 1000");
         assert_eq!(q.take_nested(100), 0, "taken once");
         assert_eq!(q.take_nested(1000), 1);

src/runner.rs

@@ -34,6 +34,9 @@ pub struct Opts {
     pub min_cpu: Option<f64>,
     pub min_mem_kb: Option<u64>,
     pub now: bool,
+    /// The command wraps other taskguard jobs: it runs straight through, and
+    /// the jobs inside it queue on their own.
+    pub wraps: bool,
     pub bg: bool,
     pub pipe: bool,
     pub quiet: bool,
@@ -99,14 +102,34 @@ fn held_by_ancestor(q: &Queue) -> bool {
 
 /// Tell the running job this nested call runs inside that it wraps other
 /// taskguard jobs: see `queue::Queue::note_nested`.
-fn note_nested(q: &Queue) {
+fn note_nested(q: &Queue, o: &Opts) {
     let procs = sys::list_procs();
     let me = std::process::id() as i32;
     if let Some(outer) = sys::ancestors(me, &procs).into_iter().skip(1).find(|p| q.run_path(*p).exists()) {
-        q.note_nested(outer, me);
+        let key = match (&o.key, std::env::current_dir()) {
+            (Some(k), _) => k.clone(),
+            (None, Ok(cwd)) => key::run_key(&cwd, &o.cmd),
+            (None, Err(_)) => String::new(),
+        };
+        q.note_nested(outer, me, &key);
     }
 }
 
+/// A command as one line to paste into a shell: words with spaces or
+/// shell characters in single quotes.
+fn shell_words(cmd: &[String]) -> String {
+    cmd.iter()
+        .map(|w| {
+            if !w.is_empty() && w.chars().all(|c| c.is_ascii_alphanumeric() || "-_./:=@%+,".contains(c)) {
+                w.clone()
+            } else {
+                format!("'{}'", w.replace('\'', "'\\''"))
+            }
+        })
+        .collect::<Vec<_>>()
+        .join(" ")
+}
+
 /// Task runners whose run groups the jobs it starts. A job belongs to the
 /// nearest one above it.
 const LAUNCHERS: &[&str] = &["turbo", "nx", "make", "gmake", "moon", "lage", "just"];
@@ -196,7 +219,7 @@ pub fn run(mut o: Opts) -> Result<i32> {
         exec(&o.cmd);
     }
     if held_by_ancestor(&q) {
-        note_nested(&q);
+        note_nested(&q, &o);
         exec(&o.cmd);
     }
 
@@ -208,6 +231,12 @@ pub fn run(mut o: Opts) -> Result<i32> {
     if !cfg.enabled || cls.passthrough {
         exec(&o.cmd);
     }
+    if o.wraps {
+        if !o.quiet {
+            say(&format!("straight through {jkey} - --wraps: the taskguard jobs inside it queue on their own"));
+        }
+        exec(&o.cmd);
+    }
     if o.bg {
         return run_bg(&o);
     }
@@ -268,7 +297,7 @@ pub fn run(mut o: Opts) -> Result<i32> {
     {
         if !o.quiet {
             say(&format!(
-                "straight through {jkey} - its recent runs started up to {n} taskguard jobs inside it, and those queue on their own (taskguard prune '{jkey}' --apply queues it again)"
+                "straight through {jkey} - its recent runs started up to {n} taskguard jobs inside it, and those queue on their own; pass --wraps to say so up front (taskguard prune '{jkey}' --apply queues it again)"
             ));
         }
         exec(&o.cmd);
@@ -829,6 +858,13 @@ pub fn run(mut o: Opts) -> Result<i32> {
     } else {
         None
     };
+    let nested_jobs = q.take_nested(me_pid);
+    if nested_jobs > 0 && startup.is_none() && pool.is_none() && min_mem.is_none() && min_cpu.is_none() {
+        lines.advice(&[format!(
+            "{jkey} started {nested_jobs} taskguard jobs inside it, which ran under its claim instead of queuing on their own. Next time it runs straight through by itself; to say so up front, run: taskguard --wraps {}",
+            shell_words(&o.cmd)
+        )]);
+    }
     if let Some(d) = &database {
         let _ = d.finish_run(
             run_id,
@@ -847,7 +883,7 @@ pub fn run(mut o: Opts) -> Result<i32> {
                 measured,
                 paused_s: me.paused_s,
                 steady: tracker.steady(ended, me.paused_s - paused_before_steady),
-                nested_jobs: q.take_nested(me_pid),
+                nested_jobs,
             },
         );
     }

src/sys/linux.rs

@@ -139,6 +139,13 @@ pub fn proc_name(pid: i32) -> String {
     fs::read_to_string(format!("/proc/{pid}/comm")).map(|s| s.trim().to_string()).unwrap_or_else(|_| format!("pid {pid}"))
 }
 
+/// The command line of a process. None when it is gone or a kernel thread.
+pub fn proc_args(pid: i32) -> Option<Vec<String>> {
+    let raw = fs::read(format!("/proc/{pid}/cmdline")).ok()?;
+    let args: Vec<String> = raw.split(|b| *b == 0).filter(|a| !a.is_empty()).map(|a| String::from_utf8_lossy(a).into_owned()).collect();
+    (!args.is_empty()).then_some(args)
+}
+
 /// The program's path, for grouping processes that share a name.
 pub fn proc_path(pid: i32) -> Option<String> {
     fs::read_link(format!("/proc/{pid}/exe")).ok().map(|p| p.display().to_string())

src/sys/macos.rs

@@ -174,6 +174,33 @@ pub fn proc_name(pid: i32) -> String {
     String::from_utf8_lossy(&buf[..n as usize]).into_owned()
 }
 
+/// The command line of a process, from KERN_PROCARGS2: the argument count,
+/// the executable path, padding, then the arguments, each ending in a NUL.
+/// None when the process is gone or belongs to another user.
+pub fn proc_args(pid: i32) -> Option<Vec<String>> {
+    let mut argmax: libc::c_int = 0;
+    let mut len = size_of::<libc::c_int>();
+    let mut mib = [libc::CTL_KERN, libc::KERN_ARGMAX];
+    let r = unsafe { libc::sysctl(mib.as_mut_ptr(), 2, &mut argmax as *mut _ as *mut c_void, &mut len, std::ptr::null_mut(), 0) };
+    if r != 0 || argmax <= 0 {
+        return None;
+    }
+    let mut buf = vec![0u8; argmax as usize];
+    let mut len = buf.len();
+    let mut mib = [libc::CTL_KERN, libc::KERN_PROCARGS2, pid];
+    let r = unsafe { libc::sysctl(mib.as_mut_ptr(), 3, buf.as_mut_ptr() as *mut c_void, &mut len, std::ptr::null_mut(), 0) };
+    if r != 0 || len < size_of::<libc::c_int>() {
+        return None;
+    }
+    let argc = i32::from_ne_bytes(buf[..4].try_into().ok()?) as usize;
+    let rest = &buf[4..len];
+    // Skip the executable path and the NULs after it.
+    let path_end = rest.iter().position(|b| *b == 0)?;
+    let start = path_end + rest[path_end..].iter().position(|b| *b != 0)?;
+    let args: Vec<String> = rest[start..].split(|b| *b == 0).take(argc).map(|a| String::from_utf8_lossy(a).into_owned()).collect();
+    (!args.is_empty()).then_some(args)
+}
+
 /// The program's path. Some programs have a name that says little: Claude
 /// Code's process is named after its version, for example "2.1.282".
 pub fn proc_path(pid: i32) -> Option<String> {

src/sys/mod.rs

@@ -156,6 +156,24 @@ pub fn ancestors(pid: i32, procs: &[ProcInfo]) -> Vec<i32> {
 mod tests {
     use super::*;
 
+    #[test]
+    fn proc_args_reads_a_command_line() {
+        let mut child = std::process::Command::new("sleep").arg("30").spawn().unwrap();
+        // Right after the fork the child still has the parent's command line,
+        // until it execs sleep.
+        let read = || proc_args(child.id() as i32).filter(|a| a.last().is_some_and(|l| l == "30"));
+        let t0 = std::time::Instant::now();
+        let mut args = read();
+        while args.is_none() && t0.elapsed() < std::time::Duration::from_secs(2) {
+            std::thread::sleep(std::time::Duration::from_millis(20));
+            args = read();
+        }
+        child.kill().unwrap();
+        child.wait().unwrap();
+        assert_eq!(args.map(|a| a[1..].to_vec()), Some(vec!["30".to_string()]));
+        assert_eq!(proc_args(-1), None);
+    }
+
     #[test]
     fn descendants_walks_the_whole_tree() {
         let procs = [