Make --check JSON agree with its exit code; show recipe-less outputs in status

docs/api/materialize.md

@@ -16,7 +16,7 @@ driver's stderr, independently of success or failure, leaving stdout for the rep
 |---|---|
 | `materialize(root, targets, *, cluster_id, refresh)` | Project checks → graph → cluster connection → fetch/converge → schedule → save/restore → crate converge. |
 | `check(root, targets, *, refresh)` | The same classification without executing, committing, or fetching. Exempt from the dirty refusal. |
-| `status(root)` | The report: every output's state and provenance commit, plus the mode/image/sandbox header facts. |
+| `status(root)` | The report: every output's state and provenance commit, including `no recipe` outputs, plus the mode/image/sandbox header facts. |
 | `MaterializeReport` / `StatusReport` | The JSON surfaces; `ok` and `up_to_date` first. |
 | `cluster_for_run(cluster_id)` | Borrow the cluster; expose resource validation, submission, and completion. |
 | `run_record(...)` / `datalad_run_subject(...)` | The commit message `datalad rerun` replays, and the one spelling of its subject line — shared with the foreign-write comparator, because two strings here would drift. |
@@ -59,9 +59,10 @@ driver's stderr, independently of success or failure, leaving stdout for the rep
   measured-safe (the clean filter renames over the path, which never
   stops existing) and must not be "fixed" by moving the save into the
   task.
-- **`up_to_date` is `ok and not made and not planned`** — a run where
-  every recipe failed must not report "nothing to do", and `behind`
-  never counts against it.
+- **`ok` is a successful run or a passed check gate** — execution requires
+  no failures or blocked tasks; check mode also requires an empty `planned`
+  mapping. `up_to_date` means `ok and not made`; a failed run must not report
+  "nothing to do", and `behind` never counts against it.
 - **A read-only verb never tracebacks.** Anything `check`/`status`
   cannot read classifies as "will be remade" and the real error
   belongs to the recipe that follows.

docs/api/plan.md

@@ -13,7 +13,7 @@ Source: `src/lightcone/engine/plan.py`.
 | Symbol | Role |
 |---|---|
 | `build(root)` | Validate the spec with ASTRA's own validators, resolve every universe, return the `Graph`. |
-| `Graph` | Tasks keyed on `(universe_id, output_id)`; `order()` for the read-only topological walk, `resolve(targets)` for what a user typed, `closure(keys)` to narrow a run. |
+| `Graph` | Recipe tasks keyed on `(universe_id, output_id)` plus `no_recipe` keys retained for status; `order()` for the read-only topological walk, `resolve(targets)` for what a user typed, `closure(keys)` to narrow a run. |
 | `Task` | One output in one universe, frozen, retaining ASTRA's resource declaration in `resources`. |
 | `declared_path(root, path)` | The one rule that names a path: project-relative inside the tree, absolute outside, never resolved. |
 
@@ -31,6 +31,9 @@ Source: `src/lightcone/engine/plan.py`.
   schema, file, and universe validators before resolving anything —
   resolution answers what a *valid* spec means and does not re-check
   that it is one.
+- **Outputs without recipes remain reportable.** `Graph.no_recipe` lists active
+  `(universe, output)` pairs ASTRA resolved without a command. They are not tasks
+  and cannot be materialized; `lc status` shows them as `no recipe`.
 - **Resource declarations survive resolution.** `build` reads
   `recipe.resources` from ASTRA's resolved output definition and preserves the
   mapping. A valid declaration remains readable by `status` and

docs/cli/materialize.md

@@ -121,10 +121,12 @@ operation, and stronger consent than a flag.
 }
 ```
 
-The first two keys are the ones to branch on: `ok` — everything
-attempted finished; `up_to_date` — nothing needed doing (a failed run
-is never up to date, and `behind` outputs don't count against it).
-`planned` is check mode's answer, mapping each would-run output to why;
+The first two keys are the ones to branch on. `ok` is the command result:
+in execution mode, every attempted output finished; in check mode, no
+output would run. A failed run or a check with planned work returns
+`ok: false` and exits 1. `up_to_date` means nothing was or needs to be
+done (a failed run is never up to date, and `behind` outputs do not count
+against it). `planned` is check mode's answer, mapping each would-run output to why;
 `behind` maps each left-alone output to the commit that can rebuild its
 environment. `notes` carries sandbox messages verbatim — denial
 remedies are built to be pasted.

docs/cli/status.md

@@ -31,12 +31,13 @@ The header is repository facts: which mode the project executes in
 what enforcement a run on this host would get. No runtime and no
 network is needed to answer either.
 
-Then one line per output the spec declares, in dependency order: its
-state, **the commit it was made at**, and — for anything not current —
-why. The commit column is the verb's reason to exist: "which code made
-this?" has an answer for a current output too, and for a `behind`
-output that commit is where the environment that produced it can be
-read back.
+Then one line per output the spec declares: recipe outputs appear in
+dependency order, and outputs without recipes appear as `no recipe`.
+Each line gives its state, **the commit it was made at**, and — for
+stale or behind outputs — why. The commit column is the verb's reason to
+exist: "which code made this?" has an answer for a current output too,
+and for a `behind` output that commit is where the environment that
+produced it can be read back.
 
 ## States
 
@@ -46,6 +47,7 @@ read back.
 - `stale` — contradicts the project: definition changed, an input's
   content changed, or the output was edited by hand since it was made
   (a *foreign write* — the offending commit is named).
+- `no recipe` — declared but not executable yet; add a recipe to make it.
 
 ## Report vs gate
 
@@ -67,7 +69,7 @@ eyes, check for exit codes.
   "mode": "direct",
   "image": null,
   "sandbox": "landlock (fs: declared, network: allowed)",
-  "counts": {"current": 4, "behind": 0, "stale": 0},
+  "counts": {"current": 4, "behind": 0, "stale": 0, "no recipe": 0},
   "outputs": [
     {
       "output": "baseline/fit",

src/lightcone/cli/commands.py

@@ -298,7 +298,7 @@ def build(as_json: bool) -> None:
     is_flag=True,
     help=(
         "Report what would run and why, without executing or committing "
-        "anything; exit 1 if anything is out of date."
+        "anything. Exit 1 if any output would run; in JSON, `ok` is false."
     ),
 )
 @click.option(
@@ -391,7 +391,7 @@ def materialize(
             click.echo("\n".join(["", *report.notes]), err=True)
         _render_materialize_output(report, root, dry_run=check_only)
 
-    if not report.ok or (check_only and not report.up_to_date):
+    if not report.ok:
         sys.exit(1)
 
 
@@ -441,7 +441,10 @@ def status(as_json: bool) -> None:
     lines.append(f"  sandbox: {escape(report.sandbox)}")
     lines.append(f"  crate:   {escape(report.crate)}")
     lines.append("")
-    marks = {"current": "[dim]·[/dim]", "behind": "[cyan]·[/cyan]", "stale": "[yellow]![/yellow]"}
+    marks = {
+        "current": "[dim]·[/dim]", "behind": "[cyan]·[/cyan]",
+        "stale": "[yellow]![/yellow]", "no recipe": "[yellow]·[/yellow]",
+    }
     width = max((len(o.output) for o in report.outputs), default=0)
     # The commit gets a column of its own, for every state and not only
     # the interesting ones: "which code made this" is the question the
@@ -450,15 +453,15 @@ def status(as_json: bool) -> None:
     # in `why`, so this one path covers it; the dedicated field exists
     # for machine consumers of `--json`.
     lines += [
-        f"  {marks[o.status]} {o.status:<8} {o.output:<{width}}  "
+        f"  {marks[o.status]} {o.status:<9} {o.output:<{width}}  "
         f"{o.git_sha[:7] or '—':<7}" + (f"  [dim]{escape(o.why)}[/dim]" if o.why else "")
         for o in report.outputs
     ]
     lines += [f"  [yellow]![/yellow] {escape(warning)}" for warning in report.warnings]
 
     counts = report.counts
     if not report.outputs:
-        lines.append("[dim]The analysis declares no output with a recipe.[/dim]")
+        lines.append("[dim]The analysis declares no output.[/dim]")
     else:
         lines.append("")
         lines.append(
@@ -499,7 +502,7 @@ def _render_materialize_output(report: MaterializeReport, root: Path, *, dry_run
     lines += [f"  [red]✗[/red] blocked {name}" for name in report.blocked]
     lines += [f"  [yellow]![/yellow] {escape(warning)}" for warning in report.warnings]
 
-    if not report.ok:
+    if report.failed or report.blocked:
         verdict = f"[red]✗[/red] {where} did not finish"
     elif report.up_to_date:
         verdict = f"[green]✓[/green] {where} is up to date — nothing to do"

src/lightcone/engine/materialize.py

@@ -38,7 +38,7 @@
 from contextlib import contextmanager
 from dataclasses import asdict, dataclass, field, replace
 from pathlib import Path
-from typing import TYPE_CHECKING, Any, Protocol
+from typing import TYPE_CHECKING, Any, Literal, Protocol
 from uuid import uuid4
 
 from lightcone.engine import assets, container, dataset, identity, plan, project, worker
@@ -80,8 +80,8 @@ class MaterializeReport:
 
     @property
     def ok(self) -> bool:
-        """Whether everything that was attempted finished."""
-        return not self.failed and not self.blocked
+        """Whether the run finished or the check gate passed."""
+        return not self.failed and not self.blocked and not self.planned
 
     @property
     def up_to_date(self) -> bool:
@@ -97,7 +97,7 @@ def up_to_date(self) -> bool:
         so without ``ok`` here the first two keys of the JSON report would
         read "nothing to do" over a list of failures.
         """
-        return self.ok and not self.made and not self.planned
+        return self.ok and not self.made
 
     def as_dict(self) -> dict[str, Any]:
         """Return the report as JSON-ready data.
@@ -138,7 +138,8 @@ def check(root: Path, targets: Sequence[str], *, refresh: bool = False) -> Mater
             read, or a target matches nothing.
     """
     report = MaterializeReport()
-    for key, verdict, _, _ in _classified(root, targets, report, refresh=refresh):
+    _, classified = _classified(root, targets, report, refresh=refresh)
+    for key, verdict, _, _ in classified:
         name = _name(key)
         if verdict.calls_for_a_remake(refresh=refresh):
             report.planned[name] = verdict.why
@@ -151,7 +152,10 @@ def check(root: Path, targets: Sequence[str], *, refresh: bool = False) -> Mater
 
 def _classified(
     root: Path, targets: Sequence[str], report: MaterializeReport, *, refresh: bool
-) -> list[tuple[Key, assets.Verdict, assets.Manifest | None, dataset.LastWrite | None]]:
+) -> tuple[
+    Graph,
+    list[tuple[Key, assets.Verdict, assets.Manifest | None, dataset.LastWrite | None]],
+]:
     """Classify every task in topological order, reading nothing but disk.
 
     The walk both read-only modes share, so there is one answer to "what
@@ -170,8 +174,8 @@ def _classified(
             decides whether their dependents see the sentinel.
 
     Returns:
-        One ``(key, verdict, manifest, foreign write)`` per task, upstream
-        first.
+        The graph and one ``(key, verdict, manifest, foreign write)`` per
+        task, upstream first.
     """
     graph, env_version, _ = _graph(root, targets, report)
     unfetched: set[str] = set()
@@ -186,7 +190,7 @@ def _classified(
             "`lc materialize` fetches declared inputs before executing "
             "recipes. Compute is required for outputs whose inputs cannot yet be checked."
         )
-    return classified
+    return graph, classified
 
 
 def _classify_graph(
@@ -287,8 +291,8 @@ class OutputStatus:
 
     #: ``universe/output_id``.
     output: str
-    status: assets.Status
-    #: Why, for ``stale`` and ``behind``. Empty for ``current``.
+    status: assets.Status | Literal["no recipe"]
+    #: Why, for ``stale`` and ``behind``. Empty for ``current`` and ``no recipe``.
     why: str
     #: The commit the output was materialized at, or empty if it never was.
     #: This is the whole point of the verb: an artifact that is behind is
@@ -344,7 +348,7 @@ class StatusReport:
     @property
     def counts(self) -> dict[str, int]:
         """How many outputs are in each state, states with none included."""
-        tally = {"current": 0, "behind": 0, "stale": 0}
+        tally = {"current": 0, "behind": 0, "stale": 0, "no recipe": 0}
         for output in self.outputs:
             tally[output.status] += 1
         return tally
@@ -392,7 +396,8 @@ def status(root: Path) -> StatusReport:
         result.image = {"tag": tag, "state": state, "archive": archive}
     result.sandbox = _sandbox_line(result.mode)
     stamps = []
-    for key, verdict, manifest, foreign in _classified(root, [], report, refresh=False):
+    graph, classified = _classified(root, [], report, refresh=False)
+    for key, verdict, manifest, foreign in classified:
         if manifest and manifest.finished_at:
             stamps.append(manifest.finished_at)
         result.outputs.append(
@@ -405,6 +410,10 @@ def status(root: Path) -> StatusReport:
                 foreign_write=foreign.sha if foreign else "",
             )
         )
+    result.outputs.extend(
+        OutputStatus(output=_name(key), status="no recipe", why="", git_sha="", data_version="")
+        for key in graph.no_recipe
+    )
     result.crate = _crate_line(root, max(stamps, default=""))
     result.warnings = report.warnings
     return result

src/lightcone/engine/plan.py

@@ -4,7 +4,8 @@
 ``(universe, output)`` pair that has a recipe. A task carries everything
 executing it needs and nothing about *how* it will be executed: the
 rendered command, where its bytes go, what it reads, which decisions it
-was made under, and its ``definition_version``.
+was made under, and its ``definition_version``. Active outputs without a
+recipe are retained separately for status and are not executable tasks.
 
 What the spec *means* is ASTRA's to say. ``astra.resolve`` settles each
 universe's decisions, resolves every output's inputs to what supplies
@@ -73,9 +74,11 @@ def depends_on(self) -> tuple[Key, ...]:
 
 @dataclass(frozen=True)
 class Graph:
-    """Every task a run could make, and how they relate."""
+    """Every task a run could make, and outputs with no recipe."""
 
     tasks: dict[Key, Task]
+    #: Active outputs ASTRA resolved without a recipe; reportable, not executable.
+    no_recipe: tuple[Key, ...] = ()
 
     def order(self) -> list[Key]:
         """Return the tasks in dependency order.
@@ -157,8 +160,7 @@ def build(root: Path) -> Graph:
         root: The project root.
 
     Returns:
-        One task per ``(universe, output)`` pair that has a recipe and is
-        active in that universe.
+        The active recipe tasks and keys of active outputs without recipes.
 
     Raises:
         ProjectError: If the spec is missing, declares no universe, gives
@@ -181,6 +183,7 @@ def build(root: Path) -> Graph:
     spec = dict(resolve_analysis_tree(load_yaml(spec_path), root))
 
     tasks: dict[Key, Task] = {}
+    no_recipe: list[Key] = []
     declared_in: dict[str, Path] = {}
     for path in universes:
         universe = load_yaml(path)
@@ -196,10 +199,12 @@ def build(root: Path) -> Graph:
                 f"results/{universe_id}/. Give each universe its own id."
             )
         declared_in[universe_id] = path
-        for task in _tasks(root, universe_id, spec, universe):
+        universe_tasks, universe_no_recipe = _tasks(root, universe_id, spec, universe)
+        for task in universe_tasks:
             tasks[task.key] = task
+        no_recipe.extend((universe_id, output_id) for output_id in universe_no_recipe)
 
-    return Graph(tasks=tasks)
+    return Graph(tasks=tasks, no_recipe=tuple(no_recipe))
 
 
 def declared_path(root: Path, path: Path) -> str:
@@ -271,13 +276,13 @@ def _tasks(
     universe_id: str,
     spec: dict[str, object],
     universe: dict[str, object],
-) -> list[Task]:
-    """Every task one universe contributes.
+) -> tuple[list[Task], list[str]]:
+    """Resolve one universe's tasks and outputs that cannot be executed.
 
     ``resolve_outputs`` has already dropped what this universe does not
-    produce, so the only filter left is whether an output carries a
-    command: a re-export names bytes another output makes, and making it
-    twice under two ids is not a thing to do.
+    produce. Outputs without a command are not scheduled. Declared-only
+    ones are returned for status; a re-export is not, because it names
+    bytes another output makes and that output is reported in its place.
     """
     from astra.resolve import render_command, resolve_outputs
 
@@ -303,8 +308,11 @@ def file_of(out: object) -> Path:
         )
 
     tasks = []
+    no_recipe = []
     for out in resolved:
         if not out.command:
+            if out.reexports is None:
+                no_recipe.append(out.id)
             continue
         output_path = file_of(out)
         values: dict[str, str] = {}
@@ -351,4 +359,4 @@ def file_of(out: object) -> Path:
                 resources=resources,
             )
         )
-    return tasks
+    return tasks, no_recipe