Restructure the docs around one front door

.github/workflows/check-docs.yml

@@ -35,16 +35,18 @@ jobs:
             and complete given the code changes introduced by this PR.
 
             The documentation is structured, and each kind of change has a home:
-            - A verb's flags, output, JSON shape, or exit codes → its page in docs/cli/
-              (one page per verb, plus the overview's exit-code contract).
+            - A verb's flags, output, JSON shape, or exit codes → its page in
+              docs/reference/cli/ (one page per verb, plus the overview's exit-code
+              contract).
             - An engine module's responsibility, key symbols, or invariants → its page
-              in docs/api/ (hand-written module tours) and, for cross-cutting shifts,
-              docs/architecture.md.
+              in docs/developers/internals/ (hand-written module tours) and, for
+              cross-cutting shifts, docs/developers/architecture.md.
             - User-visible behavior (scaffold contents, states, refusal messages,
-              environment model, SLURM, publication) → docs/user/ (getting-started
-              quotes real console output; troubleshooting quotes real refusals) and
-              README.md's quick start.
-            - Test structure, dev workflow, or conventions → docs/contributing/.
+              environment model, SLURM, publication) → docs/get-started/ (the
+              Quickstart), docs/guides/ (guides/without-agent.md quotes real console
+              output), docs/concepts/, docs/reference/ (troubleshooting quotes real
+              refusals; glossary; installation) and README.md's quick start.
+            - Test structure, dev workflow, or conventions → docs/developers/contributing/.
 
             Steps to follow:
             1. Run: git diff --name-only ${{ github.event.pull_request.base.sha }} ${{ github.sha }}

CLAUDE.md

@@ -107,11 +107,12 @@ Each of these has been asked for in review at least once; none is optional.
 - **No dead code.** If nothing in the current layer calls it, it doesn't
   land yet. `lc --help` advertises only verbs that work.
 - **`docs/` is live again** (rewritten 2026-08, PRs #185–#188; the
-  freeze is over). The site is two tracks — user guide + developer
-  corner — and a change now lands with its docs: a new or changed verb
-  updates its `docs/cli/` page, an engine change updates its
-  `docs/api/` module page, and user-visible behavior updates the user
-  guide. The docs' own rules match this file's: document only what
+  freeze is over). The site's sections are Home, Tutorial, Guides,
+  Concepts, Reference and Developers (the nav is `zensical.toml`), and
+  a change now lands with its docs: a new or changed verb updates its
+  `docs/reference/cli/` page, an engine change updates its
+  `docs/developers/internals/` module page, and user-visible behavior
+  updates the Quickstart, guide or concept page it touches. The docs' own rules match this file's: document only what
   exists, quote refusals from real runs, and verify every command
   block by executing it. `check-docs.yml` reviews each merged PR for
   drift.

README.md

@@ -41,7 +41,7 @@ lc compute down "$CLUSTER"
 ASTRA specs are plain, structured YAML — they work well hand-written or
 drafted with any AI coding assistant.
 
-→ [Full getting-started guide](https://docs.lightconeresearch.org/user/getting-started/)
+→ [Full walkthrough](https://docs.lightconeresearch.org/guides/without-agent/)
 
 ## Capabilities
 

docs/concepts/analysis-file.md

@@ -0,0 +1,17 @@
+# Your analysis file
+
+`astra.yaml` is the one file that describes your analysis: what goes in, what comes out, which choices shape the results, and how each output is made.
+
+!!! abstract "Planned page"
+    What this page will cover:
+
+    - The sections of the file: `description`, `inputs`, `outputs`, `decisions`, `prior_insights`, `findings`, and nested `analyses`
+    - Outputs and recipes: `format`, `recipe.command`, and the `{inputs.<id>}`, `{decisions.<id>}` and `{output}` placeholders
+    - Why every dependency is declared: it is how `lc` orders the build and knows what to remake
+    - Who writes it: the agent drafts it as the scoping conversation settles, you review it, and it is validated on every save
+    - Splitting a large analysis into sub-analyses of the same shape
+    - What the file does not do: it describes the analysis; `lc` runs it
+
+    Draws on: [Use lc without an agent](../guides/without-agent.md#3-write-the-spec) (step 3), the specification and getting-started pages at astra-spec.org
+
+`astra.yaml` follows ASTRA, an open standard for describing analyses. Full format reference at [astra-spec.org](https://astra-spec.org/). How Lightcone builds on it: [Built on ASTRA](../developers/astra.md).

docs/concepts/evidence.md

@@ -0,0 +1,15 @@
+# Evidence
+
+Evidence ties a claim to something a reader can check: a quoted passage in a paper, or an output the analysis produced.
+
+!!! abstract "Planned page"
+    What this page will cover:
+
+    - Prior insights, claims from the literature that motivate a decision, and findings, claims the analysis itself makes
+    - An evidence item: a `doi` with the exact quoted text (and optionally a page), or an `artifact` naming an output
+    - The chain from option to insight to evidence to paper, and why it makes a decision auditable
+    - Quote verification: each quote is matched against a cached copy of the paper, so a citation that looks checkable is checked
+    - Evidence that points at an output is skipped, not failed, until that output has been made
+    - How the agent gathers citations, and where Lightcone Lab shows the cited papers
+
+    Draws on: [Cite papers and verify evidence](../guides/evidence.md), [Tutorial step 4](../tutorial/4-evidence.md), [Agent plugin](../reference/plugin.md), the specification's evidence sections and the getting-started guide at astra-spec.org

docs/user/concepts.md → docs/concepts/provenance.md

@@ -1,7 +1,7 @@
-# Core Concepts
+# Outputs and provenance
 
 The mental model behind `lc`, in one page. Nothing here is required to
-follow [Getting Started](getting-started.md) — come back when you want
+follow the [Quickstart](../get-started/quickstart.md) — come back when you want
 to know *why* the tool behaves the way it does.
 
 ## A project is three files
@@ -145,7 +145,7 @@ have.
 
 ## Where to next
 
-- [Running on a Cluster](cluster.md) — the same model on SLURM.
-- [Troubleshooting](troubleshooting.md) — the refusals quoted, with
+- [Run on a cluster](../guides/cluster.md) — the same model on SLURM.
+- [Troubleshooting](../reference/troubleshooting.md) — the refusals quoted, with
   their remedies.
-- [Glossary](glossary.md) — the terms, one at a time.
+- [Glossary](../reference/glossary.md) — the terms, one at a time.

docs/concepts/universes.md

@@ -0,0 +1,14 @@
+# Decisions and universes
+
+A decision is a methodological choice that could change a result; a universe is one complete set of choices, and each universe gets its own results.
+
+!!! abstract "Planned page"
+    What this page will cover:
+
+    - What deserves to be a decision, and its parts: `label`, `rationale`, `default` and `options`
+    - Universe files: `universes/<id>.yaml` picks one option per decision, and its results land in `results/<universe>/`
+    - How a choice reaches the code: as `{decisions.<id>}` in the recipe, so scripts take it as an argument and nothing is hard-coded
+    - Options that cannot go together (`requires`, `incompatible_with`), and rejected options kept on the record with their reason
+    - Comparing universes: adding one runs only its outputs, and a target like `robust/fit` narrows a run to one
+
+    Draws on: [Use lc without an agent](../guides/without-agent.md#6-sweep-the-decision) (step 6), [Add a decision and compare universes](../guides/decisions.md), [lc materialize](../reference/cli/materialize.md), the specification's Decisions, Options, Constraints and Universes sections at astra-spec.org

docs/architecture.md → docs/developers/architecture.md

@@ -1,8 +1,9 @@
 # Architecture
 
 How lightcone-cli is put together, for someone about to change it. The
-[user-guide concepts page](user/concepts.md) covers what the tool
-promises; this page covers how the promises are kept.
+[Outputs and provenance](../concepts/provenance.md) concepts page
+covers what the tool promises; this page covers how the promises are
+kept.
 
 ## The split that everything else follows
 
@@ -185,15 +186,15 @@ and one accelerator type/count. Providers translate those requests into native
 allocations; Lightcone does not depend on SkyPilot or carry its GPU alias registry.
 Stock Dask workers inherit the allocation's native CUDA mask; Lightcone does not
 probe GPU hardware. Container GPU access uses podman-hpc's native `--gpu` option.
-See [GPU setup](user/cluster.md#gpu-allocations).
+See [GPU setup](../guides/cluster.md#gpu-allocations).
 
 `compute.connect(CLUSTER_ID)` borrows a standard Dask client and closes only that
 client on exit. Both execution commands require a cluster ID. The materialization
 scheduler validates resource requests, then keeps its `submit`/`completed` seam.
 Driver preparation and existing task runtime/sandbox checks remain unchanged.
 Tasks use ordinary Dask scheduling;
 there is no separate worker-selection or preflight layer, or site-marker guard. No execution command implicitly allocates compute.
-See [compute internals](api/compute.md) and [deployment limits](user/cluster.md).
+See [compute internals](internals/compute.md) and [deployment limits](../guides/cluster.md).
 
 ## The publication view
 
@@ -228,5 +229,5 @@ src/lightcone/              # namespace — NO __init__.py
     └── templates/          # the scaffold's file content, as real files
 ```
 
-Each module's page in [Engine Internals](api/index.md) carries its
+Each module's page in [Engine internals](internals/index.md) carries its
 public surface and the invariants that bind it.

docs/developers/astra.md

@@ -0,0 +1,40 @@
+# Built on ASTRA
+
+Lightcone's analysis file, `astra.yaml`, follows ASTRA — the Agentic
+Schema for Transparent Research Analysis — an open standard for
+declaring a scientific analysis: its inputs, outputs, decisions, and
+the evidence behind them. ASTRA is a specification, not a runner. It
+says what an analysis is and stays out of execution; `lc` is the
+execution layer built on top of it. Everything about what a spec
+*means* — universe resolution, input references, conditional outputs,
+the recipe placeholder grammar — is answered by ASTRA's reference
+tooling rather than re-implemented in the engine (see
+[Architecture](architecture.md)).
+
+ASTRA is developed in the open, alongside Lightcone but separately from
+it, with its own site, schema and tools. The schema is written in
+LinkML. The `astra` CLI (the `astra-tools` package) validates specs,
+generates and checks universes, and caches papers and verifies quotes.
+A TypeScript SDK, `@astra-spec/sdk`, validates and resolves projects
+for viewers and integrations; [Lightcone Lab](../guides/lab.md) reads
+projects through it. Significant changes to the standard go through a
+public RFC process.
+
+You can work with ASTRA directly. If a spec resolves wrongly, the fix
+belongs in astra-tools, not in `lc`. If the format cannot express what
+your analysis needs, open an issue on astra-spec or propose an RFC.
+ASTRA is in early alpha, so reports from real analyses are the most
+useful input it can get.
+
+- [astra-spec.org](https://astra-spec.org) — the specification site:
+  concepts, elements, and a getting-started walkthrough.
+- [RFC process](https://astra-spec.org/latest/rfc-process/) — how a
+  proposal moves from idea to accepted; the RFCs themselves live in
+  [`astra-spec/rfcs`](https://github.com/LightconeResearch/astra-spec/tree/main/rfcs).
+- [LightconeResearch/astra-spec](https://github.com/LightconeResearch/astra-spec)
+  — the LinkML schema and the documentation source.
+- [LightconeResearch/astra-tools](https://github.com/LightconeResearch/astra-tools)
+  — the `astra` CLI and Python SDK
+  ([PyPI](https://pypi.org/project/astra-tools/)).
+- [LightconeResearch/astra-typescript](https://github.com/LightconeResearch/astra-typescript)
+  — the TypeScript SDK, published on npm as `@astra-spec/sdk`.

docs/maintainer.md → docs/developers/index.md

@@ -1,27 +1,29 @@
-# Developer corner
+# Developers
 
 `lightcone-cli` is a small engine with strong opinions: one way to
 identify an output, one way to store it, one boundary to execute it
-behind. This guide covers everything below the user surface — how the
-engine is put together, what each module owns, and how to get a
+behind. This section covers everything below the user surface — how
+the engine is put together, what each module owns, and how to get a
 working dev loop.
 
-If you're looking for the user-facing docs, the
-[user guide](user/index.md) is the other half of this site.
+If you're looking for the user-facing docs, start from the
+[home page](../index.md).
 
 ## What this covers
 
 - [Architecture](architecture.md) — the CLI/engine/ASTRA split, the
   run pipeline, identity, storage, the exec boundary, and the
   invariants that hold them together.
-- [CLI Reference](cli/index.md) — every `lc` command: flags, JSON
-  report shapes, exit codes.
-- [Engine Internals](api/index.md) — the `lightcone.engine.*`
+- [CLI reference](../reference/cli/index.md) — every `lc` command:
+  flags, JSON report shapes, exit codes.
+- [Engine internals](internals/index.md) — the `lightcone.engine.*`
   modules: what each owns, its key symbols, and what must stay true
   of it.
 - [Contributing](contributing/setup.md) — clone, install, run the
   test suite; [how the suite is shaped](contributing/testing.md); and
   [where a change belongs](contributing/extending.md).
+- [Built on ASTRA](astra.md) — the open standard the analysis file
+  follows, and where to engage with it directly.
 
 ## Get started in three commands
 

docs/api/compute.md → docs/developers/internals/compute.md

@@ -210,7 +210,7 @@ Docker or Podman are refused before image preparation; probes use a CPU policy a
 report that GPU access is unavailable while retaining their whole-worker reservation.
 Native permissions and cgroups remain authoritative. NVIDIA devices, including UVM,
 must already exist; policy construction does not load drivers or create devices.
-See [GPU deployment requirements](../user/cluster.md#gpu-allocations).
+See [GPU deployment requirements](../../guides/cluster.md#gpu-allocations).
 
 ## Execution output and teardown
 

docs/api/sandbox.md → docs/developers/internals/sandbox.md

@@ -36,7 +36,7 @@ The pure OCI rewrite adds podman-hpc's `--gpu` for GPU commands. Runtime selecti
 refuses explicit GPU recipes with ordinary Docker or Podman; probes on those
 runtimes receive a CPU policy and an explanatory note. CPU containers remain
 supported on all runtimes and set `NVIDIA_VISIBLE_DEVICES=void` to override image defaults.
-See [GPU allocations](../user/cluster.md#gpu-allocations).
+See [GPU allocations](../../guides/cluster.md#gpu-allocations).
 
 Container recipes and probes explicitly receive the worker's effective
 `OMP_NUM_THREADS`, `MKL_NUM_THREADS`, and `OPENBLAS_NUM_THREADS` values when set.

docs/get-started/how-it-works.md

@@ -0,0 +1,53 @@
+# How Lightcone works
+
+Three parts share one project directory: the agent plugin writes the analysis, `lc` runs it, and Lightcone Lab shows it.
+
+You describe what you want to learn. Your agent, guided by the plugin, writes it down as `astra.yaml`, [your analysis file](../concepts/analysis-file.md), with the scripts its recipes call. `lc materialize` runs each recipe in the project's locked environment, under a sandbox, and commits every output with a manifest of what made it ([outputs and provenance](../concepts/provenance.md)). Lab reads the same files and the same states `lc status` reports.
+
+```text
+you and your agent (with the plugin)
+        │ write
+        ▼
+astra.yaml + scripts
+        │ run by
+        ▼
+lc materialize ──▶ results/ + a manifest per output, committed
+                          │ browsed in
+                          ▼
+                    Lightcone Lab
+```
+
+## Why it's built this way
+
+Scientific results depend on methodological choices: which data to
+include, how to handle outliers, which prior to assume. In ordinary
+research code those choices are scattered across notebooks, scripts,
+comments, and memory, which makes results hard to reproduce, audit, and
+extend. Lightcone keeps the technical and the conceptual pieces of your
+work together: the code, data, and environment behind every result, and
+the decisions and evidence behind every choice. All of it is tracked,
+checked where it can be, and tied to each result as its provenance.
+
+## What you get
+
+- **Every choice on the record.** Decisions name the options that were
+  considered and why one was chosen. Claims taken from papers carry quotes
+  that are checked against the paper itself.
+- **Locked, isolated execution.** The environment is pinned, and recipes
+  run under a sandbox that keeps undeclared files out and stray writes
+  contained.
+- **Laptop to cluster.** The same project runs on your machine, in a
+  container, or across a SLURM allocation.
+- **Ready to publish.** Declare a license and Lightcone keeps an RO-Crate
+  of the project and its provenance up to date, ready to archive or
+  deposit.
+
+!!! abstract "Planned page"
+    What this page will cover:
+
+    - Which part does what, and which one you touch at each stage
+    - The project as a git repository: outputs committed with the code that made them
+    - The two paths, with your agent or by hand, and how they meet at `lc`
+    - Where the other concepts fit: [decisions and universes](../concepts/universes.md), [evidence](../concepts/evidence.md)
+
+    Draws on: [Outputs and provenance](../concepts/provenance.md), [Installation](../reference/installation.md), [Agent plugin](../reference/plugin.md)