(feat): Clarify API compatibility and planned features in docs with trademark attribution

.yfm

@@ -7,6 +7,10 @@ vars:
   product_description: "Observability Data Lake Engine"
   version: "0.1.0"
   repo_url: "https://github.com/icegatetech/icegate"
+  # Rust toolchain floor. Churns on every edition/MSRV bump and is quoted in 12 prose
+  # places across en/fr/ru, so it is the highest-churn literal in the corpus after the
+  # product name. Code blocks keep the literal (see AGENTS.md).
+  rust_version: "1.92.0"
   license: "Apache 2.0"
 
 # Markdown parsing options

AGENTS.md

@@ -24,6 +24,16 @@ npm run build:fr   # Build French only
 npm run build:ru   # Build Russian only
 ```
 
+**Do not drop `--static-content` from the build scripts.** Its help text ("allow loading custom
+resources into statically generated pages") undersells it: without the flag Diplodoc ships every
+page as an empty `<div id="root">` with the real content parked in a `diplodoc-state` JSON blob,
+so a crawler that does not run JavaScript sees ~4 words, no `<h1>`, and — because the TOC is
+rendered client-side too — no links to follow. Ahrefs found 4 of this site's pages for exactly
+that reason. With the flag, pages ship prerendered (~670 words and a real `<h1>` on a typical
+page) and the client bundle still hydrates on top, so nothing about the reading experience
+changes. Removing it breaks search and AI-crawler visibility site-wide, silently and with a
+green build.
+
 ## Project Structure
 
 ```
@@ -32,6 +42,7 @@ npm run build:ru   # Build Russian only
 ├── ru/                    # Russian documentation
 ├── llms.txt               # LLM context file — overview with key examples
 ├── llms-full.txt          # LLM context file — complete documentation content
+├── robots.txt             # Crawler policy; copied to the build root by `npm run build`
 ├── presets.yaml           # Build presets (default, development, production)
 ├── .yfm                   # Diplodoc configuration (vars, langs, settings)
 └── .yfmlint               # Linter rules configuration
@@ -72,7 +83,31 @@ When updating documentation, regenerate `llms-full.txt` after changes. `llms.txt
 
 ## Writing Documentation
 
-- Use variables from `.yfm` vars section: `{{product_name}}`, `{{version}}`, `{{repo_url}}`
+- **Use the `.yfm` vars in prose** — every mention outside code. The corpus is converted, so a
+  rename or version bump is a one-line edit in `.yfm` rather than a find-and-replace across
+  three languages. In use today: `{{product_name}}` (266 sites), `{{rust_version}}` (15),
+  `{{repo_url}}` (5), `{{license}}` (4).
+- **Adding a var is only worth it when the value appears in prose.** Measure before you add:
+  ports are the cautionary case — `3100` appears 37 times in prose but 131 times inside code
+  blocks, and since code must stay literal, a `{{loki_port}}` var would let prose and the
+  adjacent `curl` command disagree after a change. That is strictly worse than a literal,
+  because it *looks* single-sourced. Same verdict for `{{version}}` and the Helm OCI ref: code
+  only, so they stay defined but unused. `{{product_description}}` is title-case and every
+  prose site is mid-sentence lowercase, so it does not fit either.
+- **Never substitute inside code.** Fenced blocks, inline code, link targets and HTML
+  attributes keep the literal name — commands, image tags, hostnames (`icegate-query`),
+  datasource UIDs and `github.com/icegatetech/icegate` are identifiers, not prose, and a reader
+  copy-pasting `{{product_name}}` into a shell gets nothing useful.
+- **`llms.txt` and `llms-full.txt` must contain the literal name, never a variable.** The build
+  `cp`s them into `./build` verbatim, so yfm never renders them — a `{{product_name}}` there
+  ships raw to the LLM consumers the files exist for. 23 of them were doing exactly that.
+- The per-language scripts pass `-c ./.yfm`, and **the `./` is load-bearing**. `--help` says
+  relative config paths resolve from the execution directory and "other" paths from `--input`;
+  a bare `.yfm` counts as "other", so it resolves to `en/.yfm`, silently finds nothing, and the
+  build emits 121 "Variable not found" warnings while shipping raw `{{product_name}}` to disk.
+  `./.yfm` resolves from the repo root and works. `../.yfm` fails outright (ENOENT one
+  directory above the repo). Verify a change here by grepping the output for `{{`, not by
+  trusting the exit code — a config that fails to load is a warning, not an error.
 - HTML is allowed (`allowHTML: true`)
 - Files must end with newline (MD047 enforced)
 - Line length not enforced (MD013 disabled)
@@ -86,7 +121,10 @@ When updating documentation, regenerate `llms-full.txt` after changes. `llms.txt
 - Primary installation method: **Helm chart** (`oci://ghcr.io/icegatetech/charts/icegate`)
 - Development environment: **Skaffold** (`skaffold dev`) with Kustomize overlays
 - Docker Compose available as alternative for local development
-- Rust 1.92.0+ (2024 edition), 6 workspace crates: common, queue, query, ingest, maintain, jobmanager
+- Rust 1.92.0+ (2024 edition), 6 workspace crates: common, catalog-s3, queue, query, ingest, maintain. `jobmanager` is **not** a workspace crate — it lives in `icegatetech/jobmanager` and is consumed as a git-pinned dependency
+- Default catalog backend is IceGate's own S3 catalog (`backend: !s3`, state in `root.json`), not Nessie. Nessie/Glue/S3 Tables are alternatives
+- Default object store is **RustFS** (S3-compatible), not MinIO
+- Shift (WAL → Iceberg) lives in the **ingest** crate; compaction, orphan GC, and the LLM pricing crawler live in **maintain**
 - Metrics port: **9091** (not 9090). Prometheus API port is 9090.
 - Real environment variables: `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `OTEL_EXPORTER_OTLP_ENDPOINT`, `RUST_LOG`
 

c4/README.md

@@ -44,31 +44,57 @@ After running `make png`, the following files are created in `../assets/c4/`:
 | File | Description |
 |------|-------------|
 | `structurizr-SystemContext.png` | System context - IceGate and external systems |
-| `structurizr-Containers.png` | Container diagram - Services and storage |
+| `structurizr-Containers.png` | Container diagram - Services, libraries and storage |
 | `structurizr-IngestComponents.png` | Ingest Service internal components |
 | `structurizr-QueryComponents.png` | Query Service internal components |
 | `structurizr-QueueComponents.png` | Queue Library internal components |
 | `structurizr-MaintainComponents.png` | Maintain Service internal components |
+| `structurizr-CatalogComponents.png` | S3 Catalog internal components |
+| `structurizr-IngestionFlow.png` | Ingestion process (sequence) - OTLP request through shift to a committed snapshot |
+| `structurizr-QueryFlow.png` | Query process (sequence) - LogQL request to a merged WAL and Iceberg result |
+| `structurizr-MaintenanceFlow.png` | Maintenance process (sequence) - migration, compaction, orphan GC, pricing crawler |
 
 ## Workspace Structure
 
 ```
 workspace.dsl
 ├── Model
-│   ├── External Systems (OTel Collector, Grafana, Trino)
+│   ├── External Systems (OTel Collector/SDK, Grafana, BI & SQL clients,
+│   │                     Prometheus, tracing backend, Trino, LLM pricing feeds)
 │   └── IceGate System
-│       ├── Ingest Service (OTLP handlers, compactor)
-│       ├── Query Service (Loki/Prometheus/Tempo APIs)
-│       ├── Maintain Service (schema migrations)
-│       ├── Queue Library (WAL on S3)
-│       ├── Common Library (shared code)
-│       └── Storage (Queue, Iceberg, Catalog)
+│       ├── Ingest Service (OTLP handlers, transform, WAL writer, shift)
+│       ├── Query Service (Loki/Prometheus/Tempo/Flight SQL, LogQL, TraceQL)
+│       ├── Maintain Service (migrate, compaction, orphan GC, pricing crawler)
+│       ├── S3 Catalog (root.json CAS catalog; optional REST server)
+│       ├── Queue Library (Parquet WAL on S3)
+│       ├── Common Library (schemas, storage cache, sort-merge, memory guard)
+│       └── Storage (Queue/WAL, Iceberg, Catalog, Job state)
 └── Views
     ├── SystemContext
     ├── Containers
-    └── Component diagrams (per service)
+    ├── Component diagrams (Ingest, Query, Maintain, Queue, Catalog)
+    └── Process diagrams (Ingestion, Query, Maintenance)
 ```
 
+## Process Diagrams
+
+The three `*Flow` views are Structurizr [dynamic views](https://docs.structurizr.com/dsl/language#dynamic-view).
+Two things to know before editing them:
+
+- **Every step must correspond to a relationship that already exists in the model.** A dynamic
+  view may give that relationship a step-specific description, but it cannot invent an edge —
+  the DSL fails with `A relationship between X and Y does not exist in model`. When a flow needs
+  a step you have not modelled, add the relationship to the `model` block first.
+- **They render as UML sequence diagrams**, via the per-view
+  `properties { "plantuml.sequenceDiagram" "true" }`. Without it the exporter falls back to the
+  numbered box layout, which turns into unreadable long-arc spaghetti past about ten steps.
+  `autoLayout` is kept on each view as the fallback for that case.
+
+Structurizr's parallel-block syntax (`{ { … } { … } }`) is deliberately unused: both PlantUML
+exporters flatten it into duplicate step numbers with no visual grouping, so concurrent loops
+read as one pipeline. The maintenance view names the owning loop in each step description
+instead.
+
 ## Interactive Editing
 
 For the best editing experience, use Structurizr Lite: