diff --git a/.mailmap b/.mailmap index f8105ac1..bdc46477 100644 --- a/.mailmap +++ b/.mailmap @@ -7,7 +7,7 @@ # git -C apps/grobase -c mailmap.file="$PWD/.mailmap" shortlog -sne --all # # Caveat: two author/email pairs mix two members and are resolved by inference, not by -# proof (see the README, "Individual Contributions"). A new address a member starts +# proof (see wiki/project/contributions-method.md). A new address a member starts # committing with shows up as an extra line until it is added here. # dlesieur @@ -42,6 +42,7 @@ Seyed Mostafa Hashemian (shashemi) <1.318674e+08+S Seyed Mostafa Hashemian (shashemi) # prismatica 93e6b65 by "La Cebolla" carries Signed-off-by: Seyed Mostafa Hashemian. Seyed Mostafa Hashemian (shashemi) +Seyed Mostafa Hashemian (shashemi) # rstancu, a member of the original team roster (archive/wiki-2026-09/CONTRIBUTORS.md), GitHub @esettes Roxana Stancu (rstancu) @@ -51,13 +52,13 @@ opencode (AI agent) opencode (AI agent) opencode (AI agent) -# Space Bunny: the OpenCode agent team the owner ran on hydrawn (BUNNY.md, "You run a team -# of agents"), on local addresses. An AI agent, not a member, shown as one line like opencode. -Space Bunny (AI agent team, run by dlesieur) bunny -Space Bunny (AI agent team, run by dlesieur) bunny -Space Bunny (AI agent team, run by dlesieur) Space Bunny -Space Bunny (AI agent team, run by dlesieur) Space Bunny -Space Bunny (AI agent team, run by dlesieur) builder -Space Bunny (AI agent team, run by dlesieur) builder -Space Bunny (AI agent team, run by dlesieur) builder -Space Bunny (AI agent team, run by dlesieur) builder +# Space Bunny: an AI-agent identity on hydrawn (BUNNY.md describes a team of agents), on +# local addresses. Who ran it is not identified. Shown as one line, like opencode. +Space Bunny (AI agent team) bunny +Space Bunny (AI agent team) bunny +Space Bunny (AI agent team) Space Bunny +Space Bunny (AI agent team) Space Bunny +Space Bunny (AI agent team) builder +Space Bunny (AI agent team) builder +Space Bunny (AI agent team) builder +Space Bunny (AI agent team) builder diff --git a/LICENSE b/LICENSE new file mode 100644 index 00000000..e8c40b65 --- /dev/null +++ b/LICENSE @@ -0,0 +1,48 @@ +Osionos Project License + +Copyright (c) 2026 Osionos contributors. + +All rights reserved. + +This repository and all of its contents, including but not limited to source code, +configuration files, scripts, documentation, assets, examples, schemas, database +migrations, Docker files, and related materials, are the exclusive property of the +copyright holders and are protected by copyright law and other applicable +intellectual property laws. + +Permission is granted only to the contributors to this project and to persons +explicitly authorized by the copyright holders to access, inspect, run, and +contribute to the software for the limited purpose of evaluating, developing, +maintaining, and improving the project itself. + +No permission is granted for any other use. In particular, and without limiting +this restriction, you may not: + +- copy, reproduce, or redistribute this software or any part of it; +- create derivative works or modified versions for your own benefit; +- sell, license, sublicense, transfer, or otherwise commercialize the software; +- publish, distribute, or share the software outside the project without the prior + written permission of the copyright holders; +- use the software or any of its materials for private, personal, or commercial + advantage outside the scope of project participation and authorization. + +The project is intended to be developed collaboratively for the benefit of the +community. Community members may participate in the project by contributing code, +ideas, tests, bug fixes, documentation, and other improvements, but no person or +entity may copy, repurpose, or redistribute the software for their own benefit +without express written authorization from the copyright holders. + +Any contribution made to this repository is accepted only for inclusion in the +project under these terms. Unless otherwise agreed in writing, contributors keep +ownership of their original work, while agreeing that their contribution may be +used, modified, and improved as part of the project under the same restrictions. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, +INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A +PARTICULAR PURPOSE, AND NONINFRINGEMENT. IN NO EVENT SHALL THE COPYRIGHT HOLDERS +OR CONTRIBUTORS BE LIABLE FOR ANY CLAIM, DAMAGES, OR OTHER LIABILITY, WHETHER IN +AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION +WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +For permission to use this software outside the limited scope described above, +please contact the copyright holders in writing. diff --git a/README.md b/README.md index 89d64e0f..04677076 100644 --- a/README.md +++ b/README.md @@ -1,12 +1,21 @@ *This project has been created as part of the 42 curriculum by dlesieur, serjimen, danfern3, vjan-nie, shashemi.* -# Track Binocle — ft_transcendence - -**Track Binocle** is a collaborative workspace built on top of our own self-hostable -Backend-as-a-Service. Users sign up on a public site, land in **osionos** — a Notion-style block -editor with real-time collaboration, chat and video rooms — and can connect their Gmail and -Google Calendar. Every one of those apps is served by **grobase**, a generic backend that the team -wrote in Go, Rust and TypeScript, and which contains no application-specific code. +# Prismatica — ft_transcendence + +**Prismatica** is a collaborative workspace built on our own self-hostable +Backend-as-a-Service. Users sign up on the public **prismatica** site, land in +**osionos** — a Notion-style block editor with real-time collaboration, chat and +video rooms — and can connect their Gmail and Google Calendar. Every one of +those apps is served by **grobase**, a generic backend that the team wrote in +Go, Rust and TypeScript, with no per-application server code. + +> Naming: *Prismatica* is the project; *osionos* is the workspace app; +> *prismatica* (`apps/opposite-osiris`, repository +> [`Univers42/prismatica`](https://github.com/Univers42/prismatica)) is the +> marketing landing. Infrastructure identifiers keep the old codename +> `track-binocle` (the compose project, image names, `TRACK_BINOCLE_*` +> variables, `*.track-binocle.test` hosts, the local CA file): renaming them +> would break scripts and issued certificates. --- @@ -38,40 +47,50 @@ code** — and to prove it by building a real product on it. The project therefore has two halves: -- **grobase** (roughly 70% of the work) — a Backend-as-a-Service. An application is declared in a - JSON *contract* (its database, isolation strategy, roles, permissions and API keys); a generic - provisioner creates everything from that file. Routes carry no domain logic: a request names the - table it targets, and access rules are stored as data, not code. -- **The products** that run on it — a marketing and authentication site, the osionos editor, and - mail and calendar integrations. +- **grobase** — a Backend-as-a-Service (about 290,000 lines of Go, Rust and + TypeScript at its pin, roughly a third of the code the team wrote; lines + include tests). An application is declared in a JSON *contract* (its database, + isolation strategy, roles, permissions and API keys); a generic provisioner + creates everything from that file. Routes carry no domain logic: a request + names the table it targets, and access rules are stored as data, not code. +- **The products** that run on it — the prismatica landing and sign-in site, the + osionos workspace, the hydrawn whiteboard, and mail and calendar integrations. ### Overview | Product | Tech | What it is | |---|---|---| -| **opposite-osiris** | Astro | Public site: landing page, sign-up / sign-in, legal pages. The entry point. | +| **prismatica** (`apps/opposite-osiris`) | Astro | Public site: landing page, docs, sign-up / sign-in, legal pages. The entry point. | | **osionos** | React + Vite | The block editor: pages, databases-as-blocks, real-time collaboration, chat, video rooms. **The flagship.** | | **mail** | React + Vite | Gmail integration over Google OAuth | | **calendar** | React + Vite | Google Calendar integration over Google OAuth | +| **hydrawn** | SvelteKit + Fastify + Rust/WASM | Collaborative whiteboard, embedded in osionos | -These four products are supported by four platform services (not applications in their own -right): the **auth-gateway** (Go, sessions), the **osionos-bridge** (turns a site session into an -editor session), **Kong** (the single API gateway in front of grobase) and **LiveKit** (the WebRTC -media server for video rooms). +These products are supported by platform services (not applications in their own +right): the **auth-gateway** (Node, shipped by the prismatica repository: +sessions and sign-in), the **osionos-bridge** (Node: turns a site session into +an editor session), **Kong** (grobase's API gateway, behind its WAF) and +**LiveKit** (the WebRTC media server for video rooms). ### Key features -- **Contract-driven backend.** A whole application — database, isolation, roles, keys, frontend - configuration — is one JSON file. `POST /v1/tenants/me/apps` creates a new isolated application - with its own database and scoped API key. -- **Per-request owner isolation.** The Rust data plane scopes every query to its owner on every - request; permissions are rows (ABAC/RBAC), not `if` statements. -- **One API over many databases.** PostgreSQL, MySQL, MongoDB, Redis, SQLite, MSSQL, DynamoDB and - even remote HTTP APIs are queried through the same envelope and the same API key. +- **Contract-driven backend.** A whole application — database, isolation, roles, + keys, frontend configuration — is one JSON file. `POST /v1/tenants/me/apps` + creates a new isolated application with its own database and scoped API key + (mounted only when `APPS_SELFSERVE_ENABLED` is on). +- **Per-request owner isolation.** The Rust data plane scopes queries to their + owner on each request, not by connection-pool state; permissions are rows + (RBAC, plus flag-gated ABAC conditions), not `if` statements. +- **One API over many databases.** PostgreSQL, MySQL (MariaDB image), MongoDB, + Redis, SQLite, SQL Server and even remote HTTP APIs are queried through the + same envelope and the same API key. DynamoDB is an opt-in build, left out of + the default one + ([`wiki/platform/08-engines.md`](wiki/platform/08-engines.md)). - **Collaborative editor.** Block-based pages, a database block with table / board / calendar views, real-time co-editing, channels and direct messages, video rooms. -- **Single-command deployment.** `make all` builds and starts the whole stack in Docker, serves - every frontend over trusted local HTTPS, and health-checks it. +- **Single-command deployment.** `make all SKIP_SYNC=1` builds the pinned + commits in Docker, starts the backend and the frontends over local HTTPS with + a trusted CA, and health-checks them (`utils/mk/pipeline.mk:10`). --- @@ -81,14 +100,14 @@ media server for video rooms). | Requirement | Detail | Source | |---|---|---| -| **Linux host** | Tested on Debian 13; the CA-trust step uses `sudo` + `update-ca-certificates` | `utils/mk/certs.mk` | +| **Linux host** | Tested on Debian 13 (trixie), the system the born2root VM installs; the CA-trust step uses `sudo` + `update-ca-certificates` | `vendor/born2root/docker/Dockerfile:14`, `vendor/born2root/generate/create_custom_iso.sh:120`; `utils/mk/certs.mk` | | **Docker Engine ≥ 25** + **Compose v2** | Healthchecks use `start_interval`, which needs Docker 25. Tested with Docker 29.8. | `docker-compose.yml:10` | -| **RAM ≥ 8 GB** | Measured: the VM peaked at ~6.5 GB of 8 GB during `make all` | measured, 2026-09-30 | -| **Disk ≥ 30 GB free for Docker** | Measured on the dev VM with the stack running (`docker system df`, 2026-10-01): images 13.5 GB, build cache 9.5 GB, volumes 1.4 GB | measured | -| **GNU Make, git, curl, openssl** | Make drives everything; curl runs the health check; openssl generates the local secrets | `utils/mk/app.mk`, `scripts/gen-local-env.sh:68` | +| **RAM ≥ 8 GB** | Measured: the VM peaked at ~6.5 GB of 8 GB during `make all` | `wiki/runbooks/fresh-machine.md` (6266 MiB peak, v1.0.0-rc4, 2026-10-05) | +| **Free disk space for Docker** | Images, the build cache and data volumes all live in Docker's storage; no minimum is measured in the repository. `docker system df` shows the current use | — | +| **bash, GNU Make, git, curl, openssl, ssh** | Make drives everything; curl runs the health check; openssl generates the local secrets | `utils/mk/app.mk:12-15`, `scripts/gen-local-env.sh:253` | | **A GitHub SSH key** | `.gitmodules` uses `git@github.com:` URLs, so the recursive clone needs SSH access to GitHub | `.gitmodules` | -| **Google Chrome** | Current stable — the browser the project is evaluated on | | -| `certutil` (optional) | Imports the local CA into the browsers' NSS stores. Absent, `make certs-trust-user` unpacks `libnss3-tools` into `~/.cache/born2root/nss` without root; if that fails it warns and exits 0, and the CA is imported by hand (see [Access](#access-and-the-local-ca)) | `scripts/certs-trust-user.sh` | +| **Google Chrome** | Current stable — the browser the project is evaluated on | [`wiki/project/01-subject.md`](wiki/project/01-subject.md) | +| `certutil` (optional) | Imports the local CA into the browsers' NSS stores. Absent, `make certs-trust-user` unpacks `libnss3-tools` into `~/.cache/born2root/nss` without root (Debian only: it needs `apt-get` and `dpkg-deb`, bounded at 120 s); if that fails it warns and exits 0, and the CA is imported by hand (see [Access](#access-and-the-local-ca)) | `scripts/certs-trust-user.sh` | **Nothing else is installed on the host** — no Node, npm, Go or Cargo. Everything builds and runs in containers. @@ -98,47 +117,56 @@ trust store. This is intended (`certs-trust-local` in `utils/mk/certs.mk`). With sudo, `CERT_TRUST_MODE=user` trusts the browsers only and `skip` trusts nothing (see [Access](#access-and-the-local-ca)). -**Ports bound on the host** (from `docker-compose.yml`, all on `127.0.0.1` except inside a -QEMU/VirtualBox NAT VM, where `infrastructure/scripts/detect-bind-addr.sh` binds `0.0.0.0` so the -host can reach the guest): 443, 3001, 3002, 3003, 3007 (always loopback), 4000, 4100, 4200, 4322, -4323 and 7880 (always loopback: plain HTTP), 7881, 8444, 8787 and UDP 50000–50060. grobase adds Kong on `127.0.0.1:8000` -(literal loopback in its `orchestrators/compose/base/gateway.yml`). +**Ports bound on the host** (from `docker-compose.yml`, all on `127.0.0.1` +except inside a QEMU/VirtualBox NAT VM, where +`infrastructure/scripts/detect-bind-addr.sh` binds `0.0.0.0` so the host can +reach the guest): 443, 3001, 3002, 3003, 3007 (always loopback), 4000, 4100, +4200, 4322, 4323 and 7880 (always loopback: plain HTTP), 7881, 8444, 8446 +(always loopback: the IDE edge), 8787 and UDP 50000–50060. A busy port moves to +the next free one (`scripts/host-ports.sh`). grobase adds Kong on +`127.0.0.1:8000` (literal loopback in its +`orchestrators/compose/base/gateway.yml:56`; its WAF is the public entry). ### Defense bring-up (pinned) -This is the path rehearsed on a clean machine. It checks out the release tag and does **not** -follow submodule branch tips (`SKIP_SYNC=1`, see `sync-submodules-soft` in -`utils/mk/pipeline.mk`). +This is the path rehearsed on a clean machine, and the stack the +[Modules](#modules) are graded on. It builds `main` and the submodule commits +`main` pins, and does **not** follow submodule branch tips (`SKIP_SYNC=1`, see +`sync-submodules-soft` in `utils/mk/pipeline.mk`). ```bash git clone --recursive git@github.com:Univers42/groot.git cd groot -git checkout "$(git describe --tags --abbrev=0 origin/main)" +git checkout main git submodule update --init --recursive make all SKIP_SYNC=1 ``` -The `git checkout` line checks out the latest release tag on `main`; `git tag -l 'v*'` lists them. - Requirements, the VM + host-browser case, the verify script (`scripts/verify/fresh-bringup.sh`) and troubleshooting: [`wiki/runbooks/fresh-machine.md`](wiki/runbooks/fresh-machine.md). -grobase on another machine (the born2root VM, a LAN box, a VPS, a Kong URL) with the frontends -here, no sudo: put the kind of connection in `./.env.grobase-link` (`GROBASE_TARGET=ssh://b2b`; -`.env.grobase-link.example` lists the kinds), then `make link` and `make link-frontends-up`. The -latter also applies `models/*.sql` to that grobase (`make link-models-migrate`), sets the demo -account's password there (`make link-demo-login-ensure`; `make demo-login` prints it), hands the +grobase on another machine (the born2root VM, a LAN box, a VPS, a Kong URL) with +the frontends here, no sudo: put the kind of connection in `./.env.grobase-link` +(`GROBASE_TARGET=ssh://b2b`; `.env.grobase-link.example` lists the kinds), then +`make link` and `make link-frontends-up`. The latter also applies `models/*.sql` +to that grobase (`make link-models-migrate`), sets the demo account's password +there (`make link-demo-login-ensure`; `make demo-login` prints it), hands the demo rows to the app's API key (`make link-live-data-ensure`) and trusts the -local CA in this user's browser stores (`make certs-trust-user`, no sudo). Both end with -`make link-verify`: one request per grobase service from inside every frontend, the TLS edge, -then the browser stores — [`wiki/runbooks/grobase-link.md`](wiki/runbooks/grobase-link.md). -To serve the frontends to the internet through this host's Tailscale Funnel (443 first, then -8443 and 10000), `make public-env` and `make public-up`; `make public-down` closes it: -[`wiki/runbooks/public-funnel.md`](wiki/runbooks/public-funnel.md). -`make roundtrip` then proves persistence (account, sign-in, editor hand-off, a page, the rows -in postgres) and `make link-e2e` runs the Playwright smoke through the link. - -Measured time from a cold machine: **9–17 minutes**. +local CA in this user's browser stores (`make certs-trust-user`, no sudo). Both +end with `make link-verify`: one request per grobase service from inside every +frontend, the TLS edge, then the browser stores — +[`wiki/runbooks/grobase-link.md`](wiki/runbooks/grobase-link.md). To serve the +frontends to the internet through this host's Tailscale Funnel (443; add 8443 +and 10000 with `PUBLIC_PORTS=all`), `make public-env` and `make public-up`; +`make public-down` closes it: +[`wiki/runbooks/public-funnel.md`](wiki/runbooks/public-funnel.md). `make +roundtrip` then proves persistence (account, sign-in, editor hand-off, a page, +the rows in postgres) and `make link-e2e` runs the Playwright smoke through the +link. + +Measured time from a cold machine: **9–17 minutes** (8m55s on a fresh VM, 16m44s +on a laptop, 2026-09-30, [`wiki/STATUS.md`](wiki/STATUS.md); later runs in +[`wiki/runbooks/fresh-machine.md`](wiki/runbooks/fresh-machine.md)). No `.env` needs to be written by hand. With no vault key present, grobase generates its own secrets into `apps/grobase/.env`, and `env-local-ensure` derives the root `./.env.local` from it, @@ -147,8 +175,14 @@ including random `OSIONOS_*` secrets and the demo account's `DEMO_LOGIN_PASSWORD `make all`, which says so by name. `make all` runs, in order (`utils/mk/pipeline.mk`): submodule sync → secrets → -local TLS certificates → trust the local CA → grobase backend → derive `.env.local` → restore -data if the engines are empty → set the demo account's password → SQL migrations → frontends → health check → URL list. +local TLS certificates → trust the local CA → grobase backend → derive +`.env.local` → restore data if the engines are empty → set the demo account's +password → SQL migrations → live-data key and owner fix → mount re-encryption → +frontends → health check → URL list. On a machine that holds vault42 +credentials, `make all` deploys against the data-center VM instead and starts no +local grobase (ADR 0047, +[`wiki/runbooks/datacenter.md`](wiki/runbooks/datacenter.md)); `make all +GROBASE_DATACENTER=` forces the local path. ### Development bring-up @@ -157,9 +191,11 @@ git checkout develop make all ``` -Without `SKIP_SYNC=1`, `make all` first fast-forwards every submodule to the tip of its tracked -branch (`sync-submodules-soft`), so you build the newest code rather than the pinned commits. -It skips dirty submodules and never blocks the pipeline. +Without `SKIP_SYNC=1`, `make all` first fast-forwards every submodule to the tip +of its tracked branch (`sync-submodules-soft`), so you build the newest code +rather than the pinned commits. It skips dirty submodules, and leaves at its +recorded commit any submodule whose branch tip does not contain that commit; it +never blocks the pipeline. The exception is any submodule that `.gitmodules` marks `update = none`, today `apps/graph_render`. Its gitlink is the build input, so neither mode moves it: @@ -201,7 +237,7 @@ are in [ADR 0016](docs/adr/0016-self-documenting-pruned-root-make.md). | Service | URL | |---|---| -| opposite-osiris (start here) | `https://localhost:4322` | +| prismatica site (start here) | `https://localhost:4322` | | osionos editor | `https://localhost:3001` | | Whiteboard (hydrawn) | `https://localhost:3007` | | mail (opt-in) | `https://localhost:3002` | @@ -226,8 +262,9 @@ store (one `sudo`) and every browser store this user owns, through `scripts/cert and the flatpak locations. It matches old copies by the CA's subject, not by nickname, replaces them, and adds the current file as "Track Binocle Local Development CA". -- `make certs-trust-check` prints one line per store (`ok|FAIL current|stale|missing`) - and exits 1 if any store is not current. Without `certutil` it fails. +- `make certs-trust-check` prints one line per store (`ok|FAIL + current|stale|missing|untrusted`) and exits 1 if any store is not current. + Without `certutil` it fails. - `make certs-trust-user` fixes the stores without sudo; it mints the CA first only if it is missing. - `CERT_TRUST_MODE=system|user|skip` (or `TRACK_BINOCLE_CERT_TRUST`) picks what `certs-trust-local` does: system store plus browsers (default), browsers only, or nothing. @@ -238,21 +275,13 @@ If the CA is not trusted, the browser shows a warning on each port, and the **Wh osionos stays blank**, because the embedded `:3007` iframe is refused silently. Two causes of a warning while `curl --cacert` succeeds: -- **The CA was regenerated.** `make certs` on a fresh tree mints a new CA, and a store still holding - an older copy does not trust it. On 2026-10-08 the CA was regenerated at 17:06 and 7 stores were - stale and 1 missing: - - ```text - $ make certs-trust-check - FAIL chrome /home/dlesieur/.pki/nssdb stale - FAIL chrome /home/dlesieur/snap/chromium/current/.pki/nssdb stale - FAIL firefox /home/dlesieur/snap/firefox/common/.mozilla/firefox/pqdn9c63.default-release-1 stale - FAIL firefox /home/dlesieur/.var/app/org.mozilla.firefox/config/mozilla/firefox/2vgnvz7b.default missing - [certs] 8 browser store(s) do not hold the current CA; run: make certs-trust-user, then relaunch the browser - ``` +- **The CA was regenerated.** `make certs` on a fresh tree mints a new CA, and a + store still holding an older copy does not trust it. `make certs-trust-check` + then reports those stores `stale`, and `make certs-trust-user` replaces the + old copy. - **A snap refresh.** Snap Chromium's `HOME` is the revision directory behind - `~/snap/chromium/current`, so a new revision (3507 to 3551 overnight) starts without the store. + `~/snap/chromium/current`, so a new revision starts without the store. `make certs-trust-check` reports it missing; `make certs-trust-user` recreates it. Flatpak Firefox profiles can exist without the flatpak installed; they are imported too, harmlessly. @@ -271,10 +300,10 @@ certificates → Authorities → Import, or run `make certs-trust`. 3. Open the same board URL in a second window and draw: the strokes appear live in the other window (the e2e smoke asserts within 5 s — `tests/e2e/specs/dw8-realtime.spec.ts`). -Honest limitations of this demo: board editing is **loopback-only** and has **no per-user -authorization yet** — hydrawn is bound to `127.0.0.1:3007` for exactly that reason -(`docker-compose.yml:27-32`). We make **no end-to-end-encryption claim**: traffic is TLS to the -local proxy, nothing more. +Honest limitations of this demo: board editing is **loopback-only** and has **no +per-user authorization yet** — hydrawn is bound to `127.0.0.1:3007` for exactly +that reason (`docker-compose.yml:35-40`). We make **no end-to-end-encryption +claim**: traffic is TLS to the local proxy, nothing more. ### Environment configuration @@ -285,15 +314,23 @@ local proxy, nothing more. - `LIVEKIT_API_KEY` / `LIVEKIT_API_SECRET` — default to a local dev value; video rooms still work. - `GOOGLE_CLIENT_ID` / `GOOGLE_CLIENT_SECRET` in `apps/mail/.env.local` and `apps/calendar/.env.local` — without them, the mail and calendar OAuth flows are disabled. - - `SONAR_TOK`, `FLY_API_TOKEN`, `DOCKER_PAT` — CI and operations only. + - `SONAR_TOK`, `FLY_API_TOKEN`, `DOCKER_PAT` — operator tooling only (a local + SonarCloud scan, the Fly deploy, a Docker Hub login); CI uses its own + repository secrets. ### Team setup (with the vault) -Inside the team, secrets are shared through **vault42**, our zero-knowledge secret store, via the -**42ctl** CLI. A machine that holds a vault identity in `~/.config/42ctl/` pulls the whole `.env` -tree automatically during `make all` (or with `make vault42-pull-all APPLY=1`). A new identity is -created with `42ctl keys init` and enrolled with an invitation from the vault's operator. -Details: [`DATA-MIGRATION.md`](DATA-MIGRATION.md). +Inside the team, secrets are shared through **vault42**, our zero-knowledge +secret store, via the **42ctl** CLI. A machine that holds a vault identity in +`~/.config/42ctl/` pulls the `.env` tree during `make all` when +`apps/grobase/.env` is absent (`secrets-ensure`), or on demand with `make +vault42-pull-all APPLY=1`. With the vault passphrase also present, `make all` +deploys against the data-center VM instead (see [Defense +bring-up](#defense-bring-up-pinned)). A new identity: `scripts/vault42-ctl.sh +keys init`, then `scripts/vault42-ctl.sh keys enroll --org univers42`, then an +org admin grants the project +([`wiki/runbooks/another-device.md`](wiki/runbooks/another-device.md)). Details: +[`DATA-MIGRATION.md`](DATA-MIGRATION.md). The vault42 CLI is wired in beside that path: the `Secrets (vault42)` section of `make help`, and [`vendor/scripts/vault42/README.md`](vendor/scripts/vault42/README.md). Status and retirement @@ -303,82 +340,116 @@ checklist: [ADR 0017](docs/adr/0017-vault42-cli-beside-the-old-path.md). | Command | What it does | |---|---| -| `make all` | Full lifecycle: certs → backend → frontends → health check → URLs | +| `make all` | Full lifecycle: secrets → certs → backend → env → data → migrations → frontends → health check → URLs | | `make healthcheck` | Probe backend, site, editor, bridge, auth gateway and whiteboard | | `make showcase` | Print the URL list for what is actually running | -| `make pulls` | Update all submodules | +| `make pulls` | Fetch and pull the root repository and every submodule | | `make -C apps/grobase editions` | List the available backend shapes | -| `make all GROBASE_EDITION=full` | Bring up everything, including extra engines and monitoring | +| `make all GROBASE_EDITION=full` | Bring up every grobase plane except the playground demo UI, including extra engines and monitoring | | `docker compose ps` | Service status | | `docker compose logs -f ` | Follow one service's logs | -| `docker compose down` | Stop the frontends (data is kept) | +| `COMPOSE_PROFILES=dev docker compose down` | Stop and remove the root project's containers; named volumes stay (no `-v`). `make` sets `COMPOSE_PROFILES=dev` (`utils/mk/common.mk:45`), a plain shell does not | **Backend editions.** `make all` starts grobase in the **`devlean`** edition (`GROBASE_EDITION ?= devlean` in `utils/mk/grobase.mk`; listed in grobase's -`orchestrators/makes/00-config.mk:68`): the core engines (PostgreSQL, MySQL, MongoDB, Redis, -MinIO), the full application / control / data planes and realtime — without the heavy extra -engines and without the monitoring stack. `GROBASE_EDITION=migrate` adds every snapshot engine; -`GROBASE_EDITION=full` turns everything on. Some module demonstrations need a larger edition or a -feature flag — see [Modules](#modules). +`orchestrators/makes/00-config.mk:148-154`): the core engines (PostgreSQL, +MySQL, MongoDB, Redis, MinIO), the full application / control / data planes and +realtime — without the heavy extra engines and without the monitoring stack. +`GROBASE_EDITION=migrate` adds every snapshot engine; `GROBASE_EDITION=full` +turns on every plane except the playground demo. Some module demonstrations need +a larger edition or a feature flag — see [Modules](#modules). --- ## Team Information -One owner per role, as the team confirmed on 2026-11-04 (the earlier working notes in -[`wiki/project/03-team.md`](wiki/project/03-team.md) named two Product Owners and two Project Managers). -The "Areas in the history" column is what each member's commits touch -([Individual Contributions](#individual-contributions) has the numbers). +All five members are developers. Roles evolved during the project; the main +role of each is below (team answer, 2026-11-05). The "Areas in the history" +column is what each member's commits touch ([Individual +Contributions](#individual-contributions) has the numbers). -| Member | Login | GitHub | Role(s) | Responsibilities | Areas in the history | +| Member | Login | GitHub | Main role | Responsibilities | Areas in the history | |---|---|---|---|---|---| -| Dylan Lesieur | `dlesieur` | [@LESdylan](https://github.com/LESdylan) | Product Owner · Developer | Product vision, backlog and priorities; validates completed work | grobase, osionos, mail, calendar, root infrastructure and CI | -| Daniel Fernández | `danfern3` | [@danielfdez17](https://github.com/danielfdez17) | Tech Lead · Developer | Architecture, stack decisions, code quality, reviews | monkey-bot (security test bot), grobase Docker and Kubernetes deployment, osionos editor fixes | -| Sergio Jiménez de la Cruz | `serjimen` | [@DjSurgeon](https://github.com/DjSurgeon) | Developer | Frontend, secrets management, deployment | the public site (opposite-osiris), osionos accessibility and editor features | -| Vadim Jan Nieto | `vjan-nie` | [@vjan-nie](https://github.com/vjan-nie) | Project Manager · Developer | Planning, tracking, communication, unblocking | QA tooling, end-to-end tests, documentation, Docker | -| Seyed Mostafa Hashemian | `shashemi` | [@SMOSTAFAH1](https://github.com/SMOSTAFAH1) | Developer | TO CONFIRM by the team | security fixes and SAST, the site's legal pages, TLS edge | - -> TO CONFIRM by the team: `shashemi`'s responsibilities. - -`rstancu` (Roxana Stancu, [@esettes](https://github.com/esettes)) is on the original roster -([`archive/wiki-2026-09/CONTRIBUTORS.md`](archive/wiki-2026-09/CONTRIBUTORS.md)) and committed -between 2026-03-18 and 2026-05-02; see [Who committed where](#who-committed-where). +| Dylan Lesieur | `dlesieur` | [@LESdylan](https://github.com/LESdylan) | Product Owner | Product vision and priorities | grobase, osionos, mail, calendar, root infrastructure and CI | +| Vadim Jan Nieto | `vjan-nie` | [@vjan-nie](https://github.com/vjan-nie) | Project Manager | Planning, tracking and coordination | QA tooling, end-to-end tests, documentation, Docker | +| Sergio Jiménez de la Cruz | `serjimen` | [@DjSurgeon](https://github.com/DjSurgeon) | Frontend tech lead | Frontend architecture and quality | the public site (opposite-osiris), osionos accessibility and editor features | +| Daniel Fernández | `danfern3` | [@danielfdez17](https://github.com/danielfdez17) | Backend tech lead | Backend architecture and deployment | monkey-bot (security test bot), grobase Docker and Kubernetes deployment, osionos editor fixes | +| Seyed Mostafa Hashemian | `shashemi` | [@SMOSTAFAH1](https://github.com/SMOSTAFAH1) | QA / quality tech lead | Testing and code quality; joined in the last part of the project | security fixes and SAST, the site's legal pages, TLS edge | + +**Former members.** The team changed along the way: `rstancu` (Roxana Stancu, +[@esettes](https://github.com/esettes)) and Alex Caceres were temporary members +who left. Special thanks to `rstancu`, who contributed significantly to the +osionos frontend and the QA workflow during the first, toughest months. The +defense covers the current five. --- ## Project Management -What the repository records: - -- **Task tracking:** GitHub Issues on [`Univers42/groot`](https://github.com/Univers42/groot/issues) - and on each submodule's repository; milestones (M1 hardening, M2 federation, M3 coherence, - M4 observability, M5 security, M11 external-app integration) in - [`archive/wiki-2026-09/todo/`](archive/wiki-2026-09/todo/). -- **Tools:** GitHub — pull requests, Dependabot for dependency updates - ([`.github/dependabot.yml`](.github/dependabot.yml)), GitHub Actions for CI - ([`.github/workflows/`](.github/workflows/)). -- **Branch model:** work lands on short-lived `feat/`, `fix/`, `docs/`, `test/` and `chore/` - branches, merged into `develop` by pull request; `develop` is merged into `main` by pull request - (`git log --merges --format=%s | grep 'pull request'`). -- **Code workflow:** submodule-based monorepo — changes are committed inside a submodule first, - then the root records the new commit. Numbered **verification gates** - (`apps/grobase/scripts/verify/run-gate-battery.sh`) are the team's definition of "done": - grobase's CI runs named gates on pull requests and the full `--enterprise` battery, with a - nightly schedule (grobase `.github/workflows/ci.yml`). - -What the team confirmed (2026-11-04), which the repository cannot show: - -- **Communication channel:** Discord. -- **Meeting rhythm:** a weekly meeting. - -What the repository does not record: - -- **Task distribution and code review:** not recorded beyond the merged pull requests per login - ([Who committed where](#who-committed-where)). - -> TO CONFIRM by the team: who attended the weekly meeting and where its decisions were written. - -> TO CONFIRM by the team: who assigned work, and who reviewed whose pull requests. +Part of this project was an experiment in managing a team. We ran it as if it +were a demanding, high-output professional team, with a Scrum mindset in which +every member is treated as an expert and anyone may change anyone's code. The +aim was that **nobody is indispensable**. The evidence below separates what the +repositories record from what is the team's own account, and the last part +measures how far we got towards that aim. + +**How the tracking evolved.** Work started in February 2026: the first commits +date from that month (`git log --reverse` across the submodules). Between then +and the autumn the team tried GitHub Projects, Jira tickets and Notion pages +(team account: none of the three left a trace in commit messages, and the +Projects board was not readable with our token). The only written trace is +osionos +[`docs/DoD.md:39`](https://github.com/Univers42/osionos/blob/develop/docs/DoD.md), +a Definition-of-Done template that asks for evidence "in your Pull Request +description or your Trello/Jira" ticket (2026-04-11). + +**How it works now: issues, labels and pull requests on GitHub.** + +| Practice | What the repositories record (2026-11-04) | +|---|---| +| Issues | 129 on osionos, 38 on groot, 20 on prismatica, 16 on grobase (`gh issue list --state all`). Branches and commits name them: `fix/issue-278-database-probe-diagnostics` (osionos #284), `fixes #N` in 40 commit messages across groot, grobase, osionos and prismatica | +| Labels | Triage labels `priority: high`, `subject-blocker` and `account-platform` on groot and osionos, plus GitHub's defaults (`gh label list`) | +| Pull requests | 144 on groot, 182 on osionos, 78 on grobase, 50 on prismatica; merged into `develop`, and `develop` promoted to `main` by a merge commit | +| Peer review | Since 2026-09-21 (grobase, hydrawn) and up to 2026-11-04 (groot `develop`), a GitHub ruleset on each main repository requires one approving review and forbids force-pushes and branch deletion (`gh api repos//rulesets`). CodeRabbit reviews every PR since 2026-10-23 | +| Stacked branches | Feature chains merged one on top of another: groot #84–#87 (`feat/ide-*`), grobase #69–#70 (`feat/multidb-g1b…g1d`), the osionos `release/acct-w1` … `w9` series | +| Commit rhythm | At least one commit on 174 of the 232 days between 2026-03-18 and 2026-11-04, across all repositories | + +**Strict CI from the start.** CI arrived within days of each repository's first +commit: grobase `ci.yml` on 2026-03-29, osionos on 2026-04-12, groot on +2026-05-13. SonarCloud is configured in groot since 2026-05-04 and comments on +its pull requests since #9 (2026-06-01). Security scans (Semgrep, Trivy, +gitleaks, TruffleHog, ZAP, cargo-audit, govulncheck) run on every pull request, +and groot rejects one-word commit subjects +(`.github/workflows/commit-subjects.yml`). + +**Rituals.** Two-week sprints since the start, daily stand-ups and a weekly +meeting, mostly on Discord (team account, confirmed 2026-11-04; Discord leaves +no trace in the repositories). The method was Scrumban (Scrum rituals plus a +Kanban board), as described in +[`docs/dossier-project-dylan/dossier-projet.md:390-416`](https://github.com/Univers42/osionos/blob/develop/docs/dossier-project-dylan/dossier-projet.md) +in osionos. + +**Planning and decisions.** Work is planned and decided in meetings, mostly on +Discord, then recorded in issues, pull requests and the wiki. When a problem +blocked the team, it got a dedicated meeting. Test-driven development and +DevOps practices (CI on every pull request, Docker-first builds, one-command +bring-up) were the means to a frictionless workflow. + +**How far we got.** We publish these numbers so the claims above can be checked: + +- **Sprints.** The sprints were planned outside GitHub: no milestone and no + sprint label exists in any repository. Merges come in bursts: 250 of the 442 + merged pull requests on the six main repositories landed in two weeks (ISO + weeks 43–44). +- **Review.** The rulesets let a maintainer bypass them, and in a sample of the + 40 most recent merged pull requests per repository, 0–22 % carry an approval + from someone other than the author. +- **Nobody indispensable.** Shared ownership is real in breadth: every active + member committed in at least three areas, and osionos has six authors with + five or more commits. In depth we are not there yet: one member wrote 86 % of + the 8,571 human commits, and in 17 of 18 areas one author holds more than half + of the commits. Commit counts are not effort: one large commit and a typo fix + weigh the same. --- @@ -388,13 +459,14 @@ What the repository does not record: | Technology | Used in | Why | |---|---|---| -| **Astro 6** + **Sass** | opposite-osiris | Mostly static marketing content: Astro ships near-zero JavaScript by default and pre-renders pages, which keeps the public site fast. | +| **Astro 7** + **Sass** | prismatica (`apps/opposite-osiris`) | Mostly static marketing content: Astro ships near-zero JavaScript by default and pre-renders pages, which keeps the public site fast. | | **React 19** + **Vite 6** + **TypeScript** | osionos, mail, calendar | A block editor is a heavily interactive UI; React's component model fits, and Vite gives fast builds. | -| **Tailwind CSS 4**, **Radix UI**, **lucide-react** | osionos | Utility-first styling framework plus accessible, unstyled primitives for menus and popovers. | +| **Tailwind CSS 4**, **lucide-react** | osionos | Utility-first styling framework and an icon set. | | **Zustand** | osionos | Small, explicit client state store. | -| **Zod** | osionos | Schema validation of user input on the client side. | +| **Zod 4** | osionos | Schema validation: the diagram spec on the client, and the inputs of the AI service and the MCP server. | | **livekit-client** | osionos | WebRTC video rooms. | -| **Playwright** | osionos | End-to-end browser tests. | +| **Playwright** | osionos, groot `tests/e2e/` | End-to-end browser tests. | +| **SvelteKit**, **Fastify**, **Rust → WebAssembly** | hydrawn | The whiteboard: a WASM drawing engine with its own API and realtime services. | | **Tailwind CSS 4** | mail, calendar | `src/styles.css` imports Tailwind's theme and utilities through `@tailwindcss/vite`, without preflight; the app's own rules sit on top. | ### Backend — grobase @@ -406,43 +478,52 @@ independently — each written in the language that suits it. |---|---|---|---| | **Control plane** | **Go** | Provisioning, tenancy, identity (`POST /v1/keys/verify` turns an API key into an identity) | Long-running, concurrent I/O; a provisioner needs to be correct and readable, not fast. | | **Data plane** | **Rust** | Executes every query, applies per-owner scoping and ABAC locally | The hot path: predictable latency with no GC pauses, and memory safety where a bug would be a cross-tenant data leak. | -| **Realtime plane** | **Rust** | Turns data changes into events; WebSocket subscriptions | Same reasons; pluggable event bus. | -| **Application plane** | **TypeScript / NestJS** | Query and storage routers, schema, session, permissions, email, GDPR services | The original backend, progressively replaced by the Go and Rust planes. | -| **Auth** | **GoTrue** + Go **auth-gateway** | Sign-up, sign-in, sessions, email OTP | Battle-tested auth server rather than a hand-rolled one. | -| **Gateway** | **Kong** | The only public entry to the backend; key auth and rate limiting | Keeps every internal service off the public network. | -| **Media** | **LiveKit** | WebRTC SFU for video rooms | | +| **Realtime plane** | **Rust** | Turns data changes into events; WebSocket subscriptions (an opt-in profile, included in the `devlean` edition `make all` starts) | Same reasons; pluggable event bus. | +| **Application plane** | **TypeScript / NestJS** | Query and storage routers, schema service, permission engine. The older session, email and GDPR services remain only under the `legacy-node` profile; the Go orchestrator does their work | The original backend, progressively replaced by the Go and Rust planes. | +| **Auth** | **GoTrue** (grobase) + a Node **auth-gateway** (shipped by prismatica) | Sign-up, sign-in, sessions, email OTP, the hand-off to osionos | Battle-tested auth server rather than a hand-rolled one. | +| **Gateway** | **WAF** (nginx + ModSecurity/OWASP CRS) in front of **Kong** | The WAF is the public entry; Kong behind it does key auth and rate limiting | Keeps every internal service off the public network. | ### Database -- **PostgreSQL** is the primary database: the control-plane registries (tenants, roles, policies, - API keys) and all application data (see [Database Schema](#database-schema)). - **Why:** row-level security (RLS), JSONB for block content and flexible properties, strong - transactional guarantees, and a mature ecosystem. RLS is what makes per-owner isolation - enforceable inside the database itself, not only in application code. -- **MySQL, MongoDB, Redis, MinIO** (object storage) run in the default edition; **SQLite, MSSQL, - DynamoDB** and remote **HTTP** APIs are supported through the same data-plane interface. This is a - deliberate feature of the platform (one API over many engines), not a need of osionos itself. +- **PostgreSQL 16** is the primary database: the control-plane registries + (tenants, roles, policies, API keys) and all application data (see [Database + Schema](#database-schema)). **Why:** row-level security (RLS), JSONB for block + content and flexible properties, strong transactional guarantees, and a mature + ecosystem. RLS is what makes per-owner isolation enforceable inside the + database itself, not only in application code. +- **MySQL, MongoDB, Redis, MinIO** (object storage) run in the `devlean` edition + that `make all` starts; **SQLite**, **SQL Server** (needs the `engines` + plane), **DynamoDB** (an opt-in build) and remote **HTTP** APIs are supported + through the same data-plane interface. This is a deliberate feature of the + platform (one API over many engines), not a need of osionos itself. ### Other significant technologies - **Docker / Docker Compose** — every component runs in a container; one `make all`. - **A local certificate authority + TLS proxy** — all frontends served over trusted HTTPS locally. +- **LiveKit** — the WebRTC SFU for video rooms, a groot compose service started + by `make all`; the osionos bridge mints its tokens. - **vault42 / 42ctl** — our own zero-knowledge secret store and CLI (separate repositories). -- **CI security scanning** — on every pull request, [`.github/workflows/mini-baas-security.yml`](.github/workflows/mini-baas-security.yml) - runs Semgrep (SAST), `npm`/`pnpm audit`, Snyk (when a token is set) and Trivy; Dependabot - opens dependency updates. `sonar-project.properties` and `renovate.json` are present, but no - workflow in this repository runs them. +- **CI security scanning** — on every pull request, + [`.github/workflows/mini-baas-security.yml`](.github/workflows/mini-baas-security.yml) + runs Semgrep (SAST), `npm`/`pnpm audit`, Snyk (when a token is set) and Trivy; + Semgrep and Snyk report without blocking. Dependabot opens dependency updates. + SonarCloud analyses every pull request through its GitHub app + (`.sonarcloud.properties`), not through a workflow; `renovate.json` is present + but nothing runs it. ### Justification of the major choices - **Why build a BaaS instead of a single backend?** Because the same backend pieces are rebuilt for every project. We moved everything that varies between projects — routes, schemas, access rules, configuration — from code into data, so a new application is a contract, not a server. -- **Why three languages?** Each plane has a different job. Go for boring, correct orchestration; - Rust where performance and memory safety protect tenant isolation; TypeScript where it already - existed. The data plane was migrated from TypeScript to Rust behind a per-request switch with a - *shadow mode* (Rust computes and discards results for comparison), so the migration needed no - downtime and could be rolled back without a deployment. +- **Why three languages?** Each grobase plane has a different job (Node also + runs the site's auth-gateway and the osionos bridge). Go for boring, correct + orchestration; Rust where performance and memory safety protect tenant + isolation; TypeScript where it already existed. The data plane was migrated + from TypeScript to Rust behind a per-request switch + (`RUST_DATA_PLANE_FORWARD`, on in the default compose), so the migration + needed no downtime and could be rolled back without a deployment. - **Why per-request isolation?** Scoping on every request (instead of per connection) means a pooled connection can never carry one user's permissions into another user's query. @@ -454,10 +535,13 @@ The schema lives in two places: 1. **Application schema** — idempotent SQL migrations in [`models/`](models/) (`*-migration.sql`), applied to grobase's PostgreSQL by `make all` through - [`scripts/apply-models-migrations.sh`](scripts/apply-models-migrations.sh). That runner - deliberately skips the files written for the auth gateway's own database (`user.sql`, - `gdpr-migration.sql`, `auth-security-migration.sql`, `rls-hardening-migration.sql`, `seeds.sql`); - its header explains why. + [`scripts/apply-models-migrations.sh`](scripts/apply-models-migrations.sh). + That runner deliberately skips seven files that target a database with an + integer `users.id` (used by the Electron native edition) or that later files + supersede: `user.sql`, `gdpr-migration.sql`, `auth-security-migration.sql`, + `rls-hardening-migration.sql`, `seeds.sql`, + `osionos-folder-surface-migration.sql` and + `osionos-wiki-surface-migration.sql`; its header explains why. 2. **Platform schema** — grobase's control-plane registries (tenants, mounts, roles, policies, API keys), created by grobase's own migrations, plus per-application databases created at runtime from each contract (e.g. `infra/config/contracts/website.schema.sql` in grobase). @@ -505,7 +589,7 @@ erDiagram osionos_channels { uuid id PK uuid workspace_id FK - text kind "text | dm | voice | video" + text kind "text | dm | voice | video | group" text name boolean is_private jsonb abac @@ -527,20 +611,22 @@ Other table groups, by migration file: | Area | Tables | Migration | |---|---|---| | Users & sessions (auth gateway DB) | `users`, `sessions`, `user_activities`, `user_tokens` | `user.sql` | -| Auth security & GDPR (auth gateway DB) | `auth_audit_events`, `gdpr_requests`, `user_consents`, `newsletter_optins` | `auth-security-migration.sql`, `gdpr-migration.sql` | +| Auth security & GDPR | `auth_audit_events`, `gdpr_requests`, `user_consents`, `newsletter_optins` | `auth-security-migration.sql`, `gdpr-migration.sql` (integer-id flavour); `auth-audit-events-migration.sql`, `newsletter-optin-migration.sql` (applied to grobase) | | Site ↔ editor bridge | `osionos_bridge_identities`, `osionos_bridge_audit_events` | `osionos-bridge-migration.sql` | -| Social & communities | `osionos_communities`, `osionos_community_members`, `osionos_community_channels`, `osionos_connections`, `osionos_user_blocks`, `osionos_user_reports`, `osionos_feed_*` | `osionos-communities-`, `-social-`, `-feed-engagement-migration.sql` | +| Social & communities | `osionos_communities`, `osionos_community_members`, `osionos_community_channels`, `osionos_connections`, `osionos_user_blocks`, `osionos_user_reports`, `osionos_feed_*` | `osionos-communities-`, `-social-`, `-chat-`, `-feed-engagement-migration.sql` | | Databases-as-blocks | `osionos_object_databases`, `osionos_workspace_databases`, `osionos_app_connections` | `osionos-object-databases-`, `-workspace-databases-`, `-app-connections-migration.sql` | | Publishing & sharing | `osionos_published_pages`, `osionos_share_rules`, `osionos_page_links` | `osionos-publish-`, `-share-rules-`, `-page-links-migration.sql` | -| Tasks & notifications | `osionos_tasks`, `osionos_notifications`, `osionos_push_subscriptions` | `osionos-tasks-`, `-push-migration.sql` | +| Tasks & notifications | `osionos_tasks`, `osionos_notifications`, `osionos_push_subscriptions` | `osionos-tasks-`, `-engagement-`, `-push-migration.sql` | | Mail & calendar | `mail_accounts`, `mail_messages`, `calendar_accounts`, `calendar_sources`, `calendar_event_cache` | `mail-migration.sql`, `calendar-migration.sql` | ### Platform (grobase) registries -Access control is stored as data: `roles`, `user_roles` and `resource_policies` tables, evaluated -by the SQL function `public.has_permission(...)` and mirrored by the Rust data plane's own ABAC -evaluator. Every table generated by grobase receives an `owner_id` column automatically, which is -what makes one isolation rule apply to any table. +Access control is stored as data: `roles`, `user_roles` and `resource_policies` +tables, evaluated by the SQL function `public.has_permission(...)` and mirrored +by the Rust data plane's own ABAC evaluator. Every table created through +grobase's schema API gets an `owner_id` column appended when the caller does not +declare one (contract schemas declare it themselves), which is what makes one +isolation rule apply to any table. **Two `users` tables.** `models/user.sql` defines `users.id` as `SERIAL` (integer): it belongs to the auth gateway's database. The osionos tables reference users by `UUID`, in grobase's @@ -551,86 +637,117 @@ foreign-key clash (`scripts/apply-models-migrations.sh:27-38`). ## Features List -"Who" lists the authors of the commits that touch the feature's code, largest share first, with -the count (`scripts/contributors.sh --paths ...`; the paths are in the last column). +"Who" counts the non-merge commits that touch the feature's paths (last column), +reachable from the commit groot `main` pins for that repository, read through +the root [`.mailmap`](.mailmap), with bots and AI-agent identities left out; +largest share first. How the identities are joined: +[`wiki/project/contributions-method.md`](wiki/project/contributions-method.md). A commit count is not effort, and pairing or review does not show in it. | Feature | Description | Who (commits) | Paths counted | |---|---|---|---| -| Sign-up / sign-in | Email and password (hashed and salted) on the public site; email OTP enabled in production with real SMTP | dlesieur 11, serjimen 1 | opposite-osiris `src/pages/auth`, `src/pages/api/auth` | -| Site → editor handoff | After login, a one-time bridge session hands the user to osionos (`/api/auth/bridge/consume`) — no token in a query string | dlesieur 4 | osionos `scripts/bridge-auth*` | +| Sign-up / sign-in | Email and password on the public site, stored hashed by GoTrue; an email-OTP second step is off by default (`AUTH_LOGIN_EMAIL_OTP`) | dlesieur 9, serjimen 1 | opposite-osiris `src/pages/auth`, `src/pages/api/auth` | +| Site → editor handoff | After login, a one-time bridge session hands the user to osionos (`/api/auth/bridge/consume`) — the one-time token travels in the URL fragment and is redeemed once by `POST`, never in a query string | dlesieur 4 | osionos `scripts/bridge-auth*` | | Legal pages | Privacy Policy, Terms of Service, Cookie Policy, Data Rights, linked from the site | shashemi 5, vjan-nie 3, serjimen 2, dlesieur 2 | opposite-osiris `src/pages/legal` | -| Block editor | Pages built from blocks, nested pages, templates, covers, favourites, search | dlesieur 162, vjan-nie 42, danfern3 29, serjimen 14, rstancu 9, shashemi 2 | osionos `src/features/block-editor` | -| Database block | A block that is a database, with table / board / calendar views over any mounted data source | dlesieur 126, vjan-nie 2, serjimen 2, rstancu 2 | osionos `src/widgets/database*`, `src/features/*database*`, `src/shared/database*` | -| Real-time collaboration | Several users editing the same space live, over the protected `collab:` channel | dlesieur 46, shashemi 1 | osionos `src/features/collab`, `scripts/bridge-*collab*` | -| Chat | Workspace channels and direct messages, reactions, mentions, attachments, read receipts | dlesieur 72, vjan-nie 1 | osionos `src/features/chat`, `src/features/group-chat`, `scripts/bridge-*chat*` | -| Video rooms | LiveKit-based video channels inside workspaces | dlesieur 13 | osionos `scripts/bridge-*livekit*`, `src/**/*livekit*`, `src/**/*Video*` | -| Comments, sharing, publishing | Page comments, share rules, public page publishing | dlesieur 44 | osionos `src/features/comments`, `src/features/share`, `scripts/bridge-*publish*`, `scripts/bridge-*share*` | -| Communities & social feed | Communities, connections, feed with likes, comments and shares; blocking and reporting | dlesieur 38, serjimen 2 | osionos `src/features/communities`, `src/features/connections`, `scripts/bridge-*social*`, `scripts/bridge-*feed*` | -| Mail | Read Gmail inside the workspace via Google OAuth | dlesieur 35 | `apps/mail` | -| Calendar | Google Calendar events via Google OAuth | dlesieur 18 | `apps/calendar` | +| Block editor | Pages built from blocks, nested pages, templates, covers, favourites, search | dlesieur 164, vjan-nie 42, danfern3 29, rstancu 9, serjimen 8, shashemi 2 | osionos `src/features/block-editor` | +| Database block | A block that is a database, with table, board and calendar views over any mounted data source (the views live in the `notion-database-sys` submodule, `src/components/views/`) | dlesieur 125, vjan-nie 2, rstancu 2, shashemi 1, serjimen 1 | osionos `src/widgets/database*`, `src/features/*database*`, `src/shared/database*` | +| Real-time collaboration | Several users editing the same space live, over the protected `collab:` channel. Off in the default local build; `VITE_OSIO_COLLAB_SHARED=1` turns it on (osionos `src/shared/config/featureFlags.ts:73`) | dlesieur 45, shashemi 2 | osionos `src/features/collab`, `scripts/bridge-*collab*` | +| Chat | Workspace channels and direct messages, reactions, mentions, attachments, unread markers; new messages arrive over grobase's realtime WebSocket | dlesieur 72, vjan-nie 1 | osionos `src/features/chat`, `src/features/group-chat`, `scripts/bridge-*chat*` | +| Video rooms | LiveKit-based video channels inside workspaces | dlesieur 11, shashemi 1 | osionos `src/widgets/video-room`, `scripts/bridge-rtc*` | +| Comments, sharing, publishing | Page comments, share rules, public page publishing | dlesieur 45 | osionos `src/features/comments`, `src/features/share`, `scripts/bridge-*publish*`, `scripts/bridge-*share*` | +| Communities & social feed | Communities, connections, feed with likes, comments and shares; blocking and reporting | dlesieur 37, serjimen 2, shashemi 1 | osionos `src/features/communities`, `src/features/connections`, `scripts/bridge-*social*`, `scripts/bridge-*feed*` | +| Two-factor authentication and passkeys | TOTP codes and passkeys on the account | dlesieur 15 | osionos `scripts/bridge-account-mfa*`, `scripts/bridge-account-passkeys*`, `src/features/settings/profile/TwoFactorModal.tsx` | +| AI assistant | Streaming answers, retrieval over the workspace with citations, moderation, dictation and read-aloud; started with `--profile ai` | dlesieur 317, vjan-nie 9, shashemi 1 | osionos `services/osionos-ai`, `src/features/ai` | +| Whiteboard | Shared boards in osionos's Whiteboard tab; strokes reach every open copy of the board live | dlesieur 569, danfern3 20, shashemi 6, vjan-nie 4 | hydrawn, whole repository | +| Mail | Read Gmail inside the workspace via Google OAuth (mail `bridge/server.mjs`). Opt-in: `make mail-up` and Google OAuth credentials | dlesieur 39 | `apps/mail`, whole repository | +| Calendar | Google Calendar events via Google OAuth (calendar `bridge/server.mjs`). Opt-in: `make calendar-up` and Google OAuth credentials | dlesieur 22 | `apps/calendar`, whole repository | | Contract-driven provisioning | One JSON contract → isolated database, roles, keys and frontend `.env` | dlesieur 9 | grobase `scripts/provision-contract.sh`, `infra/config/contracts` | -| Self-serve applications | `POST /v1/tenants/me/apps` creates a new isolated app with its own database and key | dlesieur 33 | grobase `src/control-plane/internal/tenants` | -| Multi-engine data API | One API and one key over PostgreSQL, MySQL, MongoDB, SQLite, MSSQL, DynamoDB, Redis, HTTP | dlesieur 99, vjan-nie 3 | grobase `src/data-plane-router` | -| Public API | Key-authenticated, rate-limited API behind Kong, documented with OpenAPI at `/docs/` | dlesieur 41, vjan-nie 1 | grobase `infra/docker/services/kong`, `infra/config/openapi` | +| Self-serve applications | `POST /v1/tenants/me/apps` creates a new isolated app with its own database and key, behind `APPS_SELFSERVE_ENABLED` (off unless set) | dlesieur 32 | grobase `src/control-plane/internal/tenants` | +| Multi-engine data API | One API and one key over PostgreSQL, MySQL, MongoDB, Redis (default edition), SQLite, SQL Server (`engines` plane), HTTP and DynamoDB (opt-in build) | dlesieur 95, vjan-nie 3 | grobase `src/data-plane-router` | +| Public API | Key-authenticated, rate-limited API behind Kong, documented with OpenAPI at `/docs/` on the `:8444` edge | dlesieur 64, vjan-nie 5, shashemi 1 | grobase `infra/docker/services/kong`, `infra/config/openapi`; groot `infrastructure/tls/nginx.conf` | --- ## Modules -The subject requires **14 points** (Major = 2, Minor = 1), and a module that does not work at -the evaluation scores zero. A module is claimed below only when its evidence is something a -second person can re-run: a CI job, a test suite, or a script. Status: - -- **CI** — a CI job proves it on every change (link to the workflow or the tests it runs). -- **CI + live** — as CI, and a manual check on production is recorded, with its date. -- **Partial** — CI covers part of the subject's list; the gap is named in the row. - -The osionos links point at osionos `main`; groot's pin of `apps/osionos/app` predates some of -those tests and moves to `main` with [#172](https://github.com/Univers42/groot/issues/172). - -| # | Module | Category | Type | Pts | Status | Evidence | Who (commits) | -|---|---|---|---|--:|---|---|---| -| 1 | Backend as microservices | DevOps | Major | 2 | CI | groot [`Fresh-Machine Migration`](.github/workflows/fresh-machine.yml) runs `make all` on a clean runner and asserts every core, engine and app container healthy (green on `main`, [run 41676720776](https://github.com/Univers42/groot/actions/runs/41676720776), 2026-11-03). Demo: `docker compose ps` after `make all` | dlesieur 205, vjan-nie 2, shashemi 1 (grobase `orchestrators/compose`, `src/control-plane`) | -| 2 | Real-time features via WebSockets | Web | Major | 2 | CI | grobase CI runs gate `m201-realtime-issuer`; groot [`Modules of choice`](.github/workflows/modules-of-choice.yml) runs `m175-collab-namespace-isolation` (the realtime-core subscribe/publish authorisation tests). Demo: two users in one chat channel, a message appears without a reload | dlesieur 43, vjan-nie 5, shashemi 1 (grobase `infra/docker/services/realtime`) | -| 3 | Allow users to interact with other users (chat, profile, friends) | Web | Major | 2 | CI + live | osionos CI (`make test-canvas`, `make test-bridge`, green on `main` 2026-11-03) runs [`chat-catch-up.test.ts`](https://github.com/Univers42/osionos/blob/main/tests/canvas/chat-catch-up.test.ts) and [`bridge-chat-reply-notify.test.mjs`](https://github.com/Univers42/osionos/blob/main/tests/bridge/bridge-chat-reply-notify.test.mjs) (chat), [`bridge-account-profile.test.mjs`](https://github.com/Univers42/osionos/blob/main/tests/bridge/bridge-account-profile.test.mjs) (profile read and update), [`bridge-social-connections.test.mjs`](https://github.com/Univers42/osionos/blob/main/tests/bridge/bridge-social-connections.test.mjs) and [`bridge-connection-transitions.test.mjs`](https://github.com/Univers42/osionos/blob/main/tests/bridge/bridge-connection-transitions.test.mjs) (friends). Live on 2026-11-04 with two accounts: friend request sent, accepted, listed, removed; a DM shown in the other window without a reload ([#171](https://github.com/Univers42/groot/issues/171)) | dlesieur 72, vjan-nie 1 (chat); dlesieur 38, serjimen 2 (connections) | -| 4 | Standard user management and authentication | User Management | Major | 2 | CI + live | osionos CI runs [`bridge-avatar.test.mjs`](https://github.com/Univers42/osionos/blob/main/tests/bridge/bridge-avatar.test.mjs) (upload, type and size refusal), [`bridge-presence.test.mjs`](https://github.com/Univers42/osionos/blob/main/tests/bridge/bridge-presence.test.mjs) (online status), and row 3's profile and connections tests. Members without an image get initials (osionos `src/features/collab/model/collabPresence.ts:63`). Live: friend listed with `online=True`, profile page `200` ([#171](https://github.com/Univers42/groot/issues/171)) | dlesieur 34 (osionos `scripts/bridge-*profile*`, `*avatar*`, `*presence*`, `src/features/profile-binding`) | -| 5 | Remote authentication with OAuth 2.0 | User Management | Minor | 1 | CI + live | opposite-osiris CI (`pnpm test`, green on `main` 2026-11-03) runs [`oauth-providers.test.ts`](https://github.com/Univers42/prismatica/blob/main/tests/unit/auth/oauth-providers.test.ts) and the `gateway-oauth-*` tests. Live: `GET /api/auth/oauth/providers` answers `["google","github"]`; the working account has a linked GitHub identity ([#171](https://github.com/Univers42/groot/issues/171)). On 2026-11-04 "Continue with GitHub" led to a 404 (the site stopped forwarding GoTrue's provider paths in 85063b0); fixed in opposite-osiris dda7f95, after which the button reaches GitHub's sign-in with `redirect_uri=https://prismatica-eta.vercel.app/auth/v1/callback` and Google answers 302 to `accounts.google.com` | dlesieur 29, danfern3 1 (opposite-osiris `scripts/auth-gateway.mjs`, `src/scripts/portal-oauth.ts`, OAuth tests) | -| 6 | Use a framework for both the frontend and backend | Web | Major | 2 | CI + live | Frontend: osionos is React (osionos `package.json`). Backend: the services every app request crosses are built on axum 0.7, the data plane (grobase [`src/data-plane-router/Cargo.toml:59`](https://github.com/Univers42/grobase/blob/main/src/data-plane-router/Cargo.toml)) and realtime ([`infra/docker/services/realtime/realtime-agnostic/Cargo.toml:44`](https://github.com/Univers42/grobase/blob/main/infra/docker/services/realtime/realtime-agnostic/Cargo.toml)); grobase CI builds and tests both (`cargo test --workspace`), row 1's CI runs them. Live on 2026-11-04: `mini-baas-data-plane-router-rust` and `mini-baas-realtime` running on the production stack. Replaces the frontend-only Minor, which cannot be counted as well | dlesieur 1239, danfern3 78, vjan-nie 77, serjimen 76, rstancu 38, shashemi 8 (osionos `src`); dlesieur 126, vjan-nie 8, shashemi 1 (grobase data plane and realtime) | -| 7 | Public API (key, rate limit, docs, ≥5 endpoints) | Web | Major | 2 | CI | groot [`Public API`](.github/workflows/public-api.yml): the `:8444` edge serves the OpenAPI document at `/openapi.json` and renders it at `/docs/` ([`tests/api-docs-edge`](tests/api-docs-edge/docs.bats)); [`scripts/verify/public-api.sh`](scripts/verify/public-api.sh) shows `401` without a key, `401` with a wrong one and a `429` past the limit, against Kong 3.8.0 with grobase's plugins ([`tests/public-api`](tests/public-api/public-api.bats)). Live on 2026-11-04 (3 requests): both `401`s on `https://baas.tail10f2ea.ts.net/rest/v1/`. On a stack: `scripts/verify/public-api.sh https://localhost:8444 --key --path ` | dlesieur 41, vjan-nie 1 (grobase `infra/docker/services/kong`, `infra/config/openapi`) | -| 8 | **Module of choice:** contract-driven application factory | Free choice | Major | 2 | Partial | groot [`Modules of choice`](.github/workflows/modules-of-choice.yml) runs gates `m176-contract-isolation-live` and `m177-selfserve-apps` in their static mode (source checks and the Go slug/DSN tests). Their live half (`BAAS_VERIFY_LIVE=1`: two apps, two databases, cross-app key refused) runs in no CI job yet | dlesieur 9 (provisioning), dlesieur 33 (self-serve) | -| 9 | **Module of choice:** SSRF guard on the HTTP engine | Free choice | Minor | 1 | Partial | 30 unit tests in grobase `src/data-plane-router/crates/data-plane-pool/src/http/mod.rs`, run by grobase CI (`cargo test --workspace`). No end-to-end demo of a mount at `169.254.169.254` being refused yet, and whether it stands apart from row 8 is a team decision ([#177](https://github.com/Univers42/groot/issues/177)) | dlesieur 8 | -| 10 | Real-time collaborative features | Web | Minor | 1 | CI | osionos CI runs the `tests/bridge/bridge-collab-*.test.mjs` and `tests/canvas/collab-*.test.ts` suites; row 2's `m175` proves `collab:` is closed to wildcard tokens. Demo: two users on one page, each sees the other's cursor and edits | dlesieur 46, shashemi 1 (osionos `src/features/collab`, `scripts/bridge-*collab*`) | - -**Point calculation** - -| | Points | -|---|--:| -| Status CI or CI + live (rows 1–7, 10) | 14 | -| Status Partial (rows 8–9, the modules of choice) | 3 | -| **Total claimed** | **17** | -| Required | 14 | - -Rows 1–7 and 10 make 14 points on their own, the required total; the two modules of choice are -the margin. - -> TO CONFIRM by the team: the date and the name of the second member who ran each row's evidence -> on a fresh `make all` ([#171](https://github.com/Univers42/groot/issues/171), "Done when"). - -### Not claimed - -| Module | Category | Type | Why not | -|---|---|---|---| -| Advanced permissions | User Management | Major | Admin › Users answers `403` for the platform admin (2026-11-04 audit); needs `PERMISSION_CONDITIONS_ENABLED`, `API_KEY_ABAC_ENABLED`, migration `063` | -| Organisation system | User Management | Major | Organisations are on in production, but no management screen exists | -| Monitoring with Prometheus + Grafana | DevOps | Major | Needs `TENANT_OBS_ENABLED` and `DATA_PLANE_TENANT_OBS`; nothing to click | -| Health/status page, backups, disaster recovery | DevOps | Minor | `TENANT_BACKUP_ENABLED`, migration `042`; nothing to click | -| GDPR compliance | Data | Minor | The deletion executor is open work (Univers42/osionos#56) | -| Two-factor authentication | User Management | Minor | TOTP or passkeys need `PASSKEYS_ENABLED`; not on by default | -| File upload and management | Web | Minor | Uploaded media is readable without authentication (Univers42/osionos#233, #235), which fails "secure storage with access control" | -| Seven Artificial Intelligence modules | Artificial Intelligence | 3 Major, 4 Minor | Each needs a compose profile or flag that is off on the public stack: [M1 images](https://github.com/Univers42/osionos/blob/develop/docs/ai/modules/m1-images.md), [M2 RAG](https://github.com/Univers42/osionos/blob/develop/docs/ai/modules/m2-rag.md), [M3 recommendations](https://github.com/Univers42/osionos/blob/develop/docs/ai/modules/m3-recs.md), [M4 moderation](https://github.com/Univers42/osionos/blob/develop/docs/ai/modules/m4-moderation.md), [M5 voice](https://github.com/Univers42/osionos/blob/develop/docs/ai/modules/m5-voice.md), [M6 sentiment](https://github.com/Univers42/osionos/blob/develop/docs/ai/modules/m6-sentiment.md), [M7 image tags](https://github.com/Univers42/osionos/blob/develop/docs/ai/modules/m7-images-tags.md) | -| AI opponent | Artificial Intelligence | Major | osionos has no game | -| Multiple languages | Accessibility and Internationalization | Minor | Switching language only records a stub action (`i18n_change_stub` in osionos `src/features/settings/SettingsCenter.tsx`); no translation resources are loaded | +The subject requires **14 points** (Major = 2, Minor = 1). A module that cannot +be demonstrated live scores zero. Every row is graded against **the local stack +built from `main`** (`git checkout main`, `make all SKIP_SYNC=1`, which starts +grobase's `devlean` edition; see [Defense bring-up](#defense-bring-up-pinned)) +by **one rule**, groot's own. It keeps the clause of osionos +[`docs/ai/README.md`](https://github.com/Univers42/osionos/blob/1c1d5d849c1dc43460a194f7ee3d97e17505205c/docs/ai/README.md) +lines 7–8 (a row is never A when its demonstration needs something switched off +or unmeasured), but it is stricter than that file's legend, which grades a +documented profile A: + +- **A**: the demo works on that stack with nothing switched on, and a green CI + job drives that path against a running service. +- **B**: it works, but the demo needs something switched on (a flag, a profile, + an edition, a provider credential), or its only evidence is unit tests with + fakes or a run that was never repeated. +- **C**: partial. **Not claimed.** These rows are listed under [Not claimed + yet](#not-claimed-yet). + +**Claimed = A and B only: 30 points against the 14 required. 6 of them are A.** +Paths are prefixed by repository: `G` groot, `GB` grobase (`apps/grobase`), `OS` +osionos (`apps/osionos/app`), `OO` opposite-osiris (`apps/opposite-osiris`). +They point at the commits pinned by groot's `main` branch. CI names a workflow +job: `GB-CI` is grobase `ci.yml`, `OS-CI` is osionos `ci.yml`, `OO-CI` is +prismatica `ci.yml`. + +| # | Module | Category | Type | Pts | Grade | Core files | Tests → CI job | Docs | Demo (3 steps) | +|---|---|---|---|--:|:--:|---|---|---|---| +| 1 | Public API (key, rate limit, docs, ≥5 endpoints) | Web | Major | 2 | **A** | `GB infra/docker/services/kong/conf/kong.yml:221-245` (`/rest/v1`: key-auth + rate-limiting); `GB infra/config/openapi/grobase-public.json` | `GB scripts/verify/m102-gateway-query-path.sh` → GB-CI "Integration Tests" (`ci.yml:927`) | `GB infra/config/openapi/README.md` | `make all SKIP_SYNC=1` → `/rest/v1` without `apikey` answers 401, with it 200 → repeat until 429 | +| 2 | Backend as microservices | DevOps | Major | 2 | **A** | `GB orchestrators/compose/base/*.yml` (16 files); `GB orchestrators/makes/30-planes.mk:33`; `GB src/control-plane`; `GB src/data-plane-router` | GB-CI unit suites (`ci.yml:1436`) + "Engine conformance" legs (`ci.yml:1532`) | [`wiki/platform/02-planes.md`](wiki/platform/02-planes.md) | `make -C apps/grobase planes` → `docker compose ps` → `make healthcheck` | +| 3 | Real-time features via WebSockets | Web | Major | 2 | **B** | `GB infra/docker/services/realtime/realtime-agnostic/crates/realtime-server/src/server.rs:243` (`/ws`); `GB kong.yml:314-315` (`/realtime/v1/ws`); `OS scripts/bridge-social-core.mjs:192-201` (the bridge publishes); `OS src/services/realtime-messages/wsTransport.ts:81` (chat over the socket, on whenever a session exists) | `GB scripts/verify/m201-realtime-issuer.sh` → GB-CI "Integration Tests" (`ci.yml:855`): it checks the `/ws` handshake (`AUTH_OK`) on the running realtime, not delivery between two users; m189 (`ci.yml:735`); `rust-realtime` suite and m184 (`ci.yml:1436,1479`). osionos's chat browser test runs against a stub bridge (OS-CI "Browser tests (stub bridge URL)", `ci.yml:432-433`) | [`wiki/platform/05-realtime.md`](wiki/platform/05-realtime.md) | sign in as two users in two browsers → open the same direct message in both → a message sent in one appears in the other without a reload. A second default demo: two windows on one Whiteboard board, where each stroke appears in the other (`G tests/e2e/specs/dw8-realtime.spec.ts`, `make e2e`, run by hand) | +| 4 | Use a framework for both the frontend and backend | Web | Major | 2 | **A** | Frontend: React 19 (`OS package.json:105-106`). Backend: NestJS 11, grobase's application plane (`GB src/package.json:52`; `GB src/apps/query-router/src/main.ts:30`, `NestFactory`); axum 0.7 is a supporting example (`GB src/data-plane-router/Cargo.toml:59`) | React: OS-CI "Browser tests", Playwright against the served app in 6 shards (`ci.yml:338-339`). NestJS: GB-CI "Integration Tests" builds query-router from the pin and starts it (`ci.yml:900-904`), then m213 sends `/query/v1/execute` through Kong to it (`ci.yml:917`); the `nestjs` unit suite (`ci.yml:1436`) | [`wiki/platform/02-planes.md`](wiki/platform/02-planes.md) | `make all SKIP_SYNC=1` → `docker compose ps` lists `mini-baas-query-router` (`GB orchestrators/compose/base/app-services.yml:12`, in the `devlean` edition) → open osionos at `https://localhost:3001` | +| 5 | Allow users to interact with other users (chat, profile, friends) | Web | Major | 2 | **B** | `OS scripts/bridge-chat.mjs`; `OS scripts/bridge-account-profile.mjs`; `OS scripts/bridge-social.mjs`; `OS src/features/chat`; `OS src/features/connections` | `OS tests/canvas/chat-catch-up.test.ts` → OS-CI "quality (root)" Canvas unit tests (`ci.yml:285`); `OS tests/bridge/bridge-chat-reply-notify`, `bridge-account-profile`, `bridge-social-connections`, `bridge-connection-transitions` → Bridge tests (`ci.yml:291`). Unit tests with fakes: no CI job drives two users against a running stack | [`wiki/product/02-osionos.md`](wiki/product/02-osionos.md) | as row 3 → user A sends a friend request, user B accepts → each opens the other's profile | +| 6 | Standard user management and authentication | User management | Major | 2 | **B** | `OS scripts/bridge-account-profile.mjs`; `OS scripts/bridge-avatar.mjs`; `OS scripts/bridge-presence.mjs`; `OS src/features/collab/model/collabPresence.ts:63` (initials when there is no avatar) | `OS tests/bridge/bridge-avatar` (upload, type and size refusal), `bridge-presence` (online status), `bridge-account-profile` → OS-CI Bridge tests (`ci.yml:291`). Unit tests with fakes | [`wiki/product/02-osionos.md`](wiki/product/02-osionos.md) | sign up on `https://localhost:4322` → upload an avatar and edit the profile → a friend sees you online | +| 7 | Real-time collaboration | Web | Minor | 1 | **B** | `OS src/features/collab/model/{remoteCursors,collabPresence,blockSync}.ts`; `OS scripts/bridge-collab.mjs:626`; `GB …/realtime-auth/src/jwt/mod.rs:90-94` (`collab:` closed to wildcard tokens) | `OS tests/bridge/bridge-collab-*` (9) → OS-CI Bridge tests (`ci.yml:291`) | [`wiki/product/02-osionos.md`](wiki/product/02-osionos.md) | rebuild osionos with `VITE_OSIO_COLLAB_SHARED=1` (off by default) → two browsers on one page → watch the remote cursor and edit the same block from both | +| 8 | Advanced permissions | User management | Major | 2 | **B** | `OS scripts/bridge-roles.mjs:28-29`; `OS scripts/bridge-share-rules.mjs:38`; `OS scripts/bridge-page-access.mjs:217`; `OS src/features/admin-users/api.ts` | `OS tests/bridge/bridge-page-access*` (7), `bridge-share-rules-*` (2) → OS-CI Bridge tests | [`wiki/platform/07-isolation.md`](wiki/platform/07-isolation.md) | sign in as admin → change a member's role → that member loses access | +| 9 | Organisation system | User management | Major | 2 | **B** | `OS scripts/bridge-workspace-teams.mjs:229`; `OS scripts/bridge-workspace-invites.mjs:134`; `OS src/features/settings/teamspaces/TeamspacesPanel.tsx` | `OS tests/bridge/bridge-workspace-*` (7) → OS-CI Bridge tests | [`wiki/platform/06-contracts.md`](wiki/platform/06-contracts.md) | create a workspace → create a teamspace → invite a user, who accepts | +| 10 | Remote authentication with OAuth 2.0 | User management | Minor | 1 | **B** | `OO scripts/auth/oauth-providers.mjs`; `OO scripts/auth/oauth-link.mjs`; `OO src/scripts/portal-oauth.ts`; `GB orchestrators/compose/base/auth-api.yml:64-67` (GoTrue's Google provider) | `OO tests/unit/auth/oauth-providers`, `oauth-link`, `gateway-oauth-{start,link,collision}` (5) → OO-CI "Quality Gate" `pnpm test` (`ci.yml:31-32`) | `OO docs/adr/0002-oauth-link-existing-account.md` | set `GOOGLE_OAUTH_ENABLED=true`, `GOOGLE_CLIENT_ID` and `GOOGLE_CLIENT_SECRET` for grobase (Google provider credentials, not shipped) → `make all SKIP_SYNC=1` → "Continue with Google" on `https://localhost:4322` | +| 11 | Health/status page, backups, disaster recovery | DevOps | Minor | 1 | **B** | `GB infra/docker/services/status-page/{server,availability}.py`; `GB orchestrators/compose/base/ops.yml`; `GB scripts/verify/m241-status-backup-recovery.sh` | m241 → GB `operational-reliability.yml:27`; m192 → GB-CI (`ci.yml:582`), **red on grobase `develop`** ([run 41870904860](https://github.com/Univers42/grobase/actions/runs/41870904860)) | [`wiki/operations/04-troubleshooting.md`](wiki/operations/04-troubleshooting.md) | bring up the `ops` plane (not in the default edition) → open the status page → stop a container, then `m241 --live` | +| 12 | GDPR compliance | Data | Minor | 1 | **B** | `OS scripts/bridge-account-export.mjs:143`; `OS scripts/bridge-account-erasure.mjs:79`; `OS src/features/settings/ui/AccountExportSection.tsx`; `OS src/features/settings/ui/danger/AccountDangerZone.tsx` | `OS tests/bridge/bridge-account-export*` (3), `bridge-account-erasure` (1) → OS-CI Bridge tests; `G .github/workflows/models-grants.yml` | [`wiki/security/01-model.md`](wiki/security/01-model.md) | Settings › export my data → open the archive → delete the account | +| 13 | Two-factor authentication | User management | Minor | 1 | **B** | `OS scripts/bridge-account-mfa-totp.mjs:68`; `OS scripts/bridge-account-mfa.mjs:187`; `OS scripts/bridge-account-passkeys.mjs:100`; `OS src/features/settings/profile/TwoFactorModal.tsx` | `OS tests/bridge/bridge-account-mfa-*` (5), `bridge-account-passkeys` (1) → OS-CI Bridge tests | osionos `docs/account-platform` | enable TOTP → sign out → sign in with the code | +| 14 | **Module of choice:** contract-driven application factory | Free choice | Major | 2 | **B** | `GB src/control-plane/internal/tenants/selfserve_apps.go:55-58`; `GB src/control-plane/cmd/tenant-control/mount_appsselfserve.go:26`; `GB scripts/verify/m177-selfserve-apps.sh` | m176 → GB-CI (`ci.yml:192`, static leg, in the Shellcheck job); m176 and m177 → groot `modules-of-choice.yml:39-40`, static mode only | [`wiki/platform/06-contracts.md`](wiki/platform/06-contracts.md), [justification](#justification--module-of-choice-major-contract-driven-application-factory) | set `APPS_SELFSERVE_ENABLED` (empty by default) → OTP login, `POST /v1/tenants/me/apps` twice → one app's key reads the other: 4xx | +| 15 | **Module of choice:** SSRF guard on the HTTP engine | Free choice | Minor | 1 | **B** | `GB src/data-plane-router/crates/data-plane-pool/src/dial_guard/classify.rs:90-155`; `GB src/data-plane-router/crates/data-plane-pool/src/http/validate.rs:83-153` | `GB …/dial_guard/classify_tests.rs` → GB-CI `rust-data-plane` + "Engine conformance (http)" | [`wiki/security/04-network.md`](wiki/security/04-network.md), [justification](#justification--module-of-choice-minor-ssrf-guard-on-the-http-engine) | mount an `http` source at `169.254.169.254` → query it → refused. Whether the default edition mounts the `http` engine is unverified (guess: it does not) | +| 16 | Complete LLM system interface: text, streaming, error handling, rate limiting. Images exist but are **off by default** and not part of this claim | Artificial Intelligence | Major | 2 | **B** | osionos `services/osionos-ai/src/routes/runs.ts:178-324`, `src/runs/{runner,upstream,upstreamError,streamPump}.ts`, `src/rateLimit.ts:84`, `src/http.ts:108-129`; client `src/features/ai/model/runStream.ts`. All 12 hops: [`wiki/project/ai-llm-interface.md`](wiki/project/ai-llm-interface.md) | `services/osionos-ai/tests/{runs,complete,upstreamError*,runRateLimit,failover,quotaFailover}.test.ts` → OS-CI "AI service (tsc + suite)" (`ci.yml:314`); `tests/canvas/ai-{chat-store,run-dispatch,run-stream}.test.ts` → "quality (root)" Canvas unit tests (`ci.yml:285`) | [`wiki/project/ai-llm-interface.md`](wiki/project/ai-llm-interface.md), [`wiki/runbooks/osionos-ai.md`](wiki/runbooks/osionos-ai.md), [M1 sheet](https://github.com/Univers42/osionos/blob/develop/docs/ai/modules/m1-images.md) | `docker compose --profile ai up -d --build` → ask a question and watch it stream → reload mid-answer (it resumes), or stop it | +| 17 | Complete RAG on a large dataset | Artificial Intelligence | Major | 2 | **B** | osionos `services/osionos-ai/src/retrieval/search.ts:277,469,564`, `src/retrieval/rerank.ts:16`; `G models/osionos-ai-rag-migration.sql:60,91` | `services/osionos-ai/tests/{retrieval,aclAgreement,rerank}.test.ts` → OS-CI AI service | [M2 sheet](https://github.com/Univers42/osionos/blob/develop/docs/ai/modules/m2-rag.md), osionos `services/osionos-ai/eval/wiki/RESULTS-rag.md` | `--profile ai` → ask a workspace question → click a citation, then show the 110 363-chunk measurement (run on a separate eval stack; the reranker is off by default) | +| 18 | Content moderation AI | Artificial Intelligence | Minor | 1 | **B** | osionos `scripts/bridge-moderation-gate.mjs:70`; `services/osionos-ai/src/moderation/l1.ts:99` | `services/osionos-ai/tests/moderation*.test.ts` (4) → OS-CI AI service; `tests/bridge/bridge-moderation*` (5) → Bridge tests | [M4 sheet](https://github.com/Univers42/osionos/blob/develop/docs/ai/modules/m4-moderation.md) | set `OSIONOS_MODERATION_ACTIONS=hold` and `OSIONOS_MODERATION_STATE=1` → post a toxic comment → it waits in the admin console (no auto-removal, by design) | +| 19 | Voice / speech | Artificial Intelligence | Minor | 1 | **B** | osionos `services/osionos-ai/src/routes/voice.ts:104`, `src/features/voice/ui/DictationButton.tsx`, `src/features/voice/model/readAloud.ts`; `G docker-compose.yml:788-790` | `services/osionos-ai/tests/voice.test.ts` → OS-CI AI service | [M5 sheet](https://github.com/Univers42/osionos/blob/develop/docs/ai/modules/m5-voice.md) | `--profile ai`, with the speech model fetched into its volume → dictate into a page → read it aloud (read-aloud voices not measured yet) | + +No separate framework Minor is claimed beside row 4. + +**Point calculation** (free choice is capped at 3, +[`wiki/project/01-subject.md`](wiki/project/01-subject.md#points)) + +| | Before (2026-10) | After | +|---|--:|--:| +| Catalogue modules claimed (rows 1–13) | 18 | 21: 6 A (rows 1, 2, 4) + 15 B (rows 3, 5–13) | +| Modules of choice (rows 14–15, cap 3) | 3 | 3, all B | +| Artificial Intelligence modules (rows 16–19) | 10 | 6, all B | +| **Total claimed** | **31 listed** | **30 (A + B)** | +| Of which A | 10 | **6** | +| Required | 14 | 14 | + +What moved: under the old legend ("works as-is"), six rows were A. Under one +rule, the rows whose demonstration needs a switch or whose only evidence is +unit-level are B: WebSockets (CI checks the handshake, not delivery), real-time +collaboration (off by default), the contract factory and the SSRF guard. Row 4 +is new at A. The rows below no longer count. + +### Not claimed yet + +| Module | Was | Why it is not claimed, in one line | +|---|---|---| +| Monitoring with Prometheus + Grafana, DevOps, Major | 2, C | The default edition ships no observability plane, and its gate m85 runs in no workflow | +| File upload and management, Web, Minor | 1, C | Avatars and page files upload (`OS scripts/bridge-media.mjs:160`), but no general file manager is verified, and gate m159 runs in no workflow | +| ML recommendation system, AI, Major | 2, B | Needs 20+ real users, the offline numbers are synthetic, and the `osionos-ml` Python tests run in no CI | +| Sentiment analysis, AI, Minor | 1, C | The model licence is undeclared | +| Image recognition and tagging, AI, Minor | 1, C | The browser flow is unverified. Locally the VLM starts with `--profile ai` (`G docker-compose.yml:1004-1006`); `--profile ai-vlm` exists only in `docker-compose.public.yml:211-212` | +| AI opponent, AI | — | osionos has no game | +| Multiple languages, Accessibility | — | `i18next` is a dependency (osionos `package.json:92`), but no source file imports it and no translation resource exists | ### Justification — Module of choice (Major): contract-driven application factory @@ -640,14 +757,11 @@ the margin. database, apply its schema, create roles and policies, mint scoped API keys and emit the frontend's `PUBLIC_*` configuration — idempotently, on every boot. The self-serve endpoint `POST /v1/tenants/me/apps` does it on demand, with a fresh `CREATE DATABASE` per application. -- **Value.** A new application costs a contract instead of a new server. The public site and - vault42 are both provisioned this way in production. +- **Value.** A new application costs a contract instead of a new server. - **Why it deserves a Major.** It spans the Go control plane, the Rust data plane and the database layer, and it comes with isolation proofs: gate `m177` proves self-serve creation, and gate `m176` proves one application cannot read another's data (a foreign mount resolves to zero - rows — a 404, never a cross read). Both run in groot's - [`Modules of choice`](.github/workflows/modules-of-choice.yml) workflow in their static mode; - their live half needs a running stack (`BAAS_VERIFY_LIVE=1`). + rows — a 404, never a cross read). ### Justification — Module of choice (Minor): SSRF guard on the HTTP engine @@ -657,9 +771,9 @@ the margin. the cloud metadata endpoint `169.254.169.254` — CGNAT, IPv6 ULA and link-local, IPv4-mapped addresses and internal hostnames, then **pins** the connection to the validated public IP so a later DNS rebind cannot redirect it inward. -- **Value and weight.** Small and self-contained, and it shows security judgement on a feature we - chose to build. Its 30 unit tests run in grobase CI; the end-to-end demo (a mount pointing at - `169.254.169.254` is refused) is not recorded yet ([#177](https://github.com/Univers42/groot/issues/177)). +- **Value and weight.** Small, self-contained and demonstrable in seconds (a + mount pointing at `169.254.169.254` is refused), and it shows security + judgement on a feature we chose to build. --- @@ -667,28 +781,27 @@ the margin. ### Who committed where -Commits per member reachable from the commit groot pins for each repository, one line per person, -read through the root [`.mailmap`](.mailmap) (counted on 2026-11-04 at root `01654173`): - -```sh -git -C -c mailmap.file="$PWD/.mailmap" shortlog -sn # pins: git ls-tree HEAD apps vendor -``` +Non-merge commits per member reachable from the commit groot `main` pins for +each repository, read through the root [`.mailmap`](.mailmap), bots and AI-agent +identities left out (counted on 2026-11-05 at `main` = `0bef3cb3`). The command, +the identity joins and what is left out: +[`wiki/project/contributions-method.md`](wiki/project/contributions-method.md). Several members work almost only in submodules, so the root alone understates them. | Member | GitHub | root | osionos | grobase | opposite-osiris | hydrawn | mail | calendar | QA | monkey-bot | First → last commit | |---|---|--:|--:|--:|--:|--:|--:|--:|--:|--:|---| -| `dlesieur` | @LESdylan | 1577 | 3375 | 1569 | 126 | 676 | 42 | 27 | 1 | 2 | 2026-05-10 → 2026-11-04 | -| `danfern3` | @danielfdez17 | | 94 | 74 | 8 | 25 | | | 1 | 10 | 2026-03-24 → 2026-10-02 | -| `serjimen` | @DjSurgeon | | 81 | | 160 | | | | | | 2026-04-11 → 2026-10-29 | -| `vjan-nie` | @vjan-nie | 173 | 141 | 49 | 23 | 6 | | | 39 | | 2026-03-19 → 2026-11-01 | -| `shashemi` | @SMOSTAFAH1 | 59 | 18 | 18 | 59 | 9 | | | | | 2026-09-20 → 2026-11-02 | -| `rstancu` | @esettes | | 104 | | | | | | 2 | | 2026-03-18 → 2026-05-02 | - -Work that is not on a pinned commit does not show above: `danfern3`'s 44 root commits sit on -`fix/ci`, open as [#165](https://github.com/Univers42/groot/pull/165), and most of their 133 -monkey-bot commits are on branches the pin does not reach. A commit count is not effort: one -large commit and one typo fix weigh the same, and pairing and review do not show in it. +| `dlesieur` | @LESdylan | 1394 | 2469 | 1279 | 103 | 569 | 39 | 22 | 1 | 2 | 2026-04-05 → 2026-11-05 | +| `vjan-nie` | @vjan-nie | 136 | 119 | 29 | 12 | 4 | | | 38 | | 2026-03-19 → 2026-11-04 | +| `serjimen` | @DjSurgeon | | 82 | | 134 | | | | | | 2026-04-09 → 2026-11-03 | +| `danfern3` | @danielfdez17 | | 84 | 73 | 4 | 20 | | | 1 | 10 | 2026-03-18 → 2026-10-02 | +| `shashemi` | @SMOSTAFAH1 | 42 | 17 | 15 | 39 | 8 | | | | | 2026-09-20 → 2026-11-04 | +| `rstancu` (former member) | @esettes | | 90 | | | | | | 2 | | 2026-03-18 → 2026-05-02 | + +Dates are author dates in these nine repositories only. Work that is not on a +pinned commit (open branches and pull requests) does not show above. A commit +count is not effort: one large commit and one typo fix weigh the same, and +pairing and review do not show in it. Merged pull requests per login in the project's repositories (groot, osionos, grobase, prismatica = opposite-osiris, osionos-mail, osionos-calendar, hydrawn, QA, monkey-bot), from @@ -702,128 +815,37 @@ prismatica = opposite-osiris, osionos-mail, osionos-calendar, hydrawn, QA, monke | @DjSurgeon | 8 | prismatica 7, osionos 1 | | @danielfdez17 | 4 | prismatica 4 | -The Who columns of the [Features List](#features-list) and [Modules](#modules) were counted -on 2026-11-04, before the history rewrite of that day, with `scripts/contributors.sh --paths` -(all branches). Until the pre-rewrite refs are deleted from a clone, `--all` counts each of those -commits twice; count from the pinned commit as above instead. - -How the identities were joined (the evidence is in [`.mailmap`](.mailmap)): - -- Each member committed under several addresses, including GitHub's `noreply` ones; `.mailmap` - joins them. A commit count is not effort: one large commit and one typo fix weigh the same. -- `settes ` is **`rstancu`** (Roxana Stancu): the address belongs to GitHub - account [@esettes](https://github.com/esettes), listed for `rstancu` on the original roster - ([`archive/wiki-2026-09/CONTRIBUTORS.md`](archive/wiki-2026-09/CONTRIBUTORS.md)). The 104 - osionos commits (2026-04-10 → 2026-05-02) touch mostly the Playwright browser tests - (`tests/browser`), their CI job and committed build assets. `rstancu` is not in the current - team table. - - > TO CONFIRM by the team: `rstancu`'s status in the team, and whether that work counts for - > this evaluation. -- `La Cebolla ` is **`shashemi`**: opposite-osiris commit `93e6b65` carries - `Signed-off-by: Seyed Mostafa Hashemian`. -- Two groups of grobase commits pair one member's name with another's address. They are - attributed by inference: - - `LESdylan ` (3 commits on grobase's pinned commit, 5 on all branches, - 2026-09-21 → 2026-11-02) goes to `dlesieur`: the author name - is `LESdylan`, and `b4c1d33f` landed through grobase PR #89, opened by `LESdylan`. - - `dlesieur ` (5 commits, 2026-03-20/21) goes to `danfern3`: they sit inside - `danielfdez17`'s own Docker and Kubernetes series of the same two days. - - > TO CONFIRM by the team: both attributions above. -- `opencode` (6 commits: 1 root, 5 osionos) is an AI coding agent committing under its own name. -- `bunny`, `Space Bunny` and `builder` (111 hydrawn commits, 2026-09-27/28, addresses on - `local`, `localhost` and `drawnosaurus.local`) are not mapped: no address, sign-off or pull - request ties them to a member. - - > TO CONFIRM by the team: which member ran `opencode`, and who committed as `bunny`, - > `Space Bunny` and `builder` in hydrawn. -- Bots (`dependabot`, `coderabbitai`, `copilot-swe-agent`) are left as they are. -- On 2026-11-04 the history of groot, osionos and grobase was rewritten to remove - `Co-authored-by` trailers; trees and subjects are unchanged, and every commit from the first rewritten one onward has a new SHA. 148 root commits on `main` have a one-word subject - (147 `dlesieur`, 1 `vjan-nie`, 2026-05-01 → 2026-10-09); the - [`Commit subjects`](.github/workflows/commit-subjects.yml) check refuses new ones. - -### dlesieur — Dylan Lesieur - -- **Areas:** grobase (Go control plane, Rust data plane and realtime, Kong gateway, contracts - and self-serve apps), osionos (editor, chat, collaboration, social, profile, the bridge), mail, - calendar, the root's infrastructure, TLS edge and CI; hydrawn (676 commits: `apps/web`, - the engine, `docs/reference`). -- **Pull requests:** 260 merged (osionos 138, grobase 61, groot 56). -- **Features and modules:** most rows of the [Features List](#features-list) and every claimed - module row, as the Who columns show. -- **Challenges:** not recorded. - - > TO CONFIRM by the team: the challenges this member met. - -### danfern3 — Daniel Fernández - -- **Areas:** `vendor/monkey-bot` (133 commits on all branches, 10 on the pinned commit: form-XSS - and mutation testing, configuration); - grobase Docker images built from scratch, Kubernetes deployment, the playground and Makefile - (March 2026); osionos breadcrumbs, trash, multi-line text in blocks, the markdown engine and - code-quality fixes; hydrawn (25 commits, `apps/web`, `apps/api`, `packages/contract`, - 2026-09-24: submodule clone paths, formatting); root submodule pins and the `make all` bats fix, - on `fix/ci` ([#165](https://github.com/Univers42/groot/pull/165), open). -- **Pull requests:** 4 merged (prismatica). -- **Features and modules:** block editor (29 commits), frameworks, frontend half (row 6, 78), OAuth - (row 5, 1). -- **Challenges:** not recorded. - - > TO CONFIRM by the team: the challenges this member met. - -### serjimen — Sergio Jiménez de la Cruz - -- **Areas:** the public site, opposite-osiris (160 commits on the pinned commit: hero animation - performance, unit tests and benchmarks, security fixes, auth pages); osionos accessibility fixes, page links, - move and delete, shared UI refactor, the node installer's self-update (#277) and database probe - diagnostics (#278), the changelog. -- **Pull requests:** 8 merged (prismatica 7, osionos 1), among them the landing page in English, - Spanish and Arabic with right-to-left layout (prismatica #21, #22, #23, #35), the accessible - privacy policy and terms (#38) and a WCAG 2.2 AA audit pipeline with axe-core fixes (#39). -- **Features and modules:** sign-in pages (1), legal pages (2), block editor (14), connections - (2), frameworks, frontend half (row 6, 76). -- **Challenges:** not recorded. - - > TO CONFIRM by the team: the challenges this member met. - -### vjan-nie — Vadim Jan Nieto - -- **Areas:** `vendor/QA` (39 commits: registry, migrations, hooks, CLI); root end-to-end tests, - the wiki and README, graph-studio tests, the hydrawn TLS edge and its CORS, submodule - pins; osionos whiteboard and graph-studio fixes; grobase Docker infrastructure and dependency - fixes; hydrawn (6 commits: engine bumps, the nested-shape binding fix). -- **Pull requests:** 72 merged (groot 45, osionos 17, prismatica 6, grobase 4). -- **Features and modules:** block editor (42), legal pages (3), real-time (row 2, 5), - frameworks, frontend half (row 6, 77). -- **Challenges:** not recorded. - - > TO CONFIRM by the team: the challenges this member met. - -### shashemi — Seyed Mostafa Hashemian - -- **Areas:** security fixes across root, grobase and opposite-osiris (SAST findings, nginx scan - suppressions, IDE socket proxy hardening); the site's legal pages, SEO and accessibility; root - CI triggers and the models' export erasure paths; osionos end-to-end fixes; hydrawn (9 - commits, `apps/web`, `apps/api`, `packages/contract`: trail and notes, hydrawn #1, and the - pen-thickness-with-zoom test, #8). -- **Pull requests:** 57 merged (prismatica 20, groot 19, osionos 12, grobase 6). -- **Features and modules:** legal pages (5), collaboration (row 10, 1), real-time (row 2, 1). -- **Challenges:** not recorded. - - > TO CONFIRM by the team: the challenges this member met. - -### Team-level challenges - -From the project's history: - -- **Migrating the data plane from TypeScript to Rust with no downtime**, using a per-request - switch and a shadow mode that compared both implementations under real traffic. -- **Keeping eight database adapters at parity**, enforced by a conformance battery that runs every - adapter against a real engine and checks it serves exactly what it advertises. -- **Auditing our own authorisation path** and shipping a reversible mitigation for a weakness we - found (see [`wiki/security/03-known-weaknesses.md`](wiki/security/03-known-weaknesses.md)). +### Per member + +- **`dlesieur`**, Product Owner: grobase (control plane, data plane, realtime, + gateway, contracts), osionos (editor, chat, collaboration, the bridge, the AI + service), mail, calendar, the root's infrastructure and CI. +- **`vjan-nie`**, Project Manager: QA tooling, end-to-end tests, the wiki and + README, submodule pins, Docker infrastructure. +- **`serjimen`**, frontend tech lead: the public site (opposite-osiris), + osionos accessibility and editor features. +- **`danfern3`**, backend tech lead: grobase Docker images and Kubernetes + deployment, monkey-bot, osionos editor features. +- **`shashemi`**, QA / quality tech lead: security fixes and SAST, the site's + legal pages, the TLS edge, end-to-end fixes. + +### Challenges and lessons + +The project met many productive problems and blockers: lack of autonomy, +communication problems, frustration. We addressed them with dedicated meetings +and by applying TDD and DevOps practices, toward a frictionless workflow. We +acknowledge our technical debt openly +([`wiki/security/03-known-weaknesses.md`](wiki/security/03-known-weaknesses.md) +is one record of it). Several technologies and architectures were rebuilt from +scratch when the first choice proved wrong late: the data plane moved from +TypeScript to Rust behind a per-request switch (`RUST_DATA_PLANE_FORWARD`, on +by default in grobase `orchestrators/compose/base/app-services.yml:96`); +grobase's Node session, email and GDPR services were retired and their work +moved to the Go orchestrator (`app-services.yml:133,190,448`); and osionos's +legacy second brain was replaced by graph_render (groot `fresh-machine.yml:363` +checks it on a live stack; the legacy one survives only behind a gate, osionos +`ci.yml:383-388`). The project's maturity comes from that intense team +experience, with its ups and downs. --- @@ -850,38 +872,50 @@ From the project's history: This project was built with **heavy AI assistance**, and we state it plainly. -- **Tools.** Team members used AI assistants in different forms, according to their own work. -- **Code.** AI wrote and refactored code across the stack: grobase planes and adapters, the - frontends, SQL migrations, the Makefile pipeline and the Docker setup. -- **Testing and verification.** AI wrote verification gates and test scripts, including the - Playwright smoke in [`tests/e2e/`](tests/e2e/), and ran them against the live stack. -- **Reviews.** AI-assisted code review of pull requests and security review of the authorisation - path ([`wiki/security/03-known-weaknesses.md`](wiki/security/03-known-weaknesses.md)). -- **Documentation.** The wiki and this README were drafted and distilled with AI assistance from - the project's own design documents and code, then checked against the repository. -- **How we kept control.** A human reviews and merges every pull request; nothing reaches - `develop` or `main` by an agent alone. A change is accepted only when its verification gate - passes. Every member must be able to explain the AI-assisted code in their area. +- **Tools and tasks.** Every member used a variety of AI tools (chat + assistants, coding agents, review bots) across code, tests, reviews and + documentation. +- **What the repositories record.** Claude Code with the team's kit in groot's + [`.claude`](.claude) submodule; Gemini (`.gemini/`) and the opencode agent in + osionos; an OpenCode agent team on hydrawn (`BUNNY.md`); GitHub Copilot and + CodeRabbit on pull requests. +- **Commit history.** On 2026-11-04 the history of groot, osionos and grobase + was rewritten to remove `Co-authored-by` trailers. Since then no commit + carries a co-author line, so the history does not show which commits were + written with an AI tool, and must not be read as AI-free + ([`wiki/project/contributions-method.md`](wiki/project/contributions-method.md)). +- **How we kept control.** A human merges every pull request; nothing reaches + `develop` or `main` by an agent alone. Approval by a second member is required + by the rulesets but thin in practice (see [Project + Management](#project-management)). A change is accepted only when its + verification gate passes. Every member must be able to explain the AI-assisted + code in their area. --- ## Known limitations - **No game.** All gaming-dependent modules are out of scope. -- **Expressiveness is narrow by design.** grobase has no place for custom server logic; business - rules must be expressed as policies, schema constraints or database triggers. +- **Expressiveness is narrow by design.** grobase has no per-application server + code; business rules are expressed as policies, schema constraints, database + triggers, or functions in the flag-gated functions plane. - **Documented security weaknesses.** Known and declared in [`wiki/security/03-known-weaknesses.md`](wiki/security/03-known-weaknesses.md). -- **Several features are behind flags that are off by default** — see the confidence column in - [Modules](#modules). -- **No interface translation.** The language setting in osionos is a stub (see [Modules](#modules)). +- **Several features are behind flags that are off by default** — see the Grade + column in [Modules](#modules). +- **No interface translation in osionos.** `i18next` is declared but no + translation resource is loaded (see [Modules](#modules)). The prismatica site + itself ships English, Spanish and Arabic. - **Whiteboard editing is loopback-only and has no per-user authorization yet** (see [Demo](#demo)). ## License -There is no `LICENSE` file at the repository root. grobase carries its own licence files inside -its submodule (`apps/grobase/LICENSE`, `apps/grobase/LICENSING.md`). +- There is no `LICENSE` file at the repository root. +- grobase is open-core under the GNU AGPL v3 (`apps/grobase/LICENSE`); + `apps/grobase/LICENSING.md` maps which parts carry which licence. +- osionos, opposite-osiris (prismatica), hydrawn, mail and calendar carry no + licence file. --- @@ -894,3 +928,5 @@ its submodule (`apps/grobase/LICENSE`, `apps/grobase/LICENSING.md`). - [`wiki/security/`](wiki/security/) — security model, proofs, known weaknesses, network - [`DATA-MIGRATION.md`](DATA-MIGRATION.md) — fresh-machine and migration runbook - `apps/grobase/CLAUDE.md` — grobase's own authoritative technical document (inside the submodule) +- [Prismatica docs](https://prismatica-beta.vercel.app/docs/) — the landing's + documentation site diff --git a/docs/architecture/grobase-baas-board.html b/docs/architecture/grobase-baas-board.html new file mode 100644 index 00000000..ed4a2369 --- /dev/null +++ b/docs/architecture/grobase-baas-board.html @@ -0,0 +1,280 @@ + + + + + + Grobase BaaS Architecture + + + + +
+

Grobase BaaS runtime architecture

+

Multi-plane backend platform: edge → auth → application → control → data → realtime → storage

+ +
+
+
+flowchart TB + U[Users / Browser / SDK / CLI] + + subgraph EDGE["Public edge"] + WAF["waf\nnginx + TLS\n:80 / :443"] + KONG["kong\nAPI gateway\n:8000"] + end + + subgraph AUTH["Identity layer"] + GOTRUE["gotrue\nauth + JWT issuer\n:9999"] + MAILPIT["mailpit\ndev SMTP\n:8025"] + end + + subgraph API["API + app layers"] + PGREST["postgrest\nREST + GraphQL\n:3000"] + QR["query-router\nTS app plane"] + PE["permission-engine\nPDP"] + SR["storage-router\nobject access"] + DPR["data-plane-router\nRust execution\n:4011"] + RT["realtime\nWebSocket / event bus"] + end + + subgraph CTRL["Control plane"] + TC["tenant-control\nGo control plane\n:3022"] + AR["adapter-registry"] + ORCH["orchestrator\n:3026"] + WD["webhook-dispatcher\n:3025"] + SCHED["function-scheduler\n:3027"] + FN["functions-runtime\n:3060"] + end + + subgraph DATA["Databases + storage"] + PG[("postgres\n:5432")] + REDIS[("redis\n:6379")] + MONGO[("mongo\n:27017")] + MINIO[("minio\n:9000 / :9001")] + TRINO["trino\n:8080"] + end + + subgraph OBS["Observability"] + PROM["prometheus\n:9090"] + GRAF["grafana\n:3000"] + end + + U --> WAF --> KONG + + KONG -->|GET/POST /auth/v1| GOTRUE + KONG -->|GET/POST/PUT/PATCH/DELETE /rest/v1| PGREST + KONG -->|POST /graphql/v1| PGREST + KONG -->|GET/POST/PUT/PATCH/DELETE /query| QR + KONG -->|GET/POST/PUT/PATCH/DELETE /data| DPR + KONG -->|WS /realtime/v1/ws| RT + KONG -->|GET/POST/PUT/PATCH/DELETE /storage/v1| SR + KONG -->|GET/POST /admin/v1| TC + KONG -->|GET/POST /admin/v1| AR + KONG -->|GET/POST /admin/v1| WD + KONG -->|GET/POST /admin/v1| SCHED + KONG -->|GET/POST /sql| TRINO + + QR --> PE + QR --> DPR + DPR --> TC + DPR --> PG + DPR --> REDIS + DPR --> MONGO + DPR --> MINIO + + PGREST --> PG + GOTRUE --> PG + GOTRUE --> MAILPIT + SR --> MINIO + ORCH --> PG + ORCH --> RT + WD --> FN + FN --> REDIS + RT --> PG + RT --> MONGO + PROM --> GRAF +
+
+ +
+

Service map

+ +
waf
+
kong
+
gotrue
+
postgrest
+
query-router
+
data-plane-router
+
tenant-control
+
orchestrator
+
realtime
+
postgres
+
mongo
+
redis
+
minio
+
trino
+ +
+
/auth/v1
GoTrue login / session / OAuth
+
/rest/v1
PostgREST CRUD access to Postgres
+
/graphql/v1
GraphQL API facade
+
/query
Request orchestration / filtering / app logic
+
/data
Rust execution layer
+
/realtime/v1/ws
Realtime event stream
+
/storage/v1
Signed object uploads and retrieval
+
/admin/v1
Control-plane and management surface
+
/sql
Federated SQL via Trino
+
+ +
+
edge
+
app/API
+
control
+
data
+
+
+
+
+ + diff --git a/docs/architecture/grobase-runtime-architecture.md b/docs/architecture/grobase-runtime-architecture.md new file mode 100644 index 00000000..7dce6a0e --- /dev/null +++ b/docs/architecture/grobase-runtime-architecture.md @@ -0,0 +1,243 @@ +# Grobase runtime architecture + +This is the actual runtime topology of the Grobase BaaS used by this monorepo, based on the real deployment configuration in: + +- [apps/grobase/wiki/architecture_relation.md](../../apps/grobase/wiki/architecture_relation.md) +- [apps/grobase/deploy/helm/mini-baas/values.yaml](../../apps/grobase/deploy/helm/mini-baas/values.yaml) +- [apps/grobase/infra/docker/services/kong/conf/kong.yml](../../apps/grobase/infra/docker/services/kong/conf/kong.yml) + +It represents the real service boundaries, ports, routes, and call patterns of the platform as it is actually assembled in Docker and Kong. + +## 1) System view + +Grobase is not a single server. It is a layered platform built around a public edge, auth, application plane, control plane, data plane, realtime, and engine/storage tier. + +```mermaid +flowchart TB + Browser[Browser / SPA / SDK / CLI] + WAF[waf\nnginx + TLS\n80 / 443] + KONG[kong\nDB-less API gateway\n8000] + + GTR[gotrue\nAuth service\n9999] + PGREST[postgrest\nREST + GraphQL bridge\n3000] + MONGOAPI[mongo-api\nMongo REST facade] + QR[query-router\nTS app plane] + PE[permission-engine\nPDP] + SR[storage-router\nobject storage access] + DPR[data-plane-router\nRust execution layer\n4011] + RT[realtime\nWS/SSE event bus] + + TC[tenant-control\nGo control plane\n3022] + AR[adapter-registry] + ORCH[orchestrator\n3026] + WD[webhook-dispatcher\n3025] + FSCH[function-scheduler\n3027] + FN[functions-runtime\n3060] + + PG[(postgres\n5432)] + REDIS[(redis\n6379)] + MG[(mongo\n27017)] + MINIO[(minio\n9000 / 9001)] + TRINO[trino\n8080] + GRAF[grafana\n3000] + PROM[prometheus\n9090] + MP[mailpit\n8025] + + Browser --> WAF --> KONG + + KONG -->|GET/POST /auth/v1| GTR + KONG -->|GET/POST/PUT/PATCH/DELETE /rest/v1| PGREST + KONG -->|GET/POST/PUT/PATCH/DELETE /graphql/v1| PGREST + KONG -->|GET/POST/PUT/PATCH/DELETE /query| QR + KONG -->|GET/POST/PUT/PATCH/DELETE /data| DPR + KONG -->|WS /realtime/v1/ws| RT + KONG -->|GET/POST/PUT/PATCH/DELETE /storage/v1| SR + KONG -->|GET/POST /admin/v1| TC + KONG -->|GET/POST /admin/v1| AR + KONG -->|GET/POST /admin/v1| WD + KONG -->|GET/POST /admin/v1| FSCH + KONG -->|GET /sql| TRINO + + QR --> PE + QR --> DPR + DPR --> TC + DPR --> PG + DPR --> REDIS + DPR --> MG + DPR --> MINIO + + PGREST --> PG + GTR --> PG + GTR --> MP + SR --> MINIO + ORCH --> PG + ORCH --> RT + WD --> FN + FN --> REDIS + RT --> PG + RT --> MG + + PROM --> GRAF +``` + +## 2) Service inventory and network ports + +These are the principal services exposed by the Grobase stack and the ports they use inside the Docker network as defined in the actual Helm manifest. + +| Service | Type | Internal port(s) | Role | +|---|---|---:|---| +| waf | edge proxy | 80, 443 | TLS and public edge front door | +| kong | gateway | 8000 | API routing, auth, rate limiting, CORS, service policies | +| gotrue | auth | 9999 | identity, JWT issuance, OAuth, email OTP | +| postgrest | direct REST / GraphQL | 3000 | exposes Postgres as REST and RPC | +| query-router | TS app plane | 4001 | request orchestration and API adaptation | +| permission-engine | policy engine | 3050 | authorization / policy decision point | +| storage-router | storage API | 3040 | forms signed URLs and storage calls | +| data-plane-router | Rust execution layer | 4011 | executes engine-agnostic queries | +| realtime | realtime bus | internal event service | websockets / SSE / event propagation | +| tenant-control | Go control plane | 3022 | tenant provisioning, key resolution, identity checks | +| adapter-registry | control plane | 3021 | mount registry, engine adapter catalog | +| orchestrator | workflow control | 3026 | background tasks, outbox relay, email / GDPR flows | +| webhook-dispatcher | integration layer | 3025 | webhooks and function secret flow | +| function-scheduler | scheduler | 3027 | scheduled jobs | +| functions-runtime | functions | 3060 | serverless function execution | +| postgres | primary data store | 5432 | primary datastore | +| redis | cache / queue | 6379 | cache, rate limit, task coordination | +| mongo | NoSQL engine | 27017 | secondary datastore | +| minio | object storage | 9000, 9001 | S3-compatible object storage | +| trino | query federation | 8080 | lakehouse / SQL federation | +| grafana | observability | 3000 | dashboards | +| prometheus | observability | 9090 | metrics | +| mailpit | dev SMTP | 8025 | local email capture | + +## 3) HTTP interaction model + +The real edge behavior is described in Kong. The gateway enforces `key-auth`, `jwt`, `cors`, `rate-limiting`, and identity headers before forwarding requests. + +### Public routes + +| Path | Service | Method pattern | Notes | +|---|---|---|---| +| `/auth/v1` | gotrue | `GET`, `POST`, `PUT`, `PATCH`, `DELETE` | authentication + session handling | +| `/auth/v1/callback` | gotrue | `GET` | OAuth callback | +| `/rest/v1/*` | postgrest | `GET`, `POST`, `PUT`, `PATCH`, `DELETE` | direct database access via PostgREST | +| `/graphql/v1` | postgrest | `POST` | GraphQL endpoint | +| `/query/*` | query-router | all standard API verbs | app-plane query orchestration | +| `/data/*` | data-plane-router | all standard API verbs | execution plane | +| `/realtime/v1/ws` | realtime | WebSocket | event stream | +| `/storage/v1/*` | storage-router + minio | `GET`, `POST`, `PUT`, `DELETE` | signed object access | +| `/sql` | trino | `GET`, `POST` | federated SQL access | +| `/functions/*` | functions-runtime | all standard API verbs | serverless function dispatch | +| `/admin/v1/*` | tenant-control / webhook-dispatcher / scheduler | `GET`, `POST`, `PUT`, `PATCH`, `DELETE` | operator/admin surface | + +### Auth and identity sequence + +1. Client calls the gateway with an API key or bearer JWT. +2. Kong validates key-auth / JWT and injects identity headers. +3. The app plane uses that identity to decide what the request is allowed to do. +4. The Go control plane resolves keys and identity to tenant ownership and access scope. +5. The Rust data plane enforces per-request scoping when executing the actual database query. + +## 4) The real call flow + +### A. Browser-to-database request + +```mermaid +sequenceDiagram + autonumber + participant U as User / Browser / SDK + participant W as waf + participant K as kong + participant G as gotrue + participant QR as query-router + participant PE as permission-engine + participant TC as tenant-control + participant DPR as data-plane-router + participant PG as postgres + + U->>W: HTTPS request + W->>K: forwarded request + K->>G: auth flow if /auth/v1 + K->>QR: /query or /rest or /graphql + K->>DPR: /data request + + QR->>PE: policy check + QR->>TC: resolve key / tenant context + QR->>DPR: execute query + + DPR->>TC: key -> identity mapping + DPR->>PG: query with tenant-scoped access + PG-->>DPR: result + DPR-->>QR: response + QR-->>K: result + K-->>W: HTTP response + W-->>U: final response +``` + +### B. Realtime event flow + +```mermaid +sequenceDiagram + autonumber + participant C as Client app + participant K as kong + participant RT as realtime + participant PG as postgres + participant MG as mongo + + C->>K: websocket connect /realtime/v1/ws + K->>RT: WS upgrade + RT->>PG: CDC / change subscriptions + RT->>MG: Mongo change stream + PG-->>RT: row update event + MG-->>RT: document update event + RT-->>C: realtime payload +``` + +### C. Storage flow + +```mermaid +sequenceDiagram + autonumber + participant U as User / front-end + participant K as kong + participant SR as storage-router + participant MINIO as minio + + U->>K: PUT /storage/v1/... or GET /storage/v1/... + K->>SR: storage request + SR->>MINIO: object operation / signed URL + MINIO-->>SR: object metadata / bytes + SR-->>K: HTTP response + K-->>U: object result +``` + +## 5) Hexagonal architecture by plane + +This is the proper hexagonal model of the BaaS: + +- Edge hexagon: `waf` + `kong` handle transport, TLS, auth, routing, and enforcement. +- Identity hexagon: `gotrue` issues identity and JWTs. +- Database hexagon: `postgrest`, `mongo-api`, and the engine adapters expose storage through well-defined interfaces. +- Application hexagon: `query-router`, `permission-engine`, `storage-router`, `schema-service`, `analytics-service`, `log-service` orchestrate business behavior. +- Control hexagon: `tenant-control`, `adapter-registry`, `orchestrator`, `webhook-dispatcher`, `function-scheduler` manage provisioning and runtime control. +- Execution hexagon: `data-plane-router` and `realtime` implement the execution and event core. +- Persistence hexagon: `postgres`, `mongo`, `redis`, `minio`, `trino` are the concrete adapters and storage backends. + +This is why the platform is described as a multi-plane BaaS: the same product is assembled from different runtime roles and different technology stacks, but all of them are connected through a single public API contract. + +## 6) Practical architecture statement + +The real platform is: + +- a public API gateway (`waf` + `kong`), +- an auth service (`gotrue`), +- a request orchestration layer (`query-router`, `permission-engine`, `storage-router`), +- a Go control plane (`tenant-control`, `adapter-registry`, `orchestrator`, `webhook-dispatcher`, `function-scheduler`), +- a Rust execution layer (`data-plane-router`), +- a realtime event layer (`realtime`), +- and pluggable storage/DB engines (`postgres`, `mongo`, `redis`, `minio`, `trino`, etc.). + +The central idea is that clients do not talk to the database directly. They talk to the gateway, then to the orchestration and execution layers, which resolve tenant and identity context before hitting the storage engine. + +That is the core architecture of the Grobase BaaS product used in this monorepo. diff --git a/wiki/README.md b/wiki/README.md index 9a94c2ec..e28db799 100644 --- a/wiki/README.md +++ b/wiki/README.md @@ -85,9 +85,13 @@ zero per-project server code. | Page | What it answers | |---|---| | [01-subject](project/01-subject.md) | What ft_transcendence asks for | -| [02-modules](project/02-modules.md) | Module ledger: points, flags, evidence | +| [02-modules](project/02-modules.md) | Module ledger, superseded by the README Modules table | | [03-team](project/03-team.md) | Roles and contributions | +| [contributions-method](project/contributions-method.md) | How the README's commit counts are made, and the identity joins behind them | | [04-glossary](project/04-glossary.md) | Project vocabulary | +| [ai-llm-interface](project/ai-llm-interface.md) | The LLM interface: one question through twelve hops, and the tests behind each claim | +| [ai-audit-2026-10-28](project/ai-audit-2026-10-28.md) | Superseded 2026-11-04: the AI modules against the old pin | +| [ai-stall-diagnosis-2026-10-29](project/ai-stall-diagnosis-2026-10-29.md) | Why the assistant could stop after a few questions (diagnosis only) | --- diff --git a/wiki/STATUS.md b/wiki/STATUS.md index a46715e7..ea648fc9 100644 --- a/wiki/STATUS.md +++ b/wiki/STATUS.md @@ -40,9 +40,13 @@ Update this table whenever a page's status block changes. It is the working list | product/03-opposite-osiris | Partial | **Privacy Policy / ToS existence unverified** | | product/04-repos | Partial | What is actually inside the delivered perimeter | | project/01-subject | Unverified by nature | Source is private; re-check on each subject version | -| project/02-modules | Partial | Four open checks worth ~6 points | +| project/02-modules | Superseded | By the README Modules table, 2026-11-05 | | project/03-team | **Template** | Facts not held anywhere; fill with the team | | project/04-glossary | Verified | — | +| project/contributions-method | Verified | Counts age; recount on each pin | +| project/ai-llm-interface | Partial | nginx buffering/timeouts untested; #275 not in groot's pin yet | +| project/ai-audit-2026-10-28 | Superseded | Pin facts replaced by the README Modules table, 2026-11-04 | +| project/ai-stall-diagnosis-2026-10-29 | Diagnosis only | Hypotheses not tested live | --- diff --git a/wiki/project/02-modules.md b/wiki/project/02-modules.md index 55517aa0..9661d14d 100644 --- a/wiki/project/02-modules.md +++ b/wiki/project/02-modules.md @@ -1,10 +1,12 @@ # Module ledger -> **Status:** Partial · 2026-09-15 +> **Superseded · 2026-11-05.** This whole page is history: its rows, letters, +> legend and point totals. The current grades, one rule and the points table are +> in the [README Modules table](../../README.md#modules). +> +> **Status:** Superseded · 2026-11-05 (written as Partial · 2026-09-15) > **Evidence:** flag states read from `deploy/fly/boot.sh`; every gate named here exists in the > gate index -> **To check:** four items in the "open" table below are worth up to 6 points between them and -> none has been checked. Do those before building any screen. > **Sources:** `deploy/fly/boot.sh` · `apps/grobase/CLAUDE.md` §flag-gating · [01-subject](01-subject.md) Only catalogue modules count. **A module that cannot be demonstrated scores zero**, so each row @@ -32,7 +34,8 @@ the screen does not. | OAuth 2.0 (User mgmt, minor) | 1 | `SSO_ENABLED`, migration `053` | `m110`, `m163` | **B** | | File upload (Web, minor) | 1 | storage plane, `STORAGE_BUCKET_SCOPE_ENABLED` | `m159` | **C** | -Nominal total without free choice: **18**. Eight of those points are **A**. +Nominal total without free choice: **18**. Seven of those points are **A** (rows +1–4). **The key distinction:** organisation-system's backend is already running in production, so it needs only a screen. The **B** rows need a flag, a migration and a gate run — but no new code. @@ -65,9 +68,11 @@ in the project but hard to demonstrate live — better as an oral argument than | Does `VaultProvider` talk to HashiCorp Vault? (credential layer, migration `060`) | 2 pts and the only cybersecurity module | | What are `src/apps/ai/` and `src/apps/analytics/`? | Up to 4 pts from categories written off | -The AI category was dismissed because gaming modules need a game and we have none — but the -*LLM interface* major needs no game, and neither does the analytics dashboard major. Half a -morning of reading; six points in play. +The AI category was first dismissed because gaming modules need a game and we +have none. That no longer holds: the README now claims four AI modules (LLM +interface, RAG, moderation, voice; 6 points, all B), and the analytics dashboard +major is still unchecked. The LLM interface is explained in +[ai-llm-interface](ai-llm-interface.md). --- diff --git a/wiki/project/ai-audit-2026-10-28.md b/wiki/project/ai-audit-2026-10-28.md index 46c50787..df9a89d9 100644 --- a/wiki/project/ai-audit-2026-10-28.md +++ b/wiki/project/ai-audit-2026-10-28.md @@ -1,6 +1,11 @@ # AI audit — 2026-10-28 -> **Status:** Partial · 2026-10-28 +> **Superseded · 2026-11-04.** The pin facts below are history. groot `develop` now pins osionos +> `1c1d5d84`, which ships `services/osionos-ai` (745 files) and #270. The "0 of 10 AI points" +> line no longer holds. The current grades are in the [README Modules table](../../README.md#modules), +> and the LLM interface is explained in [ai-llm-interface](ai-llm-interface.md). +> +> **Status:** Superseded · 2026-11-04 (written as Partial · 2026-10-28) > **Evidence:** read-only `git grep` / `git ls-tree` against named refs (commands at the bottom); > no test suite and no AI container was run for this page > **To check:** every line marked ⟨unverified⟩; the five open questions at the end diff --git a/wiki/project/ai-llm-interface.md b/wiki/project/ai-llm-interface.md new file mode 100644 index 00000000..b4fb9ff1 --- /dev/null +++ b/wiki/project/ai-llm-interface.md @@ -0,0 +1,143 @@ +# The LLM interface, explained + +> **Status:** Partial · 2026-11-04 +> **Evidence:** read-only reads of osionos at groot's `develop` pin `1c1d5d84` (the AI code is +> identical to osionos `develop` `d374c21f`, per `git diff --stat` on `services/`, `docs/ai` and +> `src/features/{ai,voice,recs}`). Each cited `file:line` was opened at that pin. #275 was read +> at its merge commit `1e757c3a`. No container was run and no question was asked for this page +> **To check:** the nginx buffering and timeouts (no test covers them); the provider-error +> patterns against live providers (`upstreamError.ts:14-19`); the groot pin bump that brings in +> #275 +> **Sources:** groot #155 · osionos #270, #275 · osionos `docs/adr/0007-upstream-error-codes.md`, +> `0008-quota-aware-failover.md` · [ai-stall-diagnosis-2026-10-29](ai-stall-diagnosis-2026-10-29.md) +> · [runbooks/osionos-ai](../runbooks/osionos-ai.md) + +The README claims the **Complete LLM system interface** major (row 15) for text, streaming, +error handling and rate limiting. This page follows one question from the keyboard to the model +and back, so a reviewer can open each step and check that claim. Image generation exists, but it +is off by default and is not part of the claim. + +Paths are in osionos (`apps/osionos/app`) unless prefixed `G` (groot). Paths under +`services/osionos-ai/` drop that prefix after the first hop that enters the service (hop 5). + +--- + +## One question, start to finish + +```mermaid +sequenceDiagram + autonumber + actor U as User + participant C as Chat UI
ChatComposer, useChatStore + participant P as Inference port
aiInferencePort, runStream + participant N as nginx :4000
/api/ai/ + participant S as osionos-ai :4300
routes/runs.ts + participant R as Runner
runner.ts + Redis + participant L as Provider
gateway/resolve.ts + U->>C: Enter + C->>P: runTurn(prompt) + P->>N: POST /api/ai/runs (runId) + N->>S: proxy, buffering off + S->>R: start the run + R->>R: checkRunRateLimit (20/min per user) + alt over the limit + R-->>S: refused 429 rate_limited + S-->>P: 429 + Retry-After + P-->>C: countdown, then retry offered + else allowed + S-->>P: 200 text/event-stream + R->>L: stream call (failover chain) + loop each token + L-->>R: part + R-->>S: frame + S-->>P: id / event / data + P-->>C: append text + end + Note over S,P: ": ping" every 15 s keeps the line open + opt connection drops + P->>N: GET events, Last-Event-ID + N->>S: resume from the last id + end + S-->>P: done, or error with a stable code + P-->>C: final message or error notice + end +``` + +## The twelve hops + +| # | Hop | file:line | +|---|---|---| +| 1 | Send (Enter or the button) | `src/widgets/chat-shell/ui/ChatComposer.tsx:204`, `:305`; slash expansion `src/widgets/chat-shell/model/useSlashPalette.ts:159-169`; `ChatShell.tsx:169-179` | +| 2 | Chat store `send` and `runTurn` | `src/features/chat/model/useChatStore.ts:373`, `:204`, `:409`; error patch `:222-232`; stop `:434` | +| 3 | Inference port | `src/features/ai/model/aiInferencePort.ts:111-157` (runId `:113`, abort → cancel `:121`) | +| 3b | Stream client and resume | `src/features/ai/model/runStream.ts:77-135`: 3 resumes after 500, 1000 and 2000 ms (`:34`, `:124`); dedup + `Last-Event-ID` (`:89-95`); 410 → `run_expired` (`:115`); a 4xx is never resumed (`:109`) | +| 3c | Transport | `src/features/ai/api/aiClient.ts:118` (`/api/ai`), `:188-207`, `:330` start, `:350` resume, `:358` cancel | +| 4 | nginx | `G infrastructure/tls/nginx.conf:178-200`: upstream `osionos-ai:4300` `:179`, `proxy_buffering off` `:195`, `proxy_read_timeout 300s` `:197`. Public limit: `G infrastructure/tls/public.d/ai-public.conf:4` + `public.d/http-limits.conf:19` (2 r/s, burst 10) | +| 5 | Route table | `services/osionos-ai/src/routes/table.ts:148` (POST /runs), `:151` (events), `:157` (cancel), `:141` (/images) | +| 6 | `handleRuns` | `routes/runs.ts:178-293`: refusal + `Retry-After` `:276-280`; SSE opens `:283`; ping every 15 s `:118`; resume endpoint `:295-324` | +| 7 | Runner and per-user limit | `runs/runner.ts:205`; `checkRunRateLimit` `:260-264`; `orderByRest` `:388`; `reserveTarget` `:397` | +| 8 | Failover loop | `runs/upstream.ts:241`, `:248` (`maxRetries` 0 when a chain exists), `:274-289` move on, catch `:362-390`; `chain.ts:26`, `cooldown.ts:99,172`, `moveOn.ts:32-66`, `attempt.ts:49` (3 attempts) | +| 9 | Stream pump | `runs/streamPump.ts:232-252`; timeouts 30 s idle and 120 s total (`:25`, `:27`) | +| 10 | Provider | `gateway/resolve.ts:124-140` (anthropic, google, openai-compatible), `:176`; `gateway/platform.ts:28-32`; `connections/providers.ts:13,16` | +| 11 | SSE framing | `http.ts:108-129` (`x-accel-buffering: no`, id/event/data, `: ping`) | +| 12 | Frame dispatch → UI error | `src/features/chat/model/runEventDispatch.ts:204-214`; `ChatThread.tsx:278-285`; `src/features/chat/model/runErrorNotice.ts:42-59`; `src/features/chat/model/runErrors.ts:79-124` | + +**Why the hops are shaped this way.** The browser never talks to a provider (hops 4–10 run on +the server), so a provider key never reaches the client. The answer streams over SSE instead +of one long response, and nginx must not buffer it (hop 4, hop 11). A dropped connection resumes +from the last event id instead of starting the question again (hop 3b). And every failure ends +as a stable code the UI can name (hop 12). + +--- + +## Subject bullet → code → tests → CI + +The CI jobs are in osionos `.github/workflows/ci.yml`. "AI service" is the job +`AI service (tsc + suite)` (`:314`), which runs `services/osionos-ai/scripts/ci.sh`. "Canvas" is +the `Canvas unit tests` step of `quality (root)` (`:285`). + +| Subject bullet | Main files | Tests that prove it | CI job | What is not proven | +|---|---|---|---|---| +| Generate text | hops 1–12 | `tests/canvas/ai-chat-store.test.ts`, `ai-run-dispatch.test.ts`; `services/osionos-ai/tests/runs.test.ts`, `complete.test.ts` | Canvas; AI service | — | +| Streaming | `runStream.ts`, `aiInferencePort.ts`, `routes/runs.ts:283-324`, `http.ts:108-129` | `tests/canvas/ai-run-stream.test.ts`; `runs.test.ts`; `failover.test.ts` | Canvas; AI service | nginx buffering and timeouts; the ping and the idle/total timeouts on the happy path | +| Error handling | `runs/upstreamError.ts`, `upstream.ts:362-390`, `failover.ts`, `routes/complete.ts`; client `runErrors.ts:79-124` | `tests/upstreamError.test.ts`, `upstreamErrorRuns.test.ts`, `upstreamErrorLeak.test.ts`; `tests/canvas/ai-run-dispatch.test.ts` | AI service; Canvas | the UI shows one sentence for all the new codes (`runErrors.ts:72`, `:87-90`); the patterns were not checked against live providers | +| Rate limiting | `src/rateLimit.ts:62-84` (fixed 60 s window, Redis `INCR` + `EXPIRE NX`), `OSIONOS_AI_RUNS_PER_MINUTE` default 20 (`config.ts:48`), used at `runner.ts:260`; nginx `public_ai`; UI countdown `runErrorNotice.ts` | `tests/runRateLimit.test.ts`, `approvalRateLimit.test.ts`; `tests/canvas/ai-run-stream.test.ts` | AI service; Canvas | no direct unit test of `rateLimit.ts`; a fixed window lets up to twice the limit through at a boundary (its own caveat, `rateLimit.ts:6-9`) | +| Generate images (**not claimed**) | `routes/images.ts:141-282`, `images/engine.ts`, `promptGate.ts`, `outputCheck.ts`; client `imageRun.ts`, `AiImageJobView.tsx` | `tests/images.test.ts`, `imagesOff`, `imagesOutputCheck`, `imagesPromptGate`; `tests/canvas/ai-image-run.test.ts` | AI service; Canvas | off by default (404 unless `imagesEnabled`, `images.ts:144`); CI uses a mock engine only | + +**No groot workflow builds `--profile ai`.** The tests above run in osionos CI against the +service alone. Nothing in groot CI starts the composed stack with the AI profile. + +--- + +## What #270 and #275 added + +**osionos #270: stable codes for provider failures** (merged 2026-11-03, on develop as +`553f2afc`, **in** groot's pin). Before it, every provider failure reached the browser as +`upstream_error`, so a 429, a daily cap and an oversized prompt looked the same. + +- `runs/upstreamError.ts` maps a failure to `rate_limited`, `quota_exhausted`, + `provider_unavailable`, `context_too_long` or `upstream_error`. The error frame carries the + code, and `ai_runs.error` records `code[:status[:type]]`. +- Provider bodies and prompts are kept out of the service logs (`upstreamErrorLeak.test.ts`). +- The client reads the new codes (`runErrors.ts:88-90`), but still shows them as one sentence. +- Decision record: `docs/adr/0007-upstream-error-codes.md`. + +**osionos #275: failover that knows the free quota is shared** (merged 2026-11-04 by LESdylan, +on develop as `1e757c3a`, **not yet in groot's pin `1c1d5d84`**). Every public user spends one +OpenRouter key on `:free` routes, and the SDK retried each step twice, so one question could +spend 15 calls. + +- The SDK never retries (`maxRetries: 0`). The chain is the retry, and a run with one route + retries once on a 5xx or a network failure. +- `runs/quota.ts`: one Redis key rests every free route for everyone, for up to 300 s. While it + rests, a run whose routes are all free is refused with 429 and `Retry-After`. +- A run makes at most 7 provider calls (14 in research mode). +- Tests: `tests/quota.test.ts`, `tests/quotaFailover.test.ts`. Decision record: + `docs/adr/0008-quota-aware-failover.md`. +- It does not touch `rateLimit.ts`, the per-user limit. + +--- + +See also: [README Modules](../../README.md#modules) · [02-modules](02-modules.md) · +[ai-stall-diagnosis-2026-10-29](ai-stall-diagnosis-2026-10-29.md) · +[runbooks/osionos-ai](../runbooks/osionos-ai.md) diff --git a/wiki/project/contributions-method.md b/wiki/project/contributions-method.md new file mode 100644 index 00000000..b2a41fba --- /dev/null +++ b/wiki/project/contributions-method.md @@ -0,0 +1,87 @@ +# How the contribution counts are made + +The README's [Features List](../../README.md#features-list) and [Who committed +where](../../README.md#who-committed-where) count commits one way, described +here with the identity joins it rests on. + +## The method + +Non-merge commits reachable from the commit groot `main` pins for each +repository, read through the root [`.mailmap`](../../.mailmap), with bots and +AI-agent identities left out: + +```sh +# for the root repository, use origin/main itself as the pin +pin=$(git ls-tree origin/main apps/osionos/app | awk '{print $3}') +git -C apps/osionos/app -c mailmap.file="$PWD/.mailmap" \ + log --no-merges --format='%aN' "$pin" -- | + grep -viE 'bot|AI agent' | sort | uniq -c | sort -rn +``` + +Counted on 2026-11-05, groot `main` = `0bef3cb3`. Repositories: groot, osionos, +grobase, opposite-osiris (prismatica), hydrawn, mail, calendar, QA, monkey-bot. + +Caveat: a commit count is not effort. One large commit and one typo fix weigh +the same; pairing, review and work committed by someone else do not show; a +commit that touches several features counts once for each. Work that is not on a +pinned commit (an open branch, an unmerged pull request) is not counted. + +`scripts/contributors.sh` counts all refs (`--all`), not the pins. Until a clone +drops the refs from before the 2026-11-04 history rewrite, `--all` counts each +rewritten commit twice. + +## Left out of the counts + +At the pins, on 2026-11-05: + +- `Space Bunny` (also `bunny` and `builder`): an AI-agent identity. Its + addresses are local (for example `bunny@local`, `builder@bunny`, + `bunny@drawnosaurus.local`) and hydrawn `BUNNY.md` describes a team of agents. + Who ran it is not identified. 68 hydrawn commits. +- `opencode`: an AI-agent identity committing under its own name. Who ran it is + not identified. 1 root and 5 osionos commits. +- Bots: `dependabot[bot]` 8 root; `coderabbitai[bot]` 3 root and 4 osionos; + `Grok CI Bot` 2 root; `copilot-swe-agent[bot]` 9 opposite-osiris. + +## How the identities are joined + +The evidence for each join is in [`.mailmap`](../../.mailmap). + +- Each member committed under several addresses, including GitHub's `noreply` + ones; `.mailmap` joins them. +- `settes ` is **`rstancu`** (Roxana Stancu): the address + belongs to GitHub account [@esettes](https://github.com/esettes), listed for + `rstancu` on the original roster + ([`archive/wiki-2026-09/CONTRIBUTORS.md`](../../archive/wiki-2026-09/CONTRIBUTORS.md)). + The osionos commits (2026-04-10 → 2026-05-02) touch mostly the Playwright + browser tests (`tests/browser`), their CI job and committed build assets. + `rstancu` is a former member (README, Team Information). +- `La Cebolla ` is **`shashemi`**: opposite-osiris commit + `93e6b65` carries `Signed-off-by: Seyed Mostafa Hashemian`. +- `SMOSTAFAH1 ` is **`shashemi`**: it is + the GitHub login [@SMOSTAFAH1](https://github.com/SMOSTAFAH1) of the team + table, without the numeric id `.mailmap` already maps (2 osionos commits at + the pin). +- Two groups of grobase commits pair one member's name with another's address. + They are attributed by inference, not by proof: + - `LESdylan ` (3 commits on grobase's pinned commit, 5 on all + branches, 2026-09-21 → 2026-11-02) goes to `dlesieur`: the author name is + `LESdylan`, and `b4c1d33f` landed through grobase PR #89, opened by + `LESdylan`. + - `dlesieur ` (5 commits, 2026-03-20/21) goes to `danfern3`: + they sit inside `danielfdez17`'s own Docker and Kubernetes series of the + same two days. +- `opencode` and `Space Bunny` are AI-agent identities, shown under their own + names and left out of the member counts (list above). +- `qwO-Owp` (born2root, 2026-10-08/09) is unidentified: no address, sign-off or + pull request ties it to anyone. +- `Alex Caceres` (vendor/scripts, 2026-01-26) is a former member (team answer, + 2026-11-05). +- Neither born2root nor vendor/scripts is counted. +- On 2026-11-04 the history of groot, osionos and grobase was rewritten to + remove `Co-authored-by` trailers; trees, authors, dates and subjects are + unchanged, and every commit from the first rewritten one onward has a new SHA. + 148 root commits on `main` have a one-word subject (147 `dlesieur`, 1 + `vjan-nie`, 2026-05-01 → 2026-10-09); the [`Commit + subjects`](../../.github/workflows/commit-subjects.yml) check refuses new + ones. diff --git a/wiki/runbooks/fresh-machine.md b/wiki/runbooks/fresh-machine.md index d5649e90..d04b99a4 100644 --- a/wiki/runbooks/fresh-machine.md +++ b/wiki/runbooks/fresh-machine.md @@ -10,7 +10,7 @@ otherwise. |---|---|---| | Docker Engine ≥ 25 + Compose v2 | healthchecks use `start_interval`; measured on Docker 29.8.2 | `docker-compose.yml:3`, [README prerequisites](../../README.md#prerequisites) | | RAM ≥ 8 GB | `make all` peaked at 6.25 GB used on an 8 GB VM; for a VM, `VM_RAM_MB=8192` | [measured runs](#measured-results) | -| Disk ≥ 30 GB free for Docker | images 13.5 GB + build cache 9.5 GB + volumes 1.4 GB, measured 2026-10-01 | [README prerequisites](../../README.md#prerequisites) | +| Free disk space for Docker | no minimum is measured; `fresh-bringup.sh` warns under 10 GiB free on the Docker data root, and the 2026-10-05 VM build passed with a 65 GB `/var` (91 GB disk) | `scripts/verify/fresh-bringup.sh:34,120-122`, [measured runs](#measured-results) | | Free ports | 443, 3001–3003, 3007, 4000, 4100, 4200, 4322, 4323, 7880, 7881, 8444, 8787, UDP 50000–50060, and Kong on 8000 | `docker-compose.yml`, `apps/grobase/orchestrators/compose/base/gateway.yml` | | One `sudo` password | `make all` copies the local CA into the system store once (`certs-trust-local`); `fresh-bringup.sh` asks for it first, with `sudo -v` | `utils/mk/certs.mk`, targets `certs-trust`, `certs-trust-local`, `scripts/verify/fresh-bringup.sh` (`check_sudo`) | | A GitHub SSH key | submodules use `git@github.com:` URLs | `.gitmodules` |