diff --git a/LICENSE b/LICENSE index e8c40b65..a27ebe17 100644 --- a/LICENSE +++ b/LICENSE @@ -4,12 +4,16 @@ Copyright (c) 2026 Osionos contributors. All rights reserved. -This repository and all of its contents, including but not limited to source code, +This repository's own files, including but not limited to source code, configuration files, scripts, documentation, assets, examples, schemas, database migrations, Docker files, and related materials, are the exclusive property of the copyright holders and are protected by copyright law and other applicable intellectual property laws. +This license does not cover the git submodules listed in .gitmodules (under apps/, +vendor/ and .claude/). Each submodule keeps its own license; for example, +apps/grobase is licensed under the GNU Affero General Public License v3.0. + Permission is granted only to the contributors to this project and to persons explicitly authorized by the copyright holders to access, inspect, run, and contribute to the software for the limited purpose of evaluating, developing, diff --git a/README.md b/README.md index 04677076..275d17bf 100644 --- a/README.md +++ b/README.md @@ -1,18 +1,18 @@ *This project has been created as part of the 42 curriculum by dlesieur, serjimen, danfern3, vjan-nie, shashemi.* -# Prismatica β€” ft_transcendence - -**Prismatica** is a collaborative workspace built on our own self-hostable -Backend-as-a-Service. Users sign up on the public **prismatica** site, land in -**osionos** β€” a Notion-style block editor with real-time collaboration, chat and -video rooms β€” and can connect their Gmail and Google Calendar. Every one of -those apps is served by **grobase**, a generic backend that the team wrote in -Go, Rust and TypeScript, with no per-application server code. - -> Naming: *Prismatica* is the project; *osionos* is the workspace app; -> *prismatica* (`apps/opposite-osiris`, repository -> [`Univers42/prismatica`](https://github.com/Univers42/prismatica)) is the -> marketing landing. Infrastructure identifiers keep the old codename +# Osionos + +**A workspace where your data stays yours.** Pages, databases, chat and an AI +assistant, running on grobase, the Backend-as-a-Service we built ourselves. + +| 16 points | 5 members | Our own backend | 1 command | +|---|---|---|---| +| 10 modules (6 Majors, 4 Minors), 14 required | Over 8,400 commits | Go, Rust, NestJS and Kong | `make all` runs everything | + +> Naming: *Osionos* is the project and its editor. *prismatica* is the public +> landing (`apps/opposite-osiris`, repository +> [`Univers42/prismatica`](https://github.com/Univers42/prismatica)), the +> "Public site" below. Infrastructure identifiers keep the old codename > `track-binocle` (the compose project, image names, `TRACK_BINOCLE_*` > variables, `*.track-binocle.test` hosts, the local CA file): renaming them > would break scripts and issued certificates. @@ -21,360 +21,185 @@ Go, Rust and TypeScript, with no per-application server code. ## Table of contents -1. [Description](#description) -2. [Instructions](#instructions) -3. [Team Information](#team-information) -4. [Project Management](#project-management) -5. [Technical Stack](#technical-stack) -6. [Database Schema](#database-schema) -7. [Features List](#features-list) -8. [Modules](#modules) -9. [Individual Contributions](#individual-contributions) -10. [Resources](#resources) -11. [Known limitations](#known-limitations) -12. [Further documentation](#further-documentation) +1. [Description](#-description) +2. [Instructions](#-instructions) +3. [Team members and roles](#-team-members-and-roles) +4. [Project management](#-project-management) +5. [Technologies](#-technologies) +6. [Database schema](#-database-schema) +7. [Features and who built them](#-features-and-who-built-them) +8. [Modules and points](#-modules-and-points) +9. [Individual contributions](#-individual-contributions) +10. [Resources](#-resources) +11. [Known limitations](#️-known-limitations) +12. [License](#license) +13. [Further documentation](#further-documentation) --- -## Description +## πŸ“– Description -### Goal +### osionos -Almost every web project rebuilds the same backend before it can start on the product: -authentication, user management, CRUD, access control, file upload, email, logs. Our goal was to -turn that repeated work into infrastructure β€” **one backend, any frontend, zero per-project server -code** β€” and to prove it by building a real product on it. +The editor: pages built from blocks, database blocks, chat, live collaboration +and an AI assistant. -The project therefore has two halves: +Built with *React 19, Vite, TypeScript*. -- **grobase** β€” a Backend-as-a-Service (about 290,000 lines of Go, Rust and - TypeScript at its pin, roughly a third of the code the team wrote; lines - include tests). An application is declared in a JSON *contract* (its database, - isolation strategy, roles, permissions and API keys); a generic provisioner - creates everything from that file. Routes carry no domain logic: a request - names the table it targets, and access rules are stored as data, not code. -- **The products** that run on it β€” the prismatica landing and sign-in site, the - osionos workspace, the hydrawn whiteboard, and mail and calendar integrations. +### grobase -### Overview +Our backend: sign-in, data, real-time and storage, with no osionos-specific +code inside. -| Product | Tech | What it is | -|---|---|---| -| **prismatica** (`apps/opposite-osiris`) | Astro | Public site: landing page, docs, sign-up / sign-in, legal pages. The entry point. | -| **osionos** | React + Vite | The block editor: pages, databases-as-blocks, real-time collaboration, chat, video rooms. **The flagship.** | -| **mail** | React + Vite | Gmail integration over Google OAuth | -| **calendar** | React + Vite | Google Calendar integration over Google OAuth | -| **hydrawn** | SvelteKit + Fastify + Rust/WASM | Collaborative whiteboard, embedded in osionos | +Built with *Go, Rust, NestJS, Kong*. -These products are supported by platform services (not applications in their own -right): the **auth-gateway** (Node, shipped by the prismatica repository: -sessions and sign-in), the **osionos-bridge** (Node: turns a site session into -an editor session), **Kong** (grobase's API gateway, behind its WAF) and -**LiveKit** (the WebRTC media server for video rooms). +### Public site -### Key features +Where users start: the landing page in three languages, sign-up and the legal +pages. -- **Contract-driven backend.** A whole application β€” database, isolation, roles, - keys, frontend configuration β€” is one JSON file. `POST /v1/tenants/me/apps` - creates a new isolated application with its own database and scoped API key - (mounted only when `APPS_SELFSERVE_ENABLED` is on). -- **Per-request owner isolation.** The Rust data plane scopes queries to their - owner on each request, not by connection-pool state; permissions are rows - (RBAC, plus flag-gated ABAC conditions), not `if` statements. -- **One API over many databases.** PostgreSQL, MySQL (MariaDB image), MongoDB, - Redis, SQLite, SQL Server and even remote HTTP APIs are queried through the - same envelope and the same API key. DynamoDB is an opt-in build, left out of - the default one - ([`wiki/platform/08-engines.md`](wiki/platform/08-engines.md)). -- **Collaborative editor.** Block-based pages, a database block with table / board / calendar - views, real-time co-editing, channels and direct messages, video rooms. -- **Single-command deployment.** `make all SKIP_SYNC=1` builds the pinned - commits in Docker, starts the backend and the frontends over local HTTPS with - a trusted CA, and health-checks them (`utils/mk/pipeline.mk:10`). +Built with *Astro, Sass*. ---- +### Why we built it -## Instructions +Most web projects rebuild the same backend before they reach the product. We +built that backend once as grobase, then built a real product on it: a +workspace whose data lives in a backend the team hosts, not in a third-party +service. -### Prerequisites +### Key features -| Requirement | Detail | Source | -|---|---|---| -| **Linux host** | Tested on Debian 13 (trixie), the system the born2root VM installs; the CA-trust step uses `sudo` + `update-ca-certificates` | `vendor/born2root/docker/Dockerfile:14`, `vendor/born2root/generate/create_custom_iso.sh:120`; `utils/mk/certs.mk` | -| **Docker Engine β‰₯ 25** + **Compose v2** | Healthchecks use `start_interval`, which needs Docker 25. Tested with Docker 29.8. | `docker-compose.yml:10` | -| **RAM β‰₯ 8 GB** | Measured: the VM peaked at ~6.5 GB of 8 GB during `make all` | `wiki/runbooks/fresh-machine.md` (6266 MiB peak, v1.0.0-rc4, 2026-10-05) | -| **Free disk space for Docker** | Images, the build cache and data volumes all live in Docker's storage; no minimum is measured in the repository. `docker system df` shows the current use | β€” | -| **bash, GNU Make, git, curl, openssl, ssh** | Make drives everything; curl runs the health check; openssl generates the local secrets | `utils/mk/app.mk:12-15`, `scripts/gen-local-env.sh:253` | -| **A GitHub SSH key** | `.gitmodules` uses `git@github.com:` URLs, so the recursive clone needs SSH access to GitHub | `.gitmodules` | -| **Google Chrome** | Current stable β€” the browser the project is evaluated on | [`wiki/project/01-subject.md`](wiki/project/01-subject.md) | -| `certutil` (optional) | Imports the local CA into the browsers' NSS stores. Absent, `make certs-trust-user` unpacks `libnss3-tools` into `~/.cache/born2root/nss` without root (Debian only: it needs `apt-get` and `dpkg-deb`, bounded at 120 s); if that fails it warns and exits 0, and the CA is imported by hand (see [Access](#access-and-the-local-ca)) | `scripts/certs-trust-user.sh` | - -**Nothing else is installed on the host** β€” no Node, npm, Go or Cargo. Everything builds and runs -in containers. - -`make all` asks for **one `sudo` password** the first time: it copies the local CA into the system -trust store. This is intended (`certs-trust-local` in `utils/mk/certs.mk`). Without -sudo, `CERT_TRUST_MODE=user` trusts the browsers only and `skip` trusts nothing (see -[Access](#access-and-the-local-ca)). - -**Ports bound on the host** (from `docker-compose.yml`, all on `127.0.0.1` -except inside a QEMU/VirtualBox NAT VM, where -`infrastructure/scripts/detect-bind-addr.sh` binds `0.0.0.0` so the host can -reach the guest): 443, 3001, 3002, 3003, 3007 (always loopback), 4000, 4100, -4200, 4322, 4323 and 7880 (always loopback: plain HTTP), 7881, 8444, 8446 -(always loopback: the IDE edge), 8787 and UDP 50000–50060. A busy port moves to -the next free one (`scripts/host-ports.sh`). grobase adds Kong on -`127.0.0.1:8000` (literal loopback in its -`orchestrators/compose/base/gateway.yml:56`; its WAF is the public entry). - -### Defense bring-up (pinned) - -This is the path rehearsed on a clean machine, and the stack the -[Modules](#modules) are graded on. It builds `main` and the submodule commits -`main` pins, and does **not** follow submodule branch tips (`SKIP_SYNC=1`, see -`sync-submodules-soft` in `utils/mk/pipeline.mk`). +- **Block editor**: nested pages, templates, comments, sharing and publishing +- **Database blocks**: table, board and calendar views, with search, filters, + sort and paging +- **Live collaboration**: several people on one page, with presence +- **Chat and people**: channels, direct messages, profiles and contacts +- **AI assistant**: streamed answers from a language model +- **Design system**: `@osionos/ui`, 20+ components and generated palettes + +### All the applications + +| Application | Built with | Role | +| --- | --- | --- | +| osionos | React, Vite | The editor, the subject of this page | +| opposite-osiris | Astro, Sass | Public site, sign-up, legal pages | +| osionos bridge | Node | Trust boundary: holds the service key, calls the backend | +| auth-gateway | Node | Sessions and OAuth, hands the user to osionos | +| grobase | Go, Rust, NestJS, Kong | The backend | +| mail, calendar | React, Vite | Opt-in Gmail and Google Calendar apps | +| hydrawn | Own engine | The whiteboard | + +--- + +## πŸš€ Instructions ```bash git clone --recursive git@github.com:Univers42/groot.git cd groot -git checkout main +git checkout "$(git describe --tags --abbrev=0 origin/main)" git submodule update --init --recursive make all SKIP_SYNC=1 ``` -Requirements, the VM + host-browser case, the verify script (`scripts/verify/fresh-bringup.sh`) and -troubleshooting: [`wiki/runbooks/fresh-machine.md`](wiki/runbooks/fresh-machine.md). - -grobase on another machine (the born2root VM, a LAN box, a VPS, a Kong URL) with -the frontends here, no sudo: put the kind of connection in `./.env.grobase-link` -(`GROBASE_TARGET=ssh://b2b`; `.env.grobase-link.example` lists the kinds), then -`make link` and `make link-frontends-up`. The latter also applies `models/*.sql` -to that grobase (`make link-models-migrate`), sets the demo account's password -there (`make link-demo-login-ensure`; `make demo-login` prints it), hands the -demo rows to the app's API key (`make link-live-data-ensure`) and trusts the -local CA in this user's browser stores (`make certs-trust-user`, no sudo). Both -end with `make link-verify`: one request per grobase service from inside every -frontend, the TLS edge, then the browser stores β€” -[`wiki/runbooks/grobase-link.md`](wiki/runbooks/grobase-link.md). To serve the -frontends to the internet through this host's Tailscale Funnel (443; add 8443 -and 10000 with `PUBLIC_PORTS=all`), `make public-env` and `make public-up`; -`make public-down` closes it: -[`wiki/runbooks/public-funnel.md`](wiki/runbooks/public-funnel.md). `make -roundtrip` then proves persistence (account, sign-in, editor hand-off, a page, -the rows in postgres) and `make link-e2e` runs the Playwright smoke through the -link. - -Measured time from a cold machine: **9–17 minutes** (8m55s on a fresh VM, 16m44s -on a laptop, 2026-09-30, [`wiki/STATUS.md`](wiki/STATUS.md); later runs in -[`wiki/runbooks/fresh-machine.md`](wiki/runbooks/fresh-machine.md)). - -No `.env` needs to be written by hand. With no vault key present, grobase generates its own -secrets into `apps/grobase/.env`, and `env-local-ensure` derives the root `./.env.local` from it, -including random `OSIONOS_*` secrets and the demo account's `DEMO_LOGIN_PASSWORD` -(`scripts/gen-local-env.sh`). An older `.env.local` without it gets one minted on the next -`make all`, which says so by name. - -`make all` runs, in order (`utils/mk/pipeline.mk`): submodule sync β†’ secrets β†’ -local TLS certificates β†’ trust the local CA β†’ grobase backend β†’ derive -`.env.local` β†’ restore data if the engines are empty β†’ set the demo account's -password β†’ SQL migrations β†’ live-data key and owner fix β†’ mount re-encryption β†’ -frontends β†’ health check β†’ URL list. On a machine that holds vault42 -credentials, `make all` deploys against the data-center VM instead and starts no -local grobase (ADR 0047, -[`wiki/runbooks/datacenter.md`](wiki/runbooks/datacenter.md)); `make all -GROBASE_DATACENTER=` forces the local path. - -### Development bring-up +Then open `https://localhost:4322` and create an account. It takes 9 to 17 +minutes on a cold machine, and `make showcase` lists what is running. -```bash -git checkout develop -make all -``` +> [!WARNING] +> **Three modules need the deployed stack.** On a plain `make all`, live +> editing (W9), the AI assistant (I3) and OAuth (U3) are off. That is 4 points, +> so demonstrate them on the deployed stack. + +### Prerequisites -Without `SKIP_SYNC=1`, `make all` first fast-forwards every submodule to the tip -of its tracked branch (`sync-submodules-soft`), so you build the newest code -rather than the pinned commits. It skips dirty submodules, and leaves at its -recorded commit any submodule whose branch tip does not contain that commit; it -never blocks the pipeline. - -The exception is any submodule that `.gitmodules` marks `update = none`, today `apps/graph_render`. Its -gitlink is the build input, so neither mode moves it: -- `git submodule update --recursive` skips it, which also keeps its SSH-only nested `SciGraphs` out; -- the sync skips it and everything under it; -- `frontends-up` runs `make submodules-pinned` (`scripts/pin-submodules.sh`), which checks it out - at the recorded commit if it is absent, and fails if it is anywhere else or has edits to tracked files. - -What the check does not do: -- It does not move a checked-out copy. After a pointer bump, run the fix it prints: - `git submodule update --checkout -- apps/graph_render`. -- It ignores nested submodules and untracked files. -- It reads `.gitmodules`, so a local `.git/config` `update` override is not seen. -- It cannot guard a raw `docker compose build`. -- It is not offline-safe: the first `make all` after the pin lands clones graph_render over HTTPS, - and fails if the network is down. A refused connection fails at once. A black-holed one waits - for the TCP connect timeout (~1.5 min per attempt, and git retries once) before it fails. - -The osionos pointer's own `branch = main` override is out of scope here (tracked with the pin audit). - -### Finding make targets - -`make help` lists every root target by section; `make help-` prints one target's summary, -prerequisites and usage lines (for example `make help-link`). Each target in `utils/mk/*.mk` -carries its summary after `##` on the target line and sits under a `##@ Section` heading; those -markers are the only source of the help output. The convention and the list of deleted targets -are in [ADR 0016](docs/adr/0016-self-documenting-pruned-root-make.md). - -### Checks +| You need | Why | +| --- | --- | +| Docker 25+ with Compose v2 | Everything builds and runs in containers | +| GNU Make, git, curl, openssl | They drive the build and generate local secrets | +| 8 GB RAM, 30 GB of disk | Measured peak about 6.5 GB; images and cache take 23 GB | +| A GitHub SSH key | The recursive clone uses `git@github.com` URLs | +| Chrome, current stable | The browser the project is evaluated on | + +Nothing else goes on the host, and no `.env` is written by hand: secrets are +generated on first run. + +### Where to open each service + +| Service | Address | +| --- | --- | +| Public site, start here | | +| osionos editor | | +| Whiteboard | | +| Bridge | | +| grobase API | | + +### More on running it + +The measured runs, the VM case, the verify script +(`scripts/verify/fresh-bringup.sh`) and troubleshooting are in +[`wiki/runbooks/fresh-machine.md`](wiki/runbooks/fresh-machine.md). `make help` +lists every root target; `make help-` explains one. + +**Local certificate authority.** Every frontend is served with a certificate +from the local CA `apps/grobase/certs/track-binocle-local-ca.pem`. `make all` +asks for one `sudo` password the first time, to copy that CA into the system +trust store (`certs-trust-local` in `utils/mk/certs.mk`), and trusts it in this +user's Chrome, Chromium and Firefox stores without sudo +(`scripts/certs-trust-user.sh`). `CERT_TRUST_MODE=user` trusts the browsers +only, `skip` trusts nothing. `make certs-trust-check` prints one line per +store; `make certs-trust-user` fixes them. A browser reads its store at launch +only, so relaunch it afterwards. While the CA is untrusted, the Whiteboard tab +in osionos stays blank: the browser refuses the embedded `:3007` frame +silently. + +**Checks.** | Command | What it proves | -|---|---| +| --- | --- | | `make healthcheck` | grobase auth, bridge, osionos, website (and HTTPβ†’HTTPS redirect), auth gateway, hydrawn board list (`utils/mk/app.mk`) | -| `make e2e` | Playwright smoke in a pinned container against the running stack: DW3 Whiteboard tab, DW8 two browsers on one board, DW9 the board inside osionos, DW10 Mail/Calendar (`tests/e2e/README.md`) | -| `make mail-up calendar-up` | Start the opt-in Mail and Calendar apps. Without them DW10 **skips** and prints the reason; it never passes silently. | -| `make e2e-clean` | Soft-delete the `e2e-*` boards the smoke leaves behind | +| `make e2e` | Playwright smoke in a pinned container against the running stack (`tests/e2e/README.md`) | +| `make mail-up calendar-up` | Start the opt-in Mail (`https://localhost:3002`) and Calendar (`https://localhost:3003`) apps. Without them the Mail/Calendar smoke **skips** and prints the reason | -### Access and the local CA +**Demo account.** `make demo-login` prints the seeded demo account's email and +password. The password is generated per machine and kept only in the +git-ignored `./.env.local`; it applies to the local stack only. -| Service | URL | -|---|---| -| prismatica site (start here) | `https://localhost:4322` | -| osionos editor | `https://localhost:3001` | -| Whiteboard (hydrawn) | `https://localhost:3007` | -| mail (opt-in) | `https://localhost:3002` | -| calendar (opt-in) | `https://localhost:3003` | -| osionos-bridge | `https://localhost:4000` | -| auth-gateway | `https://localhost:8787/api/auth` | -| grobase API (TLS edge) | `https://localhost:8444` | -| LiveKit signalling (video rooms) | `wss://localhost:4000/livekit` | - -Only the TLS ports above are entry points. Kong's own plain-HTTP listener (`127.0.0.1:8000`) and -LiveKit's `127.0.0.1:7880` are bound to the host's loopback for the backend's own tools; a browser -never gets them (`scripts/verify/https-only.sh`, run in CI, fails on any `ws://` or `http://` URL -handed outside the backend). - -`make showcase` prints the list of what is actually running. - -Every frontend is served with a certificate from the local CA -`apps/grobase/certs/track-binocle-local-ca.pem`. `make all` trusts it in two places: the system -store (one `sudo`) and every browser store this user owns, through `scripts/certs-trust-user.sh` -(no sudo). That script covers Chrome and Chromium (`~/.pki/nssdb`, snap Chromium's -`~/snap/chromium/current/.pki/nssdb`) and every Firefox profile under `~/.mozilla/firefox`, the snap -and the flatpak locations. It matches old copies by the CA's subject, not by nickname, replaces -them, and adds the current file as "Track Binocle Local Development CA". - -- `make certs-trust-check` prints one line per store (`ok|FAIL - current|stale|missing|untrusted`) and exits 1 if any store is not current. - Without `certutil` it fails. -- `make certs-trust-user` fixes the stores without sudo; it mints the CA first only if it is missing. -- `CERT_TRUST_MODE=system|user|skip` (or `TRACK_BINOCLE_CERT_TRUST`) picks what `certs-trust-local` - does: system store plus browsers (default), browsers only, or nothing. -- **Relaunch the browser afterwards.** A browser reads its store at launch only; the script never - restarts one. `vendor/born2root/setup/host/restart_browsers.sh` does it with session restore. - -If the CA is not trusted, the browser shows a warning on each port, and the **Whiteboard tab in -osionos stays blank**, because the embedded `:3007` iframe is refused silently. Two causes of a -warning while `curl --cacert` succeeds: - -- **The CA was regenerated.** `make certs` on a fresh tree mints a new CA, and a - store still holding an older copy does not trust it. `make certs-trust-check` - then reports those stores `stale`, and `make certs-trust-user` replaces the - old copy. - -- **A snap refresh.** Snap Chromium's `HOME` is the revision directory behind - `~/snap/chromium/current`, so a new revision starts without the store. - `make certs-trust-check` reports it missing; `make certs-trust-user` recreates it. - -Flatpak Firefox profiles can exist without the flatpak installed; they are imported too, harmlessly. -The script does not restart browsers, does not touch Safari or the macOS and Windows stores, and -does not trust the CA on another machine (see `wiki/runbooks/fresh-machine.md` for a browser on the -host of a VM). To import by hand: Chrome Settings β†’ Privacy and security β†’ Security β†’ Manage -certificates β†’ Authorities β†’ Import, or run `make certs-trust`. - -### Demo - -1. Open `https://localhost:4322`, create an account, and continue to osionos. Or sign in with the - seeded demo account: `make demo-login` prints its email and password. The password is - generated per machine and kept only in `./.env.local` (git-ignored); it is never committed. - It applies to the local stack only, not to production. -2. In osionos, open **View β†’ Whiteboard**, click **+ New Board**, give it a title. -3. Open the same board URL in a second window and draw: the strokes appear live in the other - window (the e2e smoke asserts within 5 s β€” `tests/e2e/specs/dw8-realtime.spec.ts`). - -Honest limitations of this demo: board editing is **loopback-only** and has **no -per-user authorization yet** β€” hydrawn is bound to `127.0.0.1:3007` for exactly -that reason (`docker-compose.yml:35-40`). We make **no end-to-end-encryption -claim**: traffic is TLS to the local proxy, nothing more. - -### Environment configuration - -- Real credentials live only in git-ignored `.env*` files; the committed template is - [`.env.example`](.env.example), which documents every key as **required**, **recommended** or - **optional**. -- Optional integrations turn themselves off cleanly when left blank: - - `LIVEKIT_API_KEY` / `LIVEKIT_API_SECRET` β€” default to a local dev value; video rooms still work. - - `GOOGLE_CLIENT_ID` / `GOOGLE_CLIENT_SECRET` in `apps/mail/.env.local` and - `apps/calendar/.env.local` β€” without them, the mail and calendar OAuth flows are disabled. - - `SONAR_TOK`, `FLY_API_TOKEN`, `DOCKER_PAT` β€” operator tooling only (a local - SonarCloud scan, the Fly deploy, a Docker Hub login); CI uses its own - repository secrets. - -### Team setup (with the vault) - -Inside the team, secrets are shared through **vault42**, our zero-knowledge -secret store, via the **42ctl** CLI. A machine that holds a vault identity in -`~/.config/42ctl/` pulls the `.env` tree during `make all` when -`apps/grobase/.env` is absent (`secrets-ensure`), or on demand with `make -vault42-pull-all APPLY=1`. With the vault passphrase also present, `make all` -deploys against the data-center VM instead (see [Defense -bring-up](#defense-bring-up-pinned)). A new identity: `scripts/vault42-ctl.sh -keys init`, then `scripts/vault42-ctl.sh keys enroll --org univers42`, then an -org admin grants the project -([`wiki/runbooks/another-device.md`](wiki/runbooks/another-device.md)). Details: -[`DATA-MIGRATION.md`](DATA-MIGRATION.md). - -The vault42 CLI is wired in beside that path: the `Secrets (vault42)` section of `make help`, and -[`vendor/scripts/vault42/README.md`](vendor/scripts/vault42/README.md). Status and retirement -checklist: [ADR 0017](docs/adr/0017-vault42-cli-beside-the-old-path.md). - -### Everyday commands - -| Command | What it does | -|---|---| -| `make all` | Full lifecycle: secrets β†’ certs β†’ backend β†’ env β†’ data β†’ migrations β†’ frontends β†’ health check β†’ URLs | -| `make healthcheck` | Probe backend, site, editor, bridge, auth gateway and whiteboard | -| `make showcase` | Print the URL list for what is actually running | -| `make pulls` | Fetch and pull the root repository and every submodule | -| `make -C apps/grobase editions` | List the available backend shapes | -| `make all GROBASE_EDITION=full` | Bring up every grobase plane except the playground demo UI, including extra engines and monitoring | -| `docker compose ps` | Service status | -| `docker compose logs -f ` | Follow one service's logs | -| `COMPOSE_PROFILES=dev docker compose down` | Stop and remove the root project's containers; named volumes stay (no `-v`). `make` sets `COMPOSE_PROFILES=dev` (`utils/mk/common.mk:45`), a plain shell does not | - -**Backend editions.** `make all` starts grobase in the **`devlean`** edition -(`GROBASE_EDITION ?= devlean` in `utils/mk/grobase.mk`; listed in grobase's -`orchestrators/makes/00-config.mk:148-154`): the core engines (PostgreSQL, -MySQL, MongoDB, Redis, MinIO), the full application / control / data planes and -realtime β€” without the heavy extra engines and without the monitoring stack. -`GROBASE_EDITION=migrate` adds every snapshot engine; `GROBASE_EDITION=full` -turns on every plane except the playground demo. Some module demonstrations need -a larger edition or a feature flag β€” see [Modules](#modules). +**Optional integrations.** Real credentials live only in git-ignored `.env*` +files; [`.env.example`](.env.example) documents every key as required, +recommended or optional. Left blank, `LIVEKIT_API_KEY` / `LIVEKIT_API_SECRET` +default to a local value, and without `GOOGLE_CLIENT_ID` / +`GOOGLE_CLIENT_SECRET` in `apps/mail/.env.local` and `apps/calendar/.env.local` +the mail and calendar OAuth flows are off. + +**Development bring-up.** On `develop`, `make all` without `SKIP_SYNC=1` first +fast-forwards every submodule to the tip of its tracked branch +(`sync-submodules-soft` in `utils/mk/pipeline.mk`), so it builds the newest +code rather than the pinned commits. --- -## Team Information +## πŸ‘₯ Team members and roles -All five members are developers. Roles evolved during the project; the main -role of each is below (team answer, 2026-11-05). The "Areas in the history" -column is what each member's commits touch ([Individual -Contributions](#individual-contributions) has the numbers). +| role \ name | dylan lesieur | vadim | daniel | sergio | mostafa | +| --- | --- | --- | --- | --- | --- | +| project manager | ❌ | βœ… | ❌ | ❌ | ❌ | +| Product Owner | βœ… | ❌ | ❌ | ❌ | ❌ | +| tech lead | ❌ | ❌ | βœ… | ❌ | ❌ | +| developer | βœ… | βœ… | βœ… | βœ… | βœ… | -| Member | Login | GitHub | Main role | Responsibilities | Areas in the history | -|---|---|---|---|---|---| -| Dylan Lesieur | `dlesieur` | [@LESdylan](https://github.com/LESdylan) | Product Owner | Product vision and priorities | grobase, osionos, mail, calendar, root infrastructure and CI | -| Vadim Jan Nieto | `vjan-nie` | [@vjan-nie](https://github.com/vjan-nie) | Project Manager | Planning, tracking and coordination | QA tooling, end-to-end tests, documentation, Docker | -| Sergio JimΓ©nez de la Cruz | `serjimen` | [@DjSurgeon](https://github.com/DjSurgeon) | Frontend tech lead | Frontend architecture and quality | the public site (opposite-osiris), osionos accessibility and editor features | -| Daniel FernΓ‘ndez | `danfern3` | [@danielfdez17](https://github.com/danielfdez17) | Backend tech lead | Backend architecture and deployment | monkey-bot (security test bot), grobase Docker and Kubernetes deployment, osionos editor fixes | -| Seyed Mostafa Hashemian | `shashemi` | [@SMOSTAFAH1](https://github.com/SMOSTAFAH1) | QA / quality tech lead | Testing and code quality; joined in the last part of the project | security fixes and SAST, the site's legal pages, TLS edge | +**One owner per role**, as the team confirmed on 2026-11-04. + +### Logins and areas + +The last column is what each member's commits touch ([Individual +contributions](#-individual-contributions) has the numbers). + +| Member | Login | GitHub | Areas in the history | +|---|---|---|---| +| Dylan Lesieur | `dlesieur` | [@LESdylan](https://github.com/LESdylan) | grobase, osionos, mail, calendar, root infrastructure and CI | +| Vadim Jan Nieto | `vjan-nie` | [@vjan-nie](https://github.com/vjan-nie) | osionos editable block logic (`src/features/block-editor`), QA tooling, end-to-end tests, documentation | +| Daniel FernΓ‘ndez | `danfern3` | [@danielfdez17](https://github.com/danielfdez17) | monkey-bot (security test bot), grobase Docker and Kubernetes deployment, osionos editor fixes | +| Sergio JimΓ©nez de la Cruz | `serjimen` | [@DjSurgeon](https://github.com/DjSurgeon) | the public site (opposite-osiris), osionos accessibility and editor features | +| Seyed Mostafa Hashemian | `shashemi` | [@SMOSTAFAH1](https://github.com/SMOSTAFAH1) | security fixes and SAST, the site's legal pages, TLS edge | **Former members.** The team changed along the way: `rstancu` (Roxana Stancu, [@esettes](https://github.com/esettes)) and Alex Caceres were temporary members @@ -384,182 +209,84 @@ defense covers the current five. --- -## Project Management +## πŸ—‚ Project management -Part of this project was an experiment in managing a team. We ran it as if it -were a demanding, high-output professional team, with a Scrum mindset in which -every member is treated as an expert and anyone may change anyone's code. The -aim was that **nobody is indispensable**. The evidence below separates what the -repositories record from what is the team's own account, and the last part -measures how far we got towards that aim. +The team talks on Discord and meets weekly. Work lands through pull requests +into `develop`, then `main`, and a change is done when its verification gate +passes. -**How the tracking evolved.** Work started in February 2026: the first commits -date from that month (`git log --reverse` across the submodules). Between then -and the autumn the team tried GitHub Projects, Jira tickets and Notion pages -(team account: none of the three left a trace in commit messages, and the -Projects board was not readable with our token). The only written trace is -osionos -[`docs/DoD.md:39`](https://github.com/Univers42/osionos/blob/develop/docs/DoD.md), -a Definition-of-Done template that asks for evidence "in your Pull Request -description or your Trello/Jira" ticket (2026-04-11). +### Agile -**How it works now: issues, labels and pull requests on GitHub.** +| Scrum | Lean production | Kanban | +| --- | --- | --- | +| The Project Manager acts as Scrum Master. | Numbered verification gates define done. | GitHub Issues and labels track the work. | -| Practice | What the repositories record (2026-11-04) | -|---|---| -| Issues | 129 on osionos, 38 on groot, 20 on prismatica, 16 on grobase (`gh issue list --state all`). Branches and commits name them: `fix/issue-278-database-probe-diagnostics` (osionos #284), `fixes #N` in 40 commit messages across groot, grobase, osionos and prismatica | -| Labels | Triage labels `priority: high`, `subject-blocker` and `account-platform` on groot and osionos, plus GitHub's defaults (`gh label list`) | -| Pull requests | 144 on groot, 182 on osionos, 78 on grobase, 50 on prismatica; merged into `develop`, and `develop` promoted to `main` by a merge commit | -| Peer review | Since 2026-09-21 (grobase, hydrawn) and up to 2026-11-04 (groot `develop`), a GitHub ruleset on each main repository requires one approving review and forbids force-pushes and branch deletion (`gh api repos//rulesets`). CodeRabbit reviews every PR since 2026-10-23 | -| Stacked branches | Feature chains merged one on top of another: groot #84–#87 (`feat/ide-*`), grobase #69–#70 (`feat/multidb-g1b…g1d`), the osionos `release/acct-w1` … `w9` series | -| Commit rhythm | At least one commit on 174 of the 232 days between 2026-03-18 and 2026-11-04, across all repositories | - -**Strict CI from the start.** CI arrived within days of each repository's first -commit: grobase `ci.yml` on 2026-03-29, osionos on 2026-04-12, groot on -2026-05-13. SonarCloud is configured in groot since 2026-05-04 and comments on -its pull requests since #9 (2026-06-01). Security scans (Semgrep, Trivy, -gitleaks, TruffleHog, ZAP, cargo-audit, govulncheck) run on every pull request, -and groot rejects one-word commit subjects -(`.github/workflows/commit-subjects.yml`). - -**Rituals.** Two-week sprints since the start, daily stand-ups and a weekly -meeting, mostly on Discord (team account, confirmed 2026-11-04; Discord leaves -no trace in the repositories). The method was Scrumban (Scrum rituals plus a -Kanban board), as described in -[`docs/dossier-project-dylan/dossier-projet.md:390-416`](https://github.com/Univers42/osionos/blob/develop/docs/dossier-project-dylan/dossier-projet.md) -in osionos. - -**Planning and decisions.** Work is planned and decided in meetings, mostly on -Discord, then recorded in issues, pull requests and the wiki. When a problem -blocked the team, it got a dedicated meeting. Test-driven development and -DevOps practices (CI on every pull request, Docker-first builds, one-command -bring-up) were the means to a frictionless workflow. - -**How far we got.** We publish these numbers so the claims above can be checked: - -- **Sprints.** The sprints were planned outside GitHub: no milestone and no - sprint label exists in any repository. Merges come in bursts: 250 of the 442 - merged pull requests on the six main repositories landed in two weeks (ISO - weeks 43–44). -- **Review.** The rulesets let a maintainer bypass them, and in a sample of the - 40 most recent merged pull requests per repository, 0–22 % carry an approval - from someone other than the author. -- **Nobody indispensable.** Shared ownership is real in breadth: every active - member committed in at least three areas, and osionos has six authors with - five or more commits. In depth we are not there yet: one member wrote 86 % of - the 8,571 human commits, and in 17 of 18 areas one author holds more than half - of the commits. Commit counts are not effort: one large commit and a typo fix - weigh the same. +### How the work flows ---- - -## Technical Stack +- **Branches**: short-lived `feat/`, `fix/` and `docs/`, one change each +- **Repositories**: a change lands in its submodule first, then the root + records it +- **CI**: GitHub Actions on every pull request, with Dependabot for updates +- **Merged pull requests**: LESdylan 260, vjan-nie 72, SMOSTAFAH1 57, + DjSurgeon 8, danielfdez17 4 -### Frontend +### What the repositories record -| Technology | Used in | Why | -|---|---|---| -| **Astro 7** + **Sass** | prismatica (`apps/opposite-osiris`) | Mostly static marketing content: Astro ships near-zero JavaScript by default and pre-renders pages, which keeps the public site fast. | -| **React 19** + **Vite 6** + **TypeScript** | osionos, mail, calendar | A block editor is a heavily interactive UI; React's component model fits, and Vite gives fast builds. | -| **Tailwind CSS 4**, **lucide-react** | osionos | Utility-first styling framework and an icon set. | -| **Zustand** | osionos | Small, explicit client state store. | -| **Zod 4** | osionos | Schema validation: the diagram spec on the client, and the inputs of the AI service and the MCP server. | -| **livekit-client** | osionos | WebRTC video rooms. | -| **Playwright** | osionos, groot `tests/e2e/` | End-to-end browser tests. | -| **SvelteKit**, **Fastify**, **Rust β†’ WebAssembly** | hydrawn | The whiteboard: a WASM drawing engine with its own API and realtime services. | -| **Tailwind CSS 4** | mail, calendar | `src/styles.css` imports Tailwind's theme and utilities through `@tailwindcss/vite`, without preflight; the app's own rules sit on top. | - -### Backend β€” grobase +| Practice | Evidence (2026-11-04) | +|---|---| +| Issues | 129 on osionos, 38 on groot, 20 on prismatica, 16 on grobase (`gh issue list --state all`) | +| Peer review | Since 2026-09-21 (grobase, hydrawn) and up to 2026-11-04 (groot `develop`), a GitHub ruleset on each main repository requires one approving review and forbids force-pushes and branch deletion (`gh api repos//rulesets`). CodeRabbit reviews every pull request since 2026-10-23 | +| Commit subjects | groot rejects one-word commit subjects (`.github/workflows/commit-subjects.yml`) | -grobase is split into **planes** β€” groups of responsibilities that can be sized and replaced -independently β€” each written in the language that suits it. +--- -| Plane | Technology | Role | Why this language | -|---|---|---|---| -| **Control plane** | **Go** | Provisioning, tenancy, identity (`POST /v1/keys/verify` turns an API key into an identity) | Long-running, concurrent I/O; a provisioner needs to be correct and readable, not fast. | -| **Data plane** | **Rust** | Executes every query, applies per-owner scoping and ABAC locally | The hot path: predictable latency with no GC pauses, and memory safety where a bug would be a cross-tenant data leak. | -| **Realtime plane** | **Rust** | Turns data changes into events; WebSocket subscriptions (an opt-in profile, included in the `devlean` edition `make all` starts) | Same reasons; pluggable event bus. | -| **Application plane** | **TypeScript / NestJS** | Query and storage routers, schema service, permission engine. The older session, email and GDPR services remain only under the `legacy-node` profile; the Go orchestrator does their work | The original backend, progressively replaced by the Go and Rust planes. | -| **Auth** | **GoTrue** (grobase) + a Node **auth-gateway** (shipped by prismatica) | Sign-up, sign-in, sessions, email OTP, the hand-off to osionos | Battle-tested auth server rather than a hand-rolled one. | -| **Gateway** | **WAF** (nginx + ModSecurity/OWASP CRS) in front of **Kong** | The WAF is the public entry; Kong behind it does key auth and rate limiting | Keeps every internal service off the public network. | - -### Database - -- **PostgreSQL 16** is the primary database: the control-plane registries - (tenants, roles, policies, API keys) and all application data (see [Database - Schema](#database-schema)). **Why:** row-level security (RLS), JSONB for block - content and flexible properties, strong transactional guarantees, and a mature - ecosystem. RLS is what makes per-owner isolation enforceable inside the - database itself, not only in application code. -- **MySQL, MongoDB, Redis, MinIO** (object storage) run in the `devlean` edition - that `make all` starts; **SQLite**, **SQL Server** (needs the `engines` - plane), **DynamoDB** (an opt-in build) and remote **HTTP** APIs are supported - through the same data-plane interface. This is a deliberate feature of the - platform (one API over many engines), not a need of osionos itself. - -### Other significant technologies - -- **Docker / Docker Compose** β€” every component runs in a container; one `make all`. -- **A local certificate authority + TLS proxy** β€” all frontends served over trusted HTTPS locally. -- **LiveKit** β€” the WebRTC SFU for video rooms, a groot compose service started - by `make all`; the osionos bridge mints its tokens. -- **vault42 / 42ctl** β€” our own zero-knowledge secret store and CLI (separate repositories). -- **CI security scanning** β€” on every pull request, - [`.github/workflows/mini-baas-security.yml`](.github/workflows/mini-baas-security.yml) - runs Semgrep (SAST), `npm`/`pnpm audit`, Snyk (when a token is set) and Trivy; - Semgrep and Snyk report without blocking. Dependabot opens dependency updates. - SonarCloud analyses every pull request through its GitHub app - (`.sonarcloud.properties`), not through a workflow; `renovate.json` is present - but nothing runs it. - -### Justification of the major choices - -- **Why build a BaaS instead of a single backend?** Because the same backend pieces are rebuilt - for every project. We moved everything that varies between projects β€” routes, schemas, access - rules, configuration β€” from code into data, so a new application is a contract, not a server. -- **Why three languages?** Each grobase plane has a different job (Node also - runs the site's auth-gateway and the osionos bridge). Go for boring, correct - orchestration; Rust where performance and memory safety protect tenant - isolation; TypeScript where it already existed. The data plane was migrated - from TypeScript to Rust behind a per-request switch - (`RUST_DATA_PLANE_FORWARD`, on in the default compose), so the migration - needed no downtime and could be rolled back without a deployment. -- **Why per-request isolation?** Scoping on every request (instead of per connection) means a - pooled connection can never carry one user's permissions into another user's query. +## 🧰 Technologies + +| Layer | Choice | Why | +| --- | --- | --- | +| Editor | React 19, Vite, TypeScript | Block editing is highly interactive, and Vite builds fast | +| Public site | Astro, Sass | Static pages with almost no JavaScript | +| Design | @osionos/ui, Tailwind 4, Radix | Our own components, accessible primitives | +| Bridge | Node | Keeps the service key away from the browser | +| Gateway | Kong | One entry for key checks and rate limits | +| Services | NestJS | Storage, live queries and permissions | +| Control plane | Go | Concurrent and readable provisioning | +| Data, real-time | Rust, axum | Low latency, memory safety between tenants | +| Sign-in | GoTrue | A proven auth server, not a hand-written one | +| Storage | PostgreSQL, Redis, MinIO | Row-level security, fast state, files | +| Delivery | Docker, nginx | One command, HTTPS everywhere | + +### Why we made the big choices + +- **A BaaS, not a single backend**: what varies between projects moved from + code into data, so an app is a contract, not a server +- **Three backend languages**: Go for orchestration, Rust where isolation + matters, TypeScript where services already existed +- **A separate bridge**: the one place that holds the service key and checks + every caller +- **PostgreSQL**: one engine for tables, JSONB blocks and row-level security --- -## Database Schema - -The schema lives in two places: +## πŸ—„ Database schema -1. **Application schema** β€” idempotent SQL migrations in [`models/`](models/) - (`*-migration.sql`), applied to grobase's PostgreSQL by `make all` through - [`scripts/apply-models-migrations.sh`](scripts/apply-models-migrations.sh). - That runner deliberately skips seven files that target a database with an - integer `users.id` (used by the Electron native edition) or that later files - supersede: `user.sql`, `gdpr-migration.sql`, `auth-security-migration.sql`, - `rls-hardening-migration.sql`, `seeds.sql`, - `osionos-folder-surface-migration.sql` and - `osionos-wiki-surface-migration.sql`; its header explains why. -2. **Platform schema** β€” grobase's control-plane registries (tenants, mounts, roles, policies, - API keys), created by grobase's own migrations, plus per-application databases created at - runtime from each contract (e.g. `infra/config/contracts/website.schema.sql` in grobase). +- **Workspace**: has members, pages and channels +- **Page**: nests under a parent, keeps its blocks as JSONB +- **Channel**: has members and messages, with reactions -### Core application tables (osionos) +### Full diagram of the core tables ```mermaid erDiagram - osionos_workspaces ||--o{ osionos_workspace_members : "has" - osionos_workspaces ||--o{ osionos_pages : "contains" - osionos_pages ||--o{ osionos_pages : "parent of" - osionos_pages ||--o{ osionos_page_comments : "has" - osionos_pages ||--o{ osionos_page_favorites : "favourited in" - osionos_workspaces ||--o{ osionos_channels : "has" - osionos_channels ||--o{ osionos_channel_members : "has" - osionos_channels ||--o{ osionos_messages : "contains" - osionos_messages ||--o{ osionos_message_reactions : "has" - osionos_messages ||--o{ osionos_message_attachments : "has" + osionos_workspaces ||--o{ osionos_workspace_members : has + osionos_workspaces ||--o{ osionos_pages : contains + osionos_pages ||--o{ osionos_pages : parent_of + osionos_pages ||--o{ osionos_page_comments : has + osionos_pages ||--o{ osionos_page_favorites : favourited_in + osionos_workspaces ||--o{ osionos_channels : has + osionos_channels ||--o{ osionos_channel_members : has + osionos_channels ||--o{ osionos_messages : contains + osionos_messages ||--o{ osionos_message_reactions : has osionos_workspaces { uuid id PK @@ -570,9 +297,9 @@ erDiagram timestamptz created_at } osionos_workspace_members { - uuid workspace_id PK,FK + uuid workspace_id PK, FK uuid user_id PK - text role "owner | admin | editor | viewer" + text role "owner, admin, editor or viewer" text_array permissions } osionos_pages { @@ -581,20 +308,38 @@ erDiagram uuid parent_page_id FK uuid owner_id text title - text visibility "private | shared | public" + text visibility "private, shared or public" jsonb properties jsonb content "the blocks" boolean is_template + timestamptz archived_at + } + osionos_page_comments { + uuid id PK + uuid page_id FK + text block_id + uuid author_id + text content + timestamptz resolved_at + } + osionos_page_favorites { + uuid user_id PK + uuid page_id PK, FK } osionos_channels { uuid id PK uuid workspace_id FK - text kind "text | dm | voice | video | group" + text kind "text, dm, voice or video" text name boolean is_private jsonb abac text dm_key UK } + osionos_channel_members { + uuid channel_id PK, FK + uuid user_id PK + text role + } osionos_messages { uuid id PK uuid channel_id FK @@ -604,38 +349,44 @@ erDiagram timestamptz edited_at timestamptz deleted_at } + osionos_message_reactions { + uuid message_id PK, FK + uuid user_id PK + text emoji PK + } ``` -Other table groups, by migration file: - -| Area | Tables | Migration | -|---|---|---| -| Users & sessions (auth gateway DB) | `users`, `sessions`, `user_activities`, `user_tokens` | `user.sql` | -| Auth security & GDPR | `auth_audit_events`, `gdpr_requests`, `user_consents`, `newsletter_optins` | `auth-security-migration.sql`, `gdpr-migration.sql` (integer-id flavour); `auth-audit-events-migration.sql`, `newsletter-optin-migration.sql` (applied to grobase) | -| Site ↔ editor bridge | `osionos_bridge_identities`, `osionos_bridge_audit_events` | `osionos-bridge-migration.sql` | -| Social & communities | `osionos_communities`, `osionos_community_members`, `osionos_community_channels`, `osionos_connections`, `osionos_user_blocks`, `osionos_user_reports`, `osionos_feed_*` | `osionos-communities-`, `-social-`, `-chat-`, `-feed-engagement-migration.sql` | -| Databases-as-blocks | `osionos_object_databases`, `osionos_workspace_databases`, `osionos_app_connections` | `osionos-object-databases-`, `-workspace-databases-`, `-app-connections-migration.sql` | -| Publishing & sharing | `osionos_published_pages`, `osionos_share_rules`, `osionos_page_links` | `osionos-publish-`, `-share-rules-`, `-page-links-migration.sql` | -| Tasks & notifications | `osionos_tasks`, `osionos_notifications`, `osionos_push_subscriptions` | `osionos-tasks-`, `-engagement-`, `-push-migration.sql` | -| Mail & calendar | `mail_accounts`, `mail_messages`, `calendar_accounts`, `calendar_sources`, `calendar_event_cache` | `mail-migration.sql`, `calendar-migration.sql` | - -### Platform (grobase) registries - -Access control is stored as data: `roles`, `user_roles` and `resource_policies` -tables, evaluated by the SQL function `public.has_permission(...)` and mirrored -by the Rust data plane's own ABAC evaluator. Every table created through -grobase's schema API gets an `owner_id` column appended when the caller does not -declare one (contract schemas declare it themselves), which is what makes one -isolation rule apply to any table. - -**Two `users` tables.** `models/user.sql` defines `users.id` as `SERIAL` (integer): it belongs to -the auth gateway's database. The osionos tables reference users by `UUID`, in grobase's -PostgreSQL, where `make all` never applies `user.sql` β€” doing so fails on the uuid/integer -foreign-key clash (`scripts/apply-models-migrations.sh:27-38`). +### All the table groups + +| Area | Tables | +| --- | --- | +| Accounts | users, sessions, user_activities, user_tokens | +| Security, GDPR | auth_audit_events, gdpr_requests, user_consents | +| Site to editor | osionos_bridge_identities, osionos_bridge_audit_events | +| Social | osionos_communities, osionos_connections, osionos_user_blocks, osionos_feed_* | +| Databases as blocks | osionos_object_databases, osionos_workspace_databases | +| Publishing | osionos_published_pages, osionos_share_rules, osionos_page_links | +| Tasks, notifications | osionos_tasks, osionos_notifications, osionos_push_subscriptions | +| Mail, calendar | mail_accounts, mail_messages, calendar_accounts, calendar_event_cache | + +Access rules are data too: `roles`, `user_roles` and `resource_policies`. The +schema is in the SQL migrations under `models/`. There are two users tables: +`users` with integer ids belongs to the auth gateway, while osionos refers to +users by UUID. + +`make all` applies the idempotent `models/*-migration.sql` files to grobase's +PostgreSQL through +[`scripts/apply-models-migrations.sh`](scripts/apply-models-migrations.sh). +That runner skips seven files that target the integer-id `users` database or +that later files supersede; its header names them. The access rules are +evaluated by the SQL function `public.has_permission(...)` and mirrored by the +Rust data plane's own ABAC evaluator. --- -## Features List +## ✨ Features and who built them + +Who comes from the commits that touch each feature's code. "Who" counts the non-merge commits that touch the feature's paths (last column), reachable from the commit groot `main` pins for that repository, read through @@ -664,120 +415,97 @@ A commit count is not effort, and pairing or review does not show in it. | Contract-driven provisioning | One JSON contract β†’ isolated database, roles, keys and frontend `.env` | dlesieur 9 | grobase `scripts/provision-contract.sh`, `infra/config/contracts` | | Self-serve applications | `POST /v1/tenants/me/apps` creates a new isolated app with its own database and key, behind `APPS_SELFSERVE_ENABLED` (off unless set) | dlesieur 32 | grobase `src/control-plane/internal/tenants` | | Multi-engine data API | One API and one key over PostgreSQL, MySQL, MongoDB, Redis (default edition), SQLite, SQL Server (`engines` plane), HTTP and DynamoDB (opt-in build) | dlesieur 95, vjan-nie 3 | grobase `src/data-plane-router` | -| Public API | Key-authenticated, rate-limited API behind Kong, documented with OpenAPI at `/docs/` on the `:8444` edge | dlesieur 64, vjan-nie 5, shashemi 1 | grobase `infra/docker/services/kong`, `infra/config/openapi`; groot `infrastructure/tls/nginx.conf` | +| Public API | Key-authenticated, rate-limited API behind Kong (not claimed as a module: see [Modules and points](#-modules-and-points)) | dlesieur 64, vjan-nie 5, shashemi 1 | grobase `infra/docker/services/kong`, `infra/config/openapi`; groot `infrastructure/tls/nginx.conf` | --- -## Modules - -The subject requires **14 points** (Major = 2, Minor = 1). A module that cannot -be demonstrated live scores zero. Every row is graded against **the local stack -built from `main`** (`git checkout main`, `make all SKIP_SYNC=1`, which starts -grobase's `devlean` edition; see [Defense bring-up](#defense-bring-up-pinned)) -by **one rule**, groot's own. It keeps the clause of osionos -[`docs/ai/README.md`](https://github.com/Univers42/osionos/blob/1c1d5d849c1dc43460a194f7ee3d97e17505205c/docs/ai/README.md) -lines 7–8 (a row is never A when its demonstration needs something switched off -or unmeasured), but it is stricter than that file's legend, which grades a -documented profile A: - -- **A**: the demo works on that stack with nothing switched on, and a green CI - job drives that path against a running service. -- **B**: it works, but the demo needs something switched on (a flag, a profile, - an edition, a provider credential), or its only evidence is unit tests with - fakes or a run that was never repeated. -- **C**: partial. **Not claimed.** These rows are listed under [Not claimed - yet](#not-claimed-yet). - -**Claimed = A and B only: 30 points against the 14 required. 6 of them are A.** -Paths are prefixed by repository: `G` groot, `GB` grobase (`apps/grobase`), `OS` -osionos (`apps/osionos/app`), `OO` opposite-osiris (`apps/opposite-osiris`). -They point at the commits pinned by groot's `main` branch. CI names a workflow -job: `GB-CI` is grobase `ci.yml`, `OS-CI` is osionos `ci.yml`, `OO-CI` is +## 🎯 Modules and points + +**10 modules, 16 points**: 6 Majors and 4 Minors. The subject asks for 14, and +bonus points stop at 19. + +### Why we chose each module + +| Module | Points | Why | +| --- | --- | --- | +| W1 Frameworks | 2 | React for the editor, axum and NestJS for the services | +| W4 Real-time | 2 | Chat, presence and live editing need a push channel | +| W5 User interaction | 2 | Chat, profiles and contacts are the product | +| W9 Collaboration | 1 | Live editing is the point of a shared workspace | +| W12 Design system | 1 | One token system keeps every app consistent | +| W13 Advanced search | 1 | Databases are the core object, so finding rows matters | +| U1 User management | 2 | Everything hangs off a user with a profile | +| U3 OAuth 2.0 | 1 | Sign in with Google or GitHub, no new password | +| D3 Microservices | 2 | Each grobase plane scales and is replaced on its own | +| I3 LLM interface | 2 | The assistant is the workspace's main AI feature | + +W1 replaces the two Minors W2 and W3, which cannot be counted as well. + +### Evidence + +For the modules above that our earlier module table also covered: the code, +the tests and the CI job that runs them, and the docs. Paths are prefixed by +repository: `G` groot, `GB` grobase (`apps/grobase`), `OS` osionos +(`apps/osionos/app`), `OO` opposite-osiris (`apps/opposite-osiris`), at the +commits pinned by groot's `main` branch on 2026-11-05. CI names a workflow job: +`GB-CI` is grobase `ci.yml`, `OS-CI` is osionos `ci.yml`, `OO-CI` is prismatica `ci.yml`. -| # | Module | Category | Type | Pts | Grade | Core files | Tests β†’ CI job | Docs | Demo (3 steps) | -|---|---|---|---|--:|:--:|---|---|---|---| -| 1 | Public API (key, rate limit, docs, β‰₯5 endpoints) | Web | Major | 2 | **A** | `GB infra/docker/services/kong/conf/kong.yml:221-245` (`/rest/v1`: key-auth + rate-limiting); `GB infra/config/openapi/grobase-public.json` | `GB scripts/verify/m102-gateway-query-path.sh` β†’ GB-CI "Integration Tests" (`ci.yml:927`) | `GB infra/config/openapi/README.md` | `make all SKIP_SYNC=1` β†’ `/rest/v1` without `apikey` answers 401, with it 200 β†’ repeat until 429 | -| 2 | Backend as microservices | DevOps | Major | 2 | **A** | `GB orchestrators/compose/base/*.yml` (16 files); `GB orchestrators/makes/30-planes.mk:33`; `GB src/control-plane`; `GB src/data-plane-router` | GB-CI unit suites (`ci.yml:1436`) + "Engine conformance" legs (`ci.yml:1532`) | [`wiki/platform/02-planes.md`](wiki/platform/02-planes.md) | `make -C apps/grobase planes` β†’ `docker compose ps` β†’ `make healthcheck` | -| 3 | Real-time features via WebSockets | Web | Major | 2 | **B** | `GB infra/docker/services/realtime/realtime-agnostic/crates/realtime-server/src/server.rs:243` (`/ws`); `GB kong.yml:314-315` (`/realtime/v1/ws`); `OS scripts/bridge-social-core.mjs:192-201` (the bridge publishes); `OS src/services/realtime-messages/wsTransport.ts:81` (chat over the socket, on whenever a session exists) | `GB scripts/verify/m201-realtime-issuer.sh` β†’ GB-CI "Integration Tests" (`ci.yml:855`): it checks the `/ws` handshake (`AUTH_OK`) on the running realtime, not delivery between two users; m189 (`ci.yml:735`); `rust-realtime` suite and m184 (`ci.yml:1436,1479`). osionos's chat browser test runs against a stub bridge (OS-CI "Browser tests (stub bridge URL)", `ci.yml:432-433`) | [`wiki/platform/05-realtime.md`](wiki/platform/05-realtime.md) | sign in as two users in two browsers β†’ open the same direct message in both β†’ a message sent in one appears in the other without a reload. A second default demo: two windows on one Whiteboard board, where each stroke appears in the other (`G tests/e2e/specs/dw8-realtime.spec.ts`, `make e2e`, run by hand) | -| 4 | Use a framework for both the frontend and backend | Web | Major | 2 | **A** | Frontend: React 19 (`OS package.json:105-106`). Backend: NestJS 11, grobase's application plane (`GB src/package.json:52`; `GB src/apps/query-router/src/main.ts:30`, `NestFactory`); axum 0.7 is a supporting example (`GB src/data-plane-router/Cargo.toml:59`) | React: OS-CI "Browser tests", Playwright against the served app in 6 shards (`ci.yml:338-339`). NestJS: GB-CI "Integration Tests" builds query-router from the pin and starts it (`ci.yml:900-904`), then m213 sends `/query/v1/execute` through Kong to it (`ci.yml:917`); the `nestjs` unit suite (`ci.yml:1436`) | [`wiki/platform/02-planes.md`](wiki/platform/02-planes.md) | `make all SKIP_SYNC=1` β†’ `docker compose ps` lists `mini-baas-query-router` (`GB orchestrators/compose/base/app-services.yml:12`, in the `devlean` edition) β†’ open osionos at `https://localhost:3001` | -| 5 | Allow users to interact with other users (chat, profile, friends) | Web | Major | 2 | **B** | `OS scripts/bridge-chat.mjs`; `OS scripts/bridge-account-profile.mjs`; `OS scripts/bridge-social.mjs`; `OS src/features/chat`; `OS src/features/connections` | `OS tests/canvas/chat-catch-up.test.ts` β†’ OS-CI "quality (root)" Canvas unit tests (`ci.yml:285`); `OS tests/bridge/bridge-chat-reply-notify`, `bridge-account-profile`, `bridge-social-connections`, `bridge-connection-transitions` β†’ Bridge tests (`ci.yml:291`). Unit tests with fakes: no CI job drives two users against a running stack | [`wiki/product/02-osionos.md`](wiki/product/02-osionos.md) | as row 3 β†’ user A sends a friend request, user B accepts β†’ each opens the other's profile | -| 6 | Standard user management and authentication | User management | Major | 2 | **B** | `OS scripts/bridge-account-profile.mjs`; `OS scripts/bridge-avatar.mjs`; `OS scripts/bridge-presence.mjs`; `OS src/features/collab/model/collabPresence.ts:63` (initials when there is no avatar) | `OS tests/bridge/bridge-avatar` (upload, type and size refusal), `bridge-presence` (online status), `bridge-account-profile` β†’ OS-CI Bridge tests (`ci.yml:291`). Unit tests with fakes | [`wiki/product/02-osionos.md`](wiki/product/02-osionos.md) | sign up on `https://localhost:4322` β†’ upload an avatar and edit the profile β†’ a friend sees you online | -| 7 | Real-time collaboration | Web | Minor | 1 | **B** | `OS src/features/collab/model/{remoteCursors,collabPresence,blockSync}.ts`; `OS scripts/bridge-collab.mjs:626`; `GB …/realtime-auth/src/jwt/mod.rs:90-94` (`collab:` closed to wildcard tokens) | `OS tests/bridge/bridge-collab-*` (9) β†’ OS-CI Bridge tests (`ci.yml:291`) | [`wiki/product/02-osionos.md`](wiki/product/02-osionos.md) | rebuild osionos with `VITE_OSIO_COLLAB_SHARED=1` (off by default) β†’ two browsers on one page β†’ watch the remote cursor and edit the same block from both | -| 8 | Advanced permissions | User management | Major | 2 | **B** | `OS scripts/bridge-roles.mjs:28-29`; `OS scripts/bridge-share-rules.mjs:38`; `OS scripts/bridge-page-access.mjs:217`; `OS src/features/admin-users/api.ts` | `OS tests/bridge/bridge-page-access*` (7), `bridge-share-rules-*` (2) β†’ OS-CI Bridge tests | [`wiki/platform/07-isolation.md`](wiki/platform/07-isolation.md) | sign in as admin β†’ change a member's role β†’ that member loses access | -| 9 | Organisation system | User management | Major | 2 | **B** | `OS scripts/bridge-workspace-teams.mjs:229`; `OS scripts/bridge-workspace-invites.mjs:134`; `OS src/features/settings/teamspaces/TeamspacesPanel.tsx` | `OS tests/bridge/bridge-workspace-*` (7) β†’ OS-CI Bridge tests | [`wiki/platform/06-contracts.md`](wiki/platform/06-contracts.md) | create a workspace β†’ create a teamspace β†’ invite a user, who accepts | -| 10 | Remote authentication with OAuth 2.0 | User management | Minor | 1 | **B** | `OO scripts/auth/oauth-providers.mjs`; `OO scripts/auth/oauth-link.mjs`; `OO src/scripts/portal-oauth.ts`; `GB orchestrators/compose/base/auth-api.yml:64-67` (GoTrue's Google provider) | `OO tests/unit/auth/oauth-providers`, `oauth-link`, `gateway-oauth-{start,link,collision}` (5) β†’ OO-CI "Quality Gate" `pnpm test` (`ci.yml:31-32`) | `OO docs/adr/0002-oauth-link-existing-account.md` | set `GOOGLE_OAUTH_ENABLED=true`, `GOOGLE_CLIENT_ID` and `GOOGLE_CLIENT_SECRET` for grobase (Google provider credentials, not shipped) β†’ `make all SKIP_SYNC=1` β†’ "Continue with Google" on `https://localhost:4322` | -| 11 | Health/status page, backups, disaster recovery | DevOps | Minor | 1 | **B** | `GB infra/docker/services/status-page/{server,availability}.py`; `GB orchestrators/compose/base/ops.yml`; `GB scripts/verify/m241-status-backup-recovery.sh` | m241 β†’ GB `operational-reliability.yml:27`; m192 β†’ GB-CI (`ci.yml:582`), **red on grobase `develop`** ([run 41870904860](https://github.com/Univers42/grobase/actions/runs/41870904860)) | [`wiki/operations/04-troubleshooting.md`](wiki/operations/04-troubleshooting.md) | bring up the `ops` plane (not in the default edition) β†’ open the status page β†’ stop a container, then `m241 --live` | -| 12 | GDPR compliance | Data | Minor | 1 | **B** | `OS scripts/bridge-account-export.mjs:143`; `OS scripts/bridge-account-erasure.mjs:79`; `OS src/features/settings/ui/AccountExportSection.tsx`; `OS src/features/settings/ui/danger/AccountDangerZone.tsx` | `OS tests/bridge/bridge-account-export*` (3), `bridge-account-erasure` (1) β†’ OS-CI Bridge tests; `G .github/workflows/models-grants.yml` | [`wiki/security/01-model.md`](wiki/security/01-model.md) | Settings β€Ί export my data β†’ open the archive β†’ delete the account | -| 13 | Two-factor authentication | User management | Minor | 1 | **B** | `OS scripts/bridge-account-mfa-totp.mjs:68`; `OS scripts/bridge-account-mfa.mjs:187`; `OS scripts/bridge-account-passkeys.mjs:100`; `OS src/features/settings/profile/TwoFactorModal.tsx` | `OS tests/bridge/bridge-account-mfa-*` (5), `bridge-account-passkeys` (1) β†’ OS-CI Bridge tests | osionos `docs/account-platform` | enable TOTP β†’ sign out β†’ sign in with the code | -| 14 | **Module of choice:** contract-driven application factory | Free choice | Major | 2 | **B** | `GB src/control-plane/internal/tenants/selfserve_apps.go:55-58`; `GB src/control-plane/cmd/tenant-control/mount_appsselfserve.go:26`; `GB scripts/verify/m177-selfserve-apps.sh` | m176 β†’ GB-CI (`ci.yml:192`, static leg, in the Shellcheck job); m176 and m177 β†’ groot `modules-of-choice.yml:39-40`, static mode only | [`wiki/platform/06-contracts.md`](wiki/platform/06-contracts.md), [justification](#justification--module-of-choice-major-contract-driven-application-factory) | set `APPS_SELFSERVE_ENABLED` (empty by default) β†’ OTP login, `POST /v1/tenants/me/apps` twice β†’ one app's key reads the other: 4xx | -| 15 | **Module of choice:** SSRF guard on the HTTP engine | Free choice | Minor | 1 | **B** | `GB src/data-plane-router/crates/data-plane-pool/src/dial_guard/classify.rs:90-155`; `GB src/data-plane-router/crates/data-plane-pool/src/http/validate.rs:83-153` | `GB …/dial_guard/classify_tests.rs` β†’ GB-CI `rust-data-plane` + "Engine conformance (http)" | [`wiki/security/04-network.md`](wiki/security/04-network.md), [justification](#justification--module-of-choice-minor-ssrf-guard-on-the-http-engine) | mount an `http` source at `169.254.169.254` β†’ query it β†’ refused. Whether the default edition mounts the `http` engine is unverified (guess: it does not) | -| 16 | Complete LLM system interface: text, streaming, error handling, rate limiting. Images exist but are **off by default** and not part of this claim | Artificial Intelligence | Major | 2 | **B** | osionos `services/osionos-ai/src/routes/runs.ts:178-324`, `src/runs/{runner,upstream,upstreamError,streamPump}.ts`, `src/rateLimit.ts:84`, `src/http.ts:108-129`; client `src/features/ai/model/runStream.ts`. All 12 hops: [`wiki/project/ai-llm-interface.md`](wiki/project/ai-llm-interface.md) | `services/osionos-ai/tests/{runs,complete,upstreamError*,runRateLimit,failover,quotaFailover}.test.ts` β†’ OS-CI "AI service (tsc + suite)" (`ci.yml:314`); `tests/canvas/ai-{chat-store,run-dispatch,run-stream}.test.ts` β†’ "quality (root)" Canvas unit tests (`ci.yml:285`) | [`wiki/project/ai-llm-interface.md`](wiki/project/ai-llm-interface.md), [`wiki/runbooks/osionos-ai.md`](wiki/runbooks/osionos-ai.md), [M1 sheet](https://github.com/Univers42/osionos/blob/develop/docs/ai/modules/m1-images.md) | `docker compose --profile ai up -d --build` β†’ ask a question and watch it stream β†’ reload mid-answer (it resumes), or stop it | -| 17 | Complete RAG on a large dataset | Artificial Intelligence | Major | 2 | **B** | osionos `services/osionos-ai/src/retrieval/search.ts:277,469,564`, `src/retrieval/rerank.ts:16`; `G models/osionos-ai-rag-migration.sql:60,91` | `services/osionos-ai/tests/{retrieval,aclAgreement,rerank}.test.ts` β†’ OS-CI AI service | [M2 sheet](https://github.com/Univers42/osionos/blob/develop/docs/ai/modules/m2-rag.md), osionos `services/osionos-ai/eval/wiki/RESULTS-rag.md` | `--profile ai` β†’ ask a workspace question β†’ click a citation, then show the 110 363-chunk measurement (run on a separate eval stack; the reranker is off by default) | -| 18 | Content moderation AI | Artificial Intelligence | Minor | 1 | **B** | osionos `scripts/bridge-moderation-gate.mjs:70`; `services/osionos-ai/src/moderation/l1.ts:99` | `services/osionos-ai/tests/moderation*.test.ts` (4) β†’ OS-CI AI service; `tests/bridge/bridge-moderation*` (5) β†’ Bridge tests | [M4 sheet](https://github.com/Univers42/osionos/blob/develop/docs/ai/modules/m4-moderation.md) | set `OSIONOS_MODERATION_ACTIONS=hold` and `OSIONOS_MODERATION_STATE=1` β†’ post a toxic comment β†’ it waits in the admin console (no auto-removal, by design) | -| 19 | Voice / speech | Artificial Intelligence | Minor | 1 | **B** | osionos `services/osionos-ai/src/routes/voice.ts:104`, `src/features/voice/ui/DictationButton.tsx`, `src/features/voice/model/readAloud.ts`; `G docker-compose.yml:788-790` | `services/osionos-ai/tests/voice.test.ts` β†’ OS-CI AI service | [M5 sheet](https://github.com/Univers42/osionos/blob/develop/docs/ai/modules/m5-voice.md) | `--profile ai`, with the speech model fetched into its volume β†’ dictate into a page β†’ read it aloud (read-aloud voices not measured yet) | - -No separate framework Minor is claimed beside row 4. - -**Point calculation** (free choice is capped at 3, -[`wiki/project/01-subject.md`](wiki/project/01-subject.md#points)) - -| | Before (2026-10) | After | -|---|--:|--:| -| Catalogue modules claimed (rows 1–13) | 18 | 21: 6 A (rows 1, 2, 4) + 15 B (rows 3, 5–13) | -| Modules of choice (rows 14–15, cap 3) | 3 | 3, all B | -| Artificial Intelligence modules (rows 16–19) | 10 | 6, all B | -| **Total claimed** | **31 listed** | **30 (A + B)** | -| Of which A | 10 | **6** | -| Required | 14 | 14 | - -What moved: under the old legend ("works as-is"), six rows were A. Under one -rule, the rows whose demonstration needs a switch or whose only evidence is -unit-level are B: WebSockets (CI checks the handshake, not delivery), real-time -collaboration (off by default), the contract factory and the SSRF guard. Row 4 -is new at A. The rows below no longer count. - -### Not claimed yet - -| Module | Was | Why it is not claimed, in one line | -|---|---|---| -| Monitoring with Prometheus + Grafana, DevOps, Major | 2, C | The default edition ships no observability plane, and its gate m85 runs in no workflow | -| File upload and management, Web, Minor | 1, C | Avatars and page files upload (`OS scripts/bridge-media.mjs:160`), but no general file manager is verified, and gate m159 runs in no workflow | -| ML recommendation system, AI, Major | 2, B | Needs 20+ real users, the offline numbers are synthetic, and the `osionos-ml` Python tests run in no CI | -| Sentiment analysis, AI, Minor | 1, C | The model licence is undeclared | -| Image recognition and tagging, AI, Minor | 1, C | The browser flow is unverified. Locally the VLM starts with `--profile ai` (`G docker-compose.yml:1004-1006`); `--profile ai-vlm` exists only in `docker-compose.public.yml:211-212` | -| AI opponent, AI | β€” | osionos has no game | -| Multiple languages, Accessibility | β€” | `i18next` is a dependency (osionos `package.json:92`), but no source file imports it and no translation resource exists | - -### Justification β€” Module of choice (Major): contract-driven application factory - -- **Why this module.** It is the central idea of the project: a backend with zero - application-specific code, where an application is a declarative contract. -- **Technical challenge.** From one JSON file, a generic provisioner must create an isolated - database, apply its schema, create roles and policies, mint scoped API keys and emit the - frontend's `PUBLIC_*` configuration β€” idempotently, on every boot. The self-serve endpoint - `POST /v1/tenants/me/apps` does it on demand, with a fresh `CREATE DATABASE` per application. -- **Value.** A new application costs a contract instead of a new server. -- **Why it deserves a Major.** It spans the Go control plane, the Rust data plane and the - database layer, and it comes with isolation proofs: gate `m177` proves self-serve creation, and - gate `m176` proves one application cannot read another's data (a foreign mount resolves to zero - rows β€” a 404, never a cross read). - -### Justification β€” Module of choice (Minor): SSRF guard on the HTTP engine - -- **Why this module.** The data plane can mount a remote HTTP API as if it were a database β€” - which also means "let a user make our server fetch any URL", the classic SSRF hole. -- **Technical challenge.** The guard blocks loopback, private (RFC 1918), link-local β€” including - the cloud metadata endpoint `169.254.169.254` β€” CGNAT, IPv6 ULA and link-local, IPv4-mapped - addresses and internal hostnames, then **pins** the connection to the validated public IP so a - later DNS rebind cannot redirect it inward. -- **Value and weight.** Small, self-contained and demonstrable in seconds (a - mount pointing at `169.254.169.254` is refused), and it shows security - judgement on a feature we chose to build. +| Module | Core files | Tests β†’ CI job | Docs | +|---|---|---|---| +| W1 Frameworks | Frontend: React 19 (`OS package.json:105-106`). Backend: NestJS 11, grobase's application plane (`GB src/package.json:52`; `GB src/apps/query-router/src/main.ts:30`, `NestFactory`); axum 0.7 is a supporting example (`GB src/data-plane-router/Cargo.toml:59`) | React: OS-CI "Browser tests", Playwright against the served app in 6 shards (`ci.yml:338-339`). NestJS: GB-CI "Integration Tests" builds query-router from the pin and starts it (`ci.yml:900-904`), then m213 sends `/query/v1/execute` through Kong to it (`ci.yml:917`); the `nestjs` unit suite (`ci.yml:1436`) | [`wiki/platform/02-planes.md`](wiki/platform/02-planes.md) | +| W4 Real-time | `GB infra/docker/services/realtime/realtime-agnostic/crates/realtime-server/src/server.rs:243` (`/ws`); `GB kong.yml:314-315` (`/realtime/v1/ws`); `OS scripts/bridge-social-core.mjs:192-201` (the bridge publishes); `OS src/services/realtime-messages/wsTransport.ts:81` (chat over the socket, on whenever a session exists) | `GB scripts/verify/m201-realtime-issuer.sh` β†’ GB-CI "Integration Tests" (`ci.yml:855`): it checks the `/ws` handshake (`AUTH_OK`) on the running realtime, not delivery between two users; m189 (`ci.yml:735`); `rust-realtime` suite and m184 (`ci.yml:1436,1479`). osionos's chat browser test runs against a stub bridge (OS-CI "Browser tests (stub bridge URL)", `ci.yml:432-433`) | [`wiki/platform/05-realtime.md`](wiki/platform/05-realtime.md) | +| W5 User interaction | `OS scripts/bridge-chat.mjs`; `OS scripts/bridge-account-profile.mjs`; `OS scripts/bridge-social.mjs`; `OS src/features/chat`; `OS src/features/connections` | `OS tests/canvas/chat-catch-up.test.ts` β†’ OS-CI "quality (root)" Canvas unit tests (`ci.yml:285`); `OS tests/bridge/bridge-chat-reply-notify`, `bridge-account-profile`, `bridge-social-connections`, `bridge-connection-transitions` β†’ Bridge tests (`ci.yml:291`). Unit tests with fakes: no CI job drives two users against a running stack | [`wiki/product/02-osionos.md`](wiki/product/02-osionos.md) | +| W9 Collaboration | `OS src/features/collab/model/{remoteCursors,collabPresence,blockSync}.ts`; `OS scripts/bridge-collab.mjs:626`; `GB …/realtime-auth/src/jwt/mod.rs:90-94` (`collab:` closed to wildcard tokens) | `OS tests/bridge/bridge-collab-*` (9) β†’ OS-CI Bridge tests (`ci.yml:291`) | [`wiki/product/02-osionos.md`](wiki/product/02-osionos.md) | +| W12 Design system | β€” | β€” | β€” | +| W13 Advanced search | β€” | β€” | β€” | +| U1 User management | `OS scripts/bridge-account-profile.mjs`; `OS scripts/bridge-avatar.mjs`; `OS scripts/bridge-presence.mjs`; `OS src/features/collab/model/collabPresence.ts:63` (initials when there is no avatar) | `OS tests/bridge/bridge-avatar` (upload, type and size refusal), `bridge-presence` (online status), `bridge-account-profile` β†’ OS-CI Bridge tests (`ci.yml:291`). Unit tests with fakes | [`wiki/product/02-osionos.md`](wiki/product/02-osionos.md) | +| U3 OAuth 2.0 | `OO scripts/auth/oauth-providers.mjs`; `OO scripts/auth/oauth-link.mjs`; `OO src/scripts/portal-oauth.ts`; `GB orchestrators/compose/base/auth-api.yml:64-67` (GoTrue's Google provider) | `OO tests/unit/auth/oauth-providers`, `oauth-link`, `gateway-oauth-{start,link,collision}` (5) β†’ OO-CI "Quality Gate" `pnpm test` (`ci.yml:31-32`) | `OO docs/adr/0002-oauth-link-existing-account.md` | +| D3 Microservices | `GB orchestrators/compose/base/*.yml` (16 files); `GB orchestrators/makes/30-planes.mk:33`; `GB src/control-plane`; `GB src/data-plane-router` | GB-CI unit suites (`ci.yml:1436`) + "Engine conformance" legs (`ci.yml:1532`) | [`wiki/platform/02-planes.md`](wiki/platform/02-planes.md) | +| I3 LLM interface | osionos `services/osionos-ai/src/routes/runs.ts:178-324`, `src/runs/{runner,upstream,upstreamError,streamPump}.ts`, `src/rateLimit.ts:84`, `src/http.ts:108-129`; client `src/features/ai/model/runStream.ts`. All 12 hops: [`wiki/project/ai-llm-interface.md`](wiki/project/ai-llm-interface.md) | `services/osionos-ai/tests/{runs,complete,upstreamError*,runRateLimit,failover,quotaFailover}.test.ts` β†’ OS-CI "AI service (tsc + suite)" (`ci.yml:314`); `tests/canvas/ai-{chat-store,run-dispatch,run-stream}.test.ts` β†’ "quality (root)" Canvas unit tests (`ci.yml:285`) | [`wiki/project/ai-llm-interface.md`](wiki/project/ai-llm-interface.md), [`wiki/runbooks/osionos-ai.md`](wiki/runbooks/osionos-ai.md), [M1 sheet](https://github.com/Univers42/osionos/blob/develop/docs/ai/modules/m1-images.md) | + +### What we did not claim + +A module with one failing sub-task is not claimed, and the game and blockchain +modules are out because the project has no game. + +### The cheapest next points + +1. **Public API**: add PUT and DELETE to personal tokens, serve the docs (+2) +1. **App factory**: set the flag and record a demo (+2) +1. **Export and import**: CSV import with stricter validation (+1) +1. **PWA**: PNG icons, fix the offline message (+1) +1. **2FA**: the sign-in code prompt, which also fixes the lockout (+1) +1. **GDPR**: run the deletion, send the emails (+1) --- -## Individual Contributions +## πŸ™Œ Individual contributions + +> [!NOTE] +> A commit count is not effort: pairing, review and design do not show in it. +> Commits concentrate on dlesieur: 86%, 5,895 of the 6,876 non-merge member +> commits at the commits groot `main` (`948487ee`) pins, counted on 2026-11-05 +> with the [contributions method](wiki/project/contributions-method.md). Be +> ready to explain the split. + +### Per member + +- **`dlesieur`**, Product Owner and developer: grobase (control plane, data + plane, realtime, gateway, contracts), osionos (editor, chat, collaboration, + the bridge, the AI service), mail, calendar, the root's infrastructure and CI. +- **`vjan-nie`**, Project Manager and developer: the editable block logic of + osionos (`src/features/block-editor`: the block editor surface, its state + hook, the context menu, drag handles, code and Mermaid blocks, block + history). 42 of the 254 non-merge commits on that folder at the osionos + commit `main` pins (`735c40fd`), second to dlesieur's 164. Also QA tooling, + end-to-end tests, the wiki and README, and submodule pins. +- **`danfern3`**, tech lead and developer: grobase Docker images and Kubernetes + deployment, monkey-bot, osionos editor features. +- **`serjimen`**, developer: the public site (opposite-osiris), osionos + accessibility and editor features. +- **`shashemi`**, developer: security fixes and SAST, the site's legal pages, + the TLS edge, end-to-end fixes. ### Who committed where @@ -803,119 +531,93 @@ pinned commit (open branches and pull requests) does not show above. A commit count is not effort: one large commit and one typo fix weigh the same, and pairing and review do not show in it. -Merged pull requests per login in the project's repositories (groot, osionos, grobase, -prismatica = opposite-osiris, osionos-mail, osionos-calendar, hydrawn, QA, monkey-bot), from -`gh search prs --owner Univers42 --author --merged` on 2026-11-04: - -| GitHub | Merged PRs | Where | -|---|--:|---| -| @LESdylan | 260 | osionos 138, grobase 61, groot 56, prismatica 3, osionos-mail 1, osionos-calendar 1 | -| @vjan-nie | 72 | groot 45, osionos 17, prismatica 6, grobase 4 | -| @SMOSTAFAH1 | 57 | prismatica 20, groot 19, osionos 12, grobase 6 | -| @DjSurgeon | 8 | prismatica 7, osionos 1 | -| @danielfdez17 | 4 | prismatica 4 | +### Challenges the team faced -### Per member - -- **`dlesieur`**, Product Owner: grobase (control plane, data plane, realtime, - gateway, contracts), osionos (editor, chat, collaboration, the bridge, the AI - service), mail, calendar, the root's infrastructure and CI. -- **`vjan-nie`**, Project Manager: QA tooling, end-to-end tests, the wiki and - README, submodule pins, Docker infrastructure. -- **`serjimen`**, frontend tech lead: the public site (opposite-osiris), - osionos accessibility and editor features. -- **`danfern3`**, backend tech lead: grobase Docker images and Kubernetes - deployment, monkey-bot, osionos editor features. -- **`shashemi`**, QA / quality tech lead: security fixes and SAST, the site's - legal pages, the TLS edge, end-to-end fixes. - -### Challenges and lessons - -The project met many productive problems and blockers: lack of autonomy, -communication problems, frustration. We addressed them with dedicated meetings -and by applying TDD and DevOps practices, toward a frictionless workflow. We -acknowledge our technical debt openly -([`wiki/security/03-known-weaknesses.md`](wiki/security/03-known-weaknesses.md) -is one record of it). Several technologies and architectures were rebuilt from -scratch when the first choice proved wrong late: the data plane moved from -TypeScript to Rust behind a per-request switch (`RUST_DATA_PLANE_FORWARD`, on -by default in grobase `orchestrators/compose/base/app-services.yml:96`); -grobase's Node session, email and GDPR services were retired and their work -moved to the Go orchestrator (`app-services.yml:133,190,448`); and osionos's -legacy second brain was replaced by graph_render (groot `fresh-machine.yml:363` -checks it on a live stack; the legacy one survives only behind a gate, osionos -`ci.yml:383-388`). The project's maturity comes from that intense team -experience, with its ups and downs. +- Moving the data plane from TypeScript to Rust with no downtime, using a + per-request switch and a shadow mode +- Keeping eight database adapters at parity with a conformance battery against + real engines +- Auditing our own authorisation path and shipping a reversible fix --- -## Resources - -### References - -- Docker Compose β€” -- PostgreSQL row-level security β€” -- Kong Gateway β€” -- Supabase Auth (GoTrue) β€” -- The Go Programming Language documentation β€” -- The Rust Programming Language β€” -- NestJS β€” -- Astro β€” -- React β€” Β· Vite β€” Β· Tailwind CSS β€” -- LiveKit β€” -- NIST SP 800-162, *Guide to Attribute Based Access Control* β€” -- OWASP SSRF Prevention Cheat Sheet β€” -- OWASP Top 10 β€” -- GDPR, full text β€” +## πŸ“š Resources ### How AI was used -This project was built with **heavy AI assistance**, and we state it plainly. - -- **Tools and tasks.** Every member used a variety of AI tools (chat - assistants, coding agents, review bots) across code, tests, reviews and - documentation. -- **What the repositories record.** Claude Code with the team's kit in groot's +- **Code**: AI assistants wrote and refactored code across the stack +- **Tests**: verification gates and test scripts, run against the live stack +- **Review**: AI-assisted pull request review and a security review +- **Documentation**: drafted from the code, then checked against the repository +- **Agents in the history**: `opencode` (6 commits) and Space Bunny on hydrawn + (111 commits) +- **Control**: a human merges every change, and each member must be able to + explain the code in their area +- **Tools the repositories record**: Claude Code with the team's kit in groot's [`.claude`](.claude) submodule; Gemini (`.gemini/`) and the opencode agent in osionos; an OpenCode agent team on hydrawn (`BUNNY.md`); GitHub Copilot and - CodeRabbit on pull requests. -- **Commit history.** On 2026-11-04 the history of groot, osionos and grobase + CodeRabbit on pull requests +- **Commit trailers**: on 2026-11-04 the history of groot, osionos and grobase was rewritten to remove `Co-authored-by` trailers. Since then no commit carries a co-author line, so the history does not show which commits were written with an AI tool, and must not be read as AI-free - ([`wiki/project/contributions-method.md`](wiki/project/contributions-method.md)). -- **How we kept control.** A human merges every pull request; nothing reaches - `develop` or `main` by an agent alone. Approval by a second member is required - by the rulesets but thin in practice (see [Project - Management](#project-management)). A change is accepted only when its - verification gate passes. Every member must be able to explain the AI-assisted - code in their area. + ([`wiki/project/contributions-method.md`](wiki/project/contributions-method.md)) + +### References + +- [Docker Compose](https://docs.docker.com/compose/), [PostgreSQL row-level security](https://www.postgresql.org/docs/current/ddl-rowsecurity.html), [Kong Gateway](https://docs.konghq.com/gateway/latest/) +- [React](https://react.dev/), [Vite](https://vite.dev/), [Tailwind CSS](https://tailwindcss.com/docs), [Astro](https://docs.astro.build/), [NestJS](https://docs.nestjs.com/) +- [The Go documentation](https://go.dev/doc/), [The Rust Programming Language](https://doc.rust-lang.org/book/), [LiveKit](https://docs.livekit.io/) +- [OWASP Top 10](https://owasp.org/www-project-top-ten/), [NIST guide to ABAC](https://csrc.nist.gov/pubs/sp/800/162/final), [GDPR full text](https://gdpr-info.eu/) +- L. Lamport, "Time, Clocks, and the Ordering of Events in a Distributed System", 1978 +- [Supabase Auth (GoTrue)](https://github.com/supabase/auth) --- -## Known limitations - -- **No game.** All gaming-dependent modules are out of scope. -- **Expressiveness is narrow by design.** grobase has no per-application server - code; business rules are expressed as policies, schema constraints, database - triggers, or functions in the flag-gated functions plane. -- **Documented security weaknesses.** Known and declared in - [`wiki/security/03-known-weaknesses.md`](wiki/security/03-known-weaknesses.md). -- **Several features are behind flags that are off by default** β€” see the Grade - column in [Modules](#modules). -- **No interface translation in osionos.** `i18next` is declared but no - translation resource is loaded (see [Modules](#modules)). The prismatica site - itself ships English, Spanish and Arabic. -- **Whiteboard editing is loopback-only and has no per-user authorization yet** (see - [Demo](#demo)). +## ⚠️ Known limitations + +> [!CAUTION] +> **Do not enrol two-step verification on a real account.** The app has no +> sign-in prompt for the code, so the account locks itself out. + +### What does not work yet + +- **No game**, so every game module is out +- **Only the landing page is translated**: the app is English only +- **Collaboration is per block**: last write wins, not CRDT +- **The whiteboard** has no per-user authorization; its editing is + loopback-only for that reason (hydrawn is bound to `127.0.0.1:3007`) +- **Expressiveness is narrow by design**: grobase has no per-application + server code; business rules are policies, schema constraints, database + triggers, or functions in the flag-gated functions plane +- **Documented security weaknesses**: declared in + [`wiki/security/03-known-weaknesses.md`](wiki/security/03-known-weaknesses.md) + +### Bugs to fix + +- Blocked users can keep writing in an open direct message +- Port 8880 serves the API over plain HTTP where the stack runs +- The Privacy Policy promises a deletion the code never performs +- Demo passwords in `src/data/seedUsers.json` ship in the client bundle +- Page saves fail past about 330 blocks, because of a WAF limit +- 148 root commits have a one-word subject + +--- ## License -- There is no `LICENSE` file at the repository root. -- grobase is open-core under the GNU AGPL v3 (`apps/grobase/LICENSE`); - `apps/grobase/LICENSING.md` maps which parts carry which licence. -- osionos, opposite-osiris (prismatica), hydrawn, mail and calendar carry no - licence file. +Osionos Project License: copyright (c) 2026 Osionos contributors, all rights +reserved. It applies to this repository's own files; each git submodule (listed +in `.gitmodules`) keeps its own licence. Contributors and persons the copyright +holders explicitly authorize may access, inspect, run and contribute to the +software to evaluate, develop, maintain and improve the project; no other use is +granted. The full text is in [`LICENSE`](LICENSE). + +grobase (`apps/grobase`, a submodule) carries its own licence: open-core under +the GNU AGPL v3 (`apps/grobase/LICENSE`), with `apps/grobase/LICENSING.md` +mapping which parts carry which licence. monkey-bot (`vendor/monkey-bot`) is +under the MIT License. osionos, opposite-osiris, hydrawn, mail and calendar +have no licence file of their own. --- @@ -928,5 +630,5 @@ This project was built with **heavy AI assistance**, and we state it plainly. - [`wiki/security/`](wiki/security/) β€” security model, proofs, known weaknesses, network - [`DATA-MIGRATION.md`](DATA-MIGRATION.md) β€” fresh-machine and migration runbook - `apps/grobase/CLAUDE.md` β€” grobase's own authoritative technical document (inside the submodule) -- [Prismatica docs](https://prismatica-beta.vercel.app/docs/) β€” the landing's - documentation site +- [prismatica docs](https://prismatica-beta.vercel.app/docs/) β€” the public + landing's documentation site diff --git a/wiki/project/02-modules.md b/wiki/project/02-modules.md index 9661d14d..0972df15 100644 --- a/wiki/project/02-modules.md +++ b/wiki/project/02-modules.md @@ -2,7 +2,7 @@ > **Superseded Β· 2026-11-05.** This whole page is history: its rows, letters, > legend and point totals. The current grades, one rule and the points table are -> in the [README Modules table](../../README.md#modules). +> in the [README Modules table](../../README.md#-modules-and-points). > > **Status:** Superseded Β· 2026-11-05 (written as Partial Β· 2026-09-15) > **Evidence:** flag states read from `deploy/fly/boot.sh`; every gate named here exists in the diff --git a/wiki/project/ai-audit-2026-10-28.md b/wiki/project/ai-audit-2026-10-28.md index df9a89d9..434aff88 100644 --- a/wiki/project/ai-audit-2026-10-28.md +++ b/wiki/project/ai-audit-2026-10-28.md @@ -2,7 +2,7 @@ > **Superseded Β· 2026-11-04.** The pin facts below are history. groot `develop` now pins osionos > `1c1d5d84`, which ships `services/osionos-ai` (745 files) and #270. The "0 of 10 AI points" -> line no longer holds. The current grades are in the [README Modules table](../../README.md#modules), +> line no longer holds. The current grades are in the [README Modules table](../../README.md#-modules-and-points), > and the LLM interface is explained in [ai-llm-interface](ai-llm-interface.md). > > **Status:** Superseded Β· 2026-11-04 (written as Partial Β· 2026-10-28) diff --git a/wiki/project/ai-llm-interface.md b/wiki/project/ai-llm-interface.md index b4fb9ff1..53a44902 100644 --- a/wiki/project/ai-llm-interface.md +++ b/wiki/project/ai-llm-interface.md @@ -138,6 +138,6 @@ spend 15 calls. --- -See also: [README Modules](../../README.md#modules) Β· [02-modules](02-modules.md) Β· +See also: [README Modules](../../README.md#-modules-and-points) Β· [02-modules](02-modules.md) Β· [ai-stall-diagnosis-2026-10-29](ai-stall-diagnosis-2026-10-29.md) Β· [runbooks/osionos-ai](../runbooks/osionos-ai.md) diff --git a/wiki/project/contributions-method.md b/wiki/project/contributions-method.md index b2a41fba..0b183ccf 100644 --- a/wiki/project/contributions-method.md +++ b/wiki/project/contributions-method.md @@ -1,12 +1,12 @@ # How the contribution counts are made -The README's [Features List](../../README.md#features-list) and [Who committed +The README's [Features](../../README.md#-features-and-who-built-them) and [Who committed where](../../README.md#who-committed-where) count commits one way, described here with the identity joins it rests on. ## The method -Non-merge commits reachable from the commit groot `main` pins for each +Non-merge commits reachable from the commit that groot's `main` pins for each repository, read through the root [`.mailmap`](../../.mailmap), with bots and AI-agent identities left out: diff --git a/wiki/runbooks/fresh-machine.md b/wiki/runbooks/fresh-machine.md index d04b99a4..44218768 100644 --- a/wiki/runbooks/fresh-machine.md +++ b/wiki/runbooks/fresh-machine.md @@ -10,7 +10,7 @@ otherwise. |---|---|---| | Docker Engine β‰₯ 25 + Compose v2 | healthchecks use `start_interval`; measured on Docker 29.8.2 | `docker-compose.yml:3`, [README prerequisites](../../README.md#prerequisites) | | RAM β‰₯ 8 GB | `make all` peaked at 6.25 GB used on an 8 GB VM; for a VM, `VM_RAM_MB=8192` | [measured runs](#measured-results) | -| Free disk space for Docker | no minimum is measured; `fresh-bringup.sh` warns under 10 GiB free on the Docker data root, and the 2026-10-05 VM build passed with a 65 GB `/var` (91 GB disk) | `scripts/verify/fresh-bringup.sh:34,120-122`, [measured runs](#measured-results) | +| Disk β‰₯ 30 GB free for Docker | images 13.5 GB + build cache 9.5 GB (the README's 23 GB) + volumes 1.4 GB, from `docker system df` on the dev VM with the stack running, 2026-10-01 (recorded in c3a80d12). Caveat: not measured from a clean build, so a cold build may differ. `fresh-bringup.sh` warns under 10 GiB free on the Docker data root; the 2026-10-05 VM build passed with a 65 GB `/var` (91 GB disk) | [README prerequisites](../../README.md#prerequisites), `scripts/verify/fresh-bringup.sh:32-34,121-122`, [measured runs](#measured-results) | | Free ports | 443, 3001–3003, 3007, 4000, 4100, 4200, 4322, 4323, 7880, 7881, 8444, 8787, UDP 50000–50060, and Kong on 8000 | `docker-compose.yml`, `apps/grobase/orchestrators/compose/base/gateway.yml` | | One `sudo` password | `make all` copies the local CA into the system store once (`certs-trust-local`); `fresh-bringup.sh` asks for it first, with `sudo -v` | `utils/mk/certs.mk`, targets `certs-trust`, `certs-trust-local`, `scripts/verify/fresh-bringup.sh` (`check_sudo`) | | A GitHub SSH key | submodules use `git@github.com:` URLs | `.gitmodules` |