messages — Signal, Google Chat, IRC and Telegram archive

archiver/ writes all four origins into the signal MariaDB on the isis k3s cluster. The viewer (this crate and frontend/) reads them, and can reply on IRC, through the irssi that holds the connections, and on Telegram, through the feed that holds the session.

 Browser ──VPN/login──▶ messages.xinutec.org (isis, ns: signal)
                            │  Rust/axum: Nextcloud OAuth2 (identity) + sessions
                            │  + API over the archive, + sends (IRC via ssh,
                            │    Telegram and Signal through outbox tables)
                            ▼
                        signal MariaDB  ─ messages / conversations / reactions        (Signal)
                                        ├ gchat_messages / gchat_conversations…        (Google Chat)
                                        ├ irc_messages / irc_conversations             (IRC)
                                        └ telegram_messages / telegram_conversations…  (Telegram)

IRC shows only what was said: every read restricts to kind IN ('message', 'action'), leaving out joins, parts and notices, and irssi's server-notice window (is_status) is not listed.

Security model

1. Nextcloud login and allow-list, the real gate. OAuth2 identity against dash.xinutec.org; an authenticated user not in ALLOWED_USERS gets 403.
2. VPN-only by DNS. messages.xinutec.org resolves to isis's WireGuard IP. This is obscurity: the ingress also answers on the public IP.

An ingress whitelist-source-range would make the second layer real, if client source IPs survive k3s servicelb; check before relying on it.

Components

• archiver/ — the ingesters and importers, and the schema's migrations. Its README covers them.
• irclog/ — irssi autolog parsing and the irc_messages row each line becomes, used by the importer, the live tail and the viewer's send echo, so all three write the same row.
• src/ — the viewer's backend. nextcloud/identity.rs and session.rs are the login; routes/auth.rs adds the allow-list; archive/ is the origin-normalising query layer, one module per origin; irc_send.rs, telegram_send.rs and signal_send.rs send, the last two through outbox.rs; transcribe.rs is the queue recall's transcriber polls; config.rs builds the database connection from DB_*, so the app reuses signal-secret. The app owns only sessions, link_images and transcripts (src/db.rs).
• src/bin/link-fetch.rs — the link-picture fetch service, the only part that reaches the internet. It holds no credentials, database or disk.
• frontend/ — Angular: conversation list with origin filter, thread view, and a composer for IRC and Telegram (one draft per conversation; a paragraph goes to IRC line by line, to Telegram whole). The service worker keeps the app, the list, the threads opened and their pictures for reading offline (frontend/ngsw-config.json, life's policy), and the identity is remembered so it opens offline rather than at sign-in; sending offline refuses and keeps the draft. frontend/src/app/thread-window.ts keeps a window of a long thread in the DOM, and reader-place.ts owns where the reader is in it; frontend/src/app/copy-log.ts copies a selection as an irssi log (the inverse of irclog/), whether dragged over or picked by hand (a long press, then taps, as in Signal; a further long press or Shift-click picks the range); frontend/src/app/attachment.ts names attachments for both screen and clipboard. frontend/src/app/generated/ is written by ts-rs (scripts/gen-types.sh) and imported through frontend/src/app/models.ts.
• Dockerfile — xinutec/messages:latest, with both viewer binaries; archiver/Dockerfile is xinutec/signal-archiver:latest. Both build from the repository root, since the Cargo workspace spans the crates.

API (all require a session)

• GET /api/me — current user.
• GET /api/conversations — every conversation, newest activity first.
• GET /api/conversations/{origin}/{id}/avatar?v= — its picture, where the origin keeps one; v is the list's avatar version.
• GET /api/conversations/{origin}/{id}/messages?cursor=&limit=&dir=&on= — one page, oldest first. cursor is opaque, (native_ts, id), so rows sharing a timestamp are never skipped. dir is older (default), newer, or at (the cursor's own row and after, for landing). on is a day's local midnight in epoch ms; the server converts it to the origin's unit.
• GET /api/search?q=[&origin=&id=] — substring search, everywhere or in one conversation. Each hit carries a cursor that lands on it.
• POST /api/conversations/{irc,telegram,signal}/{id}/send — say something, through irssi, the Telegram feed or the Signal ingester, to a DM or group only. The body is the text and, on Telegram and Signal, reply_to, a message in the same conversation; the recipient comes from {id}. Google Chat 404s. A Signal send is archived from Signal's receipt, as the linked device never receives its own sends, and marked sent_here (src/signal_send.rs).
• GET /api/attachments/{id}, /api/gchat-attachments/{id}, /api/telegram-media/{id} — held bytes, one route per origin since attachment ids are per origin. POST /api/telegram-media/{id}/request queues an unfetched Telegram file; GET …/state reports its progress. GET /api/link-previews/{id}/image is the picture a Signal link preview carried.
• GET /api/link-images/{id}, POST /api/link-images/{id}/request — a picture for a link in a message, by the URL's hash. Only a held picture is stored; a "not a picture" or "could not reach it" answers that request and is forgotten, so the link is offered again on the next read.
• POST /api/telemetry — client events into the server log. Always 204.

recall's transcriber uses /work/v1/…, /ingest/v1/blob/… and /sync/vocabulary/prompt with its own bearer token, not a session; with no token configured they answer 404 (routes/transcribe.rs).

{origin} is signal, gchat, irc or telegram. {id} is the Signal thread_id, the Google Chat group_id, the irc_conversations.id, or Telegram's folded peer id (see v15 in archiver/src/db/migrations.rs).

Local dev

# backend (needs the database; tunnel signal-db or use a local MariaDB)
DB_HOST=127.0.0.1 DB_PORT=3306 DB_NAME=signal DB_USER=… DB_PASSWORD=… \
NC_BASE_URL=https://dash.xinutec.org NC_CLIENT_ID=… NC_CLIENT_SECRET=… \
NC_REDIRECT_URI=http://localhost:4200/auth/callback \
SESSION_SECRET=$(openssl rand -hex 32) ALLOWED_USERS=user \
  cargo run
# frontend (proxies /api, /login, /auth, /logout to :8080)
cd frontend && pnpm install && pnpm start  # http://localhost:4200

Deploy (isis, namespace signal)

Manifests are in the home monorepo (xinutec/pippijn): code/kubes/messages/k8s/ for the viewer, code/kubes/signal/k8s/ for the archiver. Push to main, wait for CI to build both images, then run code/kubes/deploy.sh messages (the viewer) or code/kubes/deploy.sh signal (the archiver) from that checkout. It refuses unless the manifests are committed and pushed and isis's checkout matches, and restarts a :latest workload only when the registry has a newer image.

One-time setup, in case it must be redone: register the OAuth2 client in Nextcloud admin (redirect URI https://messages.xinutec.org/auth/callback); put a Cloudflare Zone:DNS:Edit token in cert-manager as cloudflare-api-token and apply 00-letsencrypt-dns-issuer.yaml (isis needs DNS-01); messages → 10.100.0.2 is in code/dns; plan-run secrets (kubes README, "Secrets") makes the session key and takes the OAuth client from --values.

Tests

gate.dhall is the gate and the pre-commit hook:

nix run ../dev-lint#gate -- . gate.json

gate.json is rendered from the Dhall and committed; one check re-renders and diffs it.

• tests/archive.rs — pure units, plus end-to-end tests against a fixture in a throwaway MariaDB, whose schema the archiver's own migrations build. Those need MESSAGES_TEST_DATABASE_URL and skip without it; the gate's test row starts one via dev-lint's with-test-db.
• tests/access.rs, tests/session_cookie.rs, tests/error_responses.rs — the allow-list fails closed, sessions cannot be forged, and a 500 says nothing about itself.
• tests/api_routes.rs — requests through the real router reach the handlers behind the auth extractor. It touches no archive table, since tests/archive.rs empties the same database when it seeds.
• Frontend unit tests (pnpm test, vitest) cover logic. jsdom has no layout, fonts, or real Selection API, and does not submit forms on Enter, so pnpm run ui-check runs the Playwright suite against the production build: phone-width layout, copy, scrolling, routing, the Android keyboard and a real IME composition. Treat vitest as no evidence about the composer.
• pnpm run perf (frontend/perf/) measures rather than gates: what the thread's DOM window saves as a reader pages back, printed per depth. Run it against two builds to compare.
• archiver/tests/ — the archiver's suite, in its own database (SIGNAL_TEST_DATABASE_URL, its own gate row): its rows persist between runs, and the fixture above empties its database.

One concept, several readers

Each field below is interpreted in more than one place; add a row when a field gains another reader.

field	Rust	thread.html	copy-log.ts	search
deleted	Signal and Telegram; always false for Google Chat and IRC	hidden behind a click, body and attachments	(deleted) only, attachments included	listed, with (deleted) for the snippet
edited	Signal and Telegram, by different mechanisms; Telegram's also honours edit_hide	edited tag	(edited) on the last line	not shown
edits	Signal: revision rows via edit_of_ts, left out of pages and counts; Telegram: telegram_message_edits	behind the edited tag	not shown	Signal: a match in any version is one hit on the edited message
kind	IRC, and Telegram service events as action	* before the body	HH:MM * nick prefix	not shown
entities	Telegram entities; Signal text styles mapped to the same kinds, from an edited message's newest revision	formatted runs, overlaps combined	plain text	not shown
is_outgoing	all origins	.out class	nothing; the sender's name carries it	not shown
transcript (an attachment's)	the words, only at mean word confidence ≥ 0.75	under the player, marked "Transcribed"	[audio, transcribed: "…"]	not searched

The server sends retracted text; every reader hides it until asked. A revealed message still copies as (deleted), since the log is built from the model.

MessagesStore.title names a conversation for the list, the search results and, before the list loads, the hit's own name; the thread header says "Conversation" until then.

A message with neither body nor attachments draws an empty bubble but produces no copied line.

?at (a message's cursor) and ?on (a day) mean put me here; ?from carries a scroll position and means I was here. A landing wins on load, and the reader's first scroll replaces it with from and clears the landed-message marker. A scroll is the reader's only during their own scrolling gesture (wheel, finger, scrollbar, page keys); the thread's own scrolls, a resize and a tap or a key in the box decide nothing.

Delivery state (sent, delivered, read) appears only where the archive can say: outgoing Signal and Telegram messages sent after capture began. Telegram reports a read position and names nobody; Signal reports per-person receipts, so a group says read by 2 rather than read, and my own linked device's read sync is not counted.

Unread counts theirs after my last read, less any deleted for everyone (as the phone apps count; the list's last line still says "Message deleted"), where the archive can say: Signal from my linked devices' read syncs, Telegram from its inbox mark, Google Chat from the read position its conversation list carries (top-level messages only, as a reply inside a thread does not make a space unread there; none for the Mentions shortcut). Without a read position the count is 0, never a guess.

Sending is optional: without a usable key the app serves the archive and refuses to send on IRC. A Telegram or Signal send waits 20 seconds for the process holding the account and is withdrawn if it never took it.

Known limits

• Google Chat reactors are named only as far as gchat-archive's second capture has reached; Telegram truncates its reactor list. count stays authoritative.
• A custom-emoji Telegram reaction has no characters to draw and is left out.
• Telegram secret chats are device-local and absent.
• A Telegram message edited before the archive saw it has no earlier versions: Telegram serves only the current text.
• A Google Chat picture's bytes are held only if a harvest fetched them; the download URL needs the user's session.
• An open thread refreshes only its newest page: a deletion, edit or reaction further back shows when the conversation is next opened.
• A Telegram edit that swaps a message's photo or file keeps the old file.
• Signal attachments that failed to download before raw frames were kept (2026-09-21) cannot be fetched again: their ids were never stored.
• Signal 1:1 calls are absent: signal-cli parses no call-log sync, and a call's own signalling reaches this linked device only rarely. A group call shows once, as "started a group call", with no duration.
• signal-cli writes most sync messages from the phone as {}: only sent messages, reads, blocks and contact or group syncs come through. Views, view-once opens and message-request answers are parsed inside signal-cli but never sent on; the call log is not parsed at all. A message the archive cannot draw reads "a message this archive can't show".
• A drag selection reaches only the rendered window (400 messages); a select-all of a longer conversation ends with --- copied 400 of N messages; the rest were not loaded on screen. Messages picked by hand stay picked when they scroll out of the window, but only loaded pages can be picked from.