Skip to content

Route channel traffic into source inbox #422

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
alex-clickhouse wants to merge 10 commits into from
Open

Route channel traffic into source inbox #422

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
e185de7
Feed the source inbox from what a channel sees but does not answer
alex-clickhouse Sep 2, 2026
bf258f9
Advance the drain past a row it cannot read, and honour the spool cap
alex-clickhouse Sep 2, 2026
dc7470f
Close the observation gate's fail-open paths
alex-clickhouse Sep 2, 2026
58e3d6f
Call it a source, not an observation, and a buffer, not a spool
alex-clickhouse Sep 3, 2026
7ee58bb
Name the thing being filtered, and accept a pasted #channel
alex-clickhouse Sep 3, 2026
328ebcb
Give Telegram its own docs section, and move the reasoning to sources.md
alex-clickhouse Sep 3, 2026
7d58be8
Decide the channel and source routes independently
alex-clickhouse Sep 3, 2026
eaf6b40
Finish channel-source routing and pairing guards
alex-clickhouse Sep 3, 2026
36601c4
Stop reading a name as an id, and drain past one batch
alex-clickhouse Sep 3, 2026
c3872b4
Settle the source gate against real Slack traffic
alex-clickhouse Sep 3, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 47 additions & 4 deletions config.example.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -100,15 +100,30 @@ gateway:
# Channels
telegram:
enabled: true
# DM authorization:
# Live-message authorization in private and group chats:
# pairing — only users in allowed_users may talk to the bot. New users
# authorize with a one-time code: run `nerve pair` on the
# server, then send the bot `/pair <code>`. Paired IDs are
# persisted to config.local.yaml automatically.
# server, then send the bot `/pair <code>` in a private chat.
# Paired IDs are persisted to config.local.yaml automatically.
# open — anyone can talk to the bot (dangerous: full agent access).
dm_policy: pairing
# allowed_users: [123456789] # numeric Telegram user IDs (or pair instead)
# allowed_users: [123456789] # users allowed to use the bot (or pair instead)
stream_mode: partial # "partial" (edit messages) or "full" (wait for complete)
#
# Feed the source inbox from watched group chats, reaching it as the source
# "telegram:observed". This grant is independent of allowed_users: a sender
# who cannot use the bot may still reach the source. A live-routed message is
# not collected unless include_handled_messages is true. Only numeric IDs may
# grant; titles and @usernames can only deny. Private chats, other bots, and
# /pair commands are never collected. With privacy mode enabled, a non-admin
# bot does not receive ordinary group messages. Make it an admin or disable
# privacy mode with BotFather.
# See docs/sources.md before enabling.
# source:
# enabled: true
# allow_chats: ["-1001234567890"]
# deny_senders: ["12345"]
# include_handled_messages: false # true also collects live-routed messages

slack:
# Omitting this key leaves Slack off until both tokens below are set, so an
Expand Down Expand Up @@ -152,6 +167,34 @@ slack:
# [all] — everything, including doctor and restart
# doctor/restart affect the host; sessions lists other channels. Opt in.
# commands: [sessions, new, stop, reply]
#
# Feed the source inbox from watched channels, reaching it as the source
# "slack:observed", where poll_source and the `messages` cron gate read it.
# Nothing here starts an agent turn.
#
# A SEPARATE grant from the access rules above, decided independently of
# them: those say who may drive the agent, this says whose traffic may reach
# its inbox. So an empty allow_channels collects NOTHING rather than
# everything. A mention from someone access refuses is still collected; a
# live-routed message is not collected unless include_handled_messages is true.
# DMs (group DMs included), other apps, and the bot's own posts are never
# collected.
#
# allow_channels takes a channel ID or a channel name, with globs and a
# leading "#" tolerated; deny_channels the same. allow_senders takes a
# member ID, handle, or email — a display name can only ever deny, since
# its owner picks it. Prefer uppercase IDs: everything else uses cached Slack
# API lookups.
#
# Everything collected is untrusted input: the grant decides whose messages
# are kept, not whether their contents can be believed. See docs/sources.md.
# source:
# enabled: true
# allow_channels: ["C0456DEF", "eng-*"]
# deny_channels: ["*-social"]
# deny_senders: ["*-bot"]
# include_handled_messages: false # true also collects live-routed messages
# schedule: "*/5 * * * *" # how often the buffer drains into the inbox

# Where notify, ask_user, and propose_action deliver. The list replaces the
# default rather than adding to it, so name every transport you want. A
Expand Down
139 changes: 132 additions & 7 deletions docs/config.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1052,7 +1052,7 @@ carry text. A `.png` or `.ico` has to be committed by a human.
| `telegram.enabled` | bool | `true` | Enable Telegram bot |
| `telegram.bot_token` | string | - | Bot token from @BotFather |
| `telegram.dm_policy` | string | `pairing` | `pairing` (allowlist + one-time pairing codes) or `open` (anyone — dangerous) |
| `telegram.allowed_users` | list[int] | `[]` | Telegram user IDs allowed to DM the bot |
| `telegram.allowed_users` | list[int] | `[]` | User IDs allowed to use the bot in private and group chats |
| `telegram.stream_mode` | string | `partial` | `partial` (edit msgs) or `full` |

### Pairing
Expand All @@ -1064,13 +1064,72 @@ editing config files:
1. Run `nerve pair` on the server — it prints a one-time 6-digit code
(valid 1 hour). On a fresh install with no `allowed_users`, a code is
also generated automatically at startup and printed to the log.
2. Send the bot `/pair <code>` from the Telegram account to authorize.
2. In a private chat, send the bot `/pair <code>` from the Telegram account to
authorize.
3. The user ID is appended to `telegram.allowed_users` in
`config.local.yaml` and takes effect immediately.

An unauthorized `/start` gets a reply with the sender's numeric ID and
pairing instructions (rate-limited); all other messages from unauthorized
users are ignored.
An unauthorized `/start` gets a reply with the sender's numeric ID and pairing
instructions (rate-limited). Other unauthorized messages cannot start a live
channel turn. Matching group messages may still go to the independent channel
source.

### Channel source

Group traffic can feed the source inbox, where `poll_source`, `read_source`,
and the `messages` cron gate read it. Nothing here starts an agent turn. It
is a grant of its own, independent of `allowed_users`: see
[sources.md](sources.md#chat-channels-slack-telegram) for the routing rules
and the threat model.

```yaml
telegram:
source:
enabled: true
allow_chats: ["-1001234567890"]
deny_senders: ["12345"]
schedule: "*/5 * * * *"
```

| Key | Type | Default | Description |
|-----|------|---------|-------------|
| `telegram.source.enabled` | bool | `false` | Feed the inbox from watched chats |
| `telegram.source.allow_chats` | list[str] | `[]` | Chats to collect. **Empty means none** |
| `telegram.source.deny_chats` | list[str] | `[]` | Never collect these |
| `telegram.source.allow_senders` | list[str] | `[]` | Restrict to these senders |
| `telegram.source.deny_senders` | list[str] | `[]` | Skip these senders |
| `telegram.source.include_handled_messages` | bool | `false` | Also collect messages accepted for live routing |
| `telegram.source.schedule` | cron/interval | `*/5 * * * *` | Drain cadence |
| `telegram.source.batch_size` | int | `50` | Records per drain |
| `telegram.source.condense` | bool | `false` | LLM-condense long messages |
| `telegram.source.max_stored_messages` | int | `10000` | Buffer trim target for all Telegram chats together; checked every 100 writes |

Reaches the inbox as the source `telegram:observed` — distinct from the
`telegram` sync source, which is the Telethon pull from your *user* account.
The keys say *chats* rather than *channels* because on Telegram a channel is
a specific entity type distinct from a group.

**Setup: the bot must receive group messages.** With privacy mode enabled, a
non-admin bot does not receive ordinary group messages. Make it an admin or
disable privacy mode with BotFather (`/setprivacy` → Disable). See Telegram's
[privacy-mode documentation](https://core.telegram.org/bots/faq#what-messages-will-my-bot-get).

**What the lists match.** Only numeric IDs may grant.

| List | Matches | Example |
|---|---|---|
| `allow_chats` | numeric chat ID **only** | `-1001234567890` |
| `deny_chats` | chat ID, `@username`, or title | `-1001234567890`, `ops-room` |
| `allow_senders` | numeric user ID **only** | `42` |
| `deny_senders` | user ID, `@username`, or profile name | `42`, `mallory` |

A group's title is set by whoever runs it and a `@username` is claimable and
movable, so treating either as grantable would let anyone create a group
named `ops-room` and walk into a grant meant for someone else's. Both stay
deny-eligible, where a spoofable name can only subtract access.

**Never collected:** private chats, the bot's own messages, other bots, and
`/pair` commands.

## Slack

Expand Down Expand Up @@ -1194,6 +1253,65 @@ slack:
Deny rules alone never enable access.
- If a required name lookup fails or omits data, Nerve refuses the message.

### Channel source

Channel traffic can feed the source inbox, where `poll_source`, `read_source`,
and the `messages` cron gate read it. Nothing here starts an agent turn. It is
a grant of its own, independent of the access rules above: see
[sources.md](sources.md#chat-channels-slack-telegram) for the routing rules
and the threat model.

```yaml
slack:
source:
enabled: true
allow_channels: ["C0123ABCD", "eng-*"]
deny_channels: ["*-social"]
deny_senders: ["*-bot"]
schedule: "*/5 * * * *" # how often the buffer drains into the inbox
batch_size: 50
max_stored_messages: 10000 # all Slack channels together
```

| Key | Type | Default | Description |
|-----|------|---------|-------------|
| `slack.source.enabled` | bool | `false` | Feed the inbox from watched channels |
| `slack.source.allow_channels` | list[str] | `[]` | Channels to collect. **Empty means none** |
| `slack.source.deny_channels` | list[str] | `[]` | Never collect these |
| `slack.source.allow_senders` | list[str] | `[]` | Restrict to these senders |
| `slack.source.deny_senders` | list[str] | `[]` | Skip these senders |
| `slack.source.include_handled_messages` | bool | `false` | Also collect messages accepted for live routing |
| `slack.source.schedule` | cron/interval | `*/5 * * * *` | Drain cadence |
| `slack.source.batch_size` | int | `50` | Records per drain |
| `slack.source.condense` | bool | `false` | LLM-condense long messages |
| `slack.source.max_stored_messages` | int | `10000` | Buffer trim target for all Slack channels together; checked every 100 writes |

Reaches the inbox as the source `slack:observed`.

**What the lists match.** Case-insensitive, `*` / `?` globs, and a leading `#`
or `@` is stripped so a name pasted from Slack works as-is. Deny always wins;
a non-empty allow list must match.

| List | Matches | Example |
|---|---|---|
| `allow_channels` / `deny_channels` | channel ID, or channel name | `C0123ABCD`, `eng-backend`, `eng-*` |
| `allow_senders` | member ID, handle, or email | `U0456DEFG`, `alice`, `*@example.com` |
| `deny_senders` | same, **plus** display and real names | `*-bot`, `Alice Smith` |

A display name is edited by its owner, so it can only ever deny — the handle
and email are workspace-assigned, so they may grant.

**Never collected:** DMs, group DMs, the bot's own posts, other apps, and
join/leave noise. A group DM arrives as `channel_type="mpim"` on a `G` id, and
a `G` whose type cannot be established is skipped rather than guessed at.

**Prefer IDs for a busy channel.** Names and globs use cached Slack API lookups
(`conversations.info` or `users.info`). An ID skips the lookup, but only in
uppercase: `beckyjones` is a legal handle and `c0456def` a legal channel name,
so a lowercase pattern is resolved rather than assumed. It still matches. It
just costs the lookup. A channel the grant does not name is refused before any
sender is resolved.

### Message behavior

- In an allowed DM, the bot answers every message.
Expand Down Expand Up @@ -1289,9 +1407,16 @@ Three differences from the inbound policy:

## Sources (sync)

Sources pull data from external services on a schedule. See [sources.md](sources.md) for full details.
Most sources pull data from external services on a schedule. See
[sources.md](sources.md) for full details.

Chat channels also feed the inbox, but are configured on the channel — see
`slack.source.*` and `telegram.source.*` above. For each message delivered by
the channel transport, matching messages are written to a local buffer.
`ChannelSource` drains that buffer; it does not poll the chat service. Its
`schedule` is a drain cadence, not a poll interval.

**Common fields** (available on all sources):
**Common pull-source fields** (under `sync.<source>`; availability varies):

| Key | Type | Default | Description |
|-----|------|---------|-------------|
Expand Down
Loading
Loading