Skip to content

feat: record soundboard sounds and emoji reactions in the office - #16

Merged
ChristopherPHolder merged 3 commits into
mainfrom
feat/record-voice-effects
Oct 15, 2026
Merged

ChristopherPHolder merged 3 commits into
mainfrom
feat/record-voice-effects

Conversation

@ChristopherPHolder

@ChristopherPHolder ChristopherPHolder commented Oct 8, 2026 •

Copy link
Copy Markdown
Collaborator

What and why

Records the soundboard sounds and emoji reactions people send in the office voice channel, and lets banter see them.

Discord only sends these (VOICE_CHANNEL_EFFECT_SEND) to clients that are in the voice channel. Nothing in the REST API or the audit log has them, so the bot now sits in the office while anyone is there:

  • It joins when the office opens and leaves when it empties, already self-muted and self-deafened. It only sends the gateway's voice-state message and never opens an audio connection, so it can't hear or send anything. At startup it joins if people are already in, and otherwise leaves, in case an earlier run left it behind.
  • Each effect in the office goes to a new office.voice_effects table through the same buffered recorder: sound ID, name and volume, emoji and animation. Names come from the server's sounds (kept up to date with the non-privileged GuildExpressions intent) and Discord's default sounds (fetched once at startup). Bots are left out.
  • A new office_activity view adds PlayedSound <name>, Reacted <emoji> and SentEffect events to voice_activity. Banter now reads that view, and the prompt explains the new events.
  • The gateway drops every update and effect from the bot's own user ID, so it's never announced or recorded, even when Discord leaves out the member.
  • A member's display name is no longer overwritten with their user ID when Discord sends something without the member (this also affected voice updates before).
  • It logs a warning at startup if it lacks the Connect permission on the office channel.

Migration 0003_record_voice_effects only adds a table, indexes and a view.

Before merging: give the bot the Connect permission on the office channel. Without it, everything else keeps working, but nothing gets recorded and a warning is logged.

Known limits:

  • Whenever the bot's Discord connection is ready or resumed, it goes back to where it should be. If someone kicks it from voice by hand, it rejoins the next time the office opens.
  • I haven't checked against the real Discord yet whether effects still arrive while the bot is self-deafened. Watch for rows in voice_effects after the deploy.

How I tested it

  • Program tests for when the bot moves: it leaves an empty office at startup, joins when the office opens, leaves when it empties, and joins straight away when people are already in. Also that effects are recorded only from the office and never from bots.
  • PGlite tests for the new table and view: sounds, reactions and other effects show up next to voice activity in the banter history, and a repeated write is stored once.
  • Recorder tests: effects are written in order with everything else.
  • Ran scripts/try-banter.ts without --call-ai: the camera-party scenario now includes soundboard lines, and the prompt renders them.
  • Not tried against a real Discord server.

Checklist

  • pnpm typecheck, pnpm lint, pnpm test and pnpm build pass
  • Tests cover the change
  • README updated if what gets posted, the configuration or the deploy changed
  • New environment variables are wired through (none added)
  • Ready to ship: merging this deploys it to production (needs the Connect permission first)

Discord only sends soundboard sounds and emoji reactions
(VOICE_CHANNEL_EFFECT_SEND) to clients in the voice channel, so the bot
now sits in the office while anyone is there.

- It joins when the office opens and leaves when it empties, already
  self-muted and self-deafened. It only sends the gateway's voice-state
  message and never opens an audio connection. At startup it joins if
  people are already in, and otherwise leaves, in case an earlier run
  left it behind.
- Each effect in the office goes to a new office.voice_effects table
  through the same buffered recorder: sound ID, name and volume, emoji
  and animation. Names come from the server's sounds, kept up to date
  with the GuildExpressions intent, and Discord's default sounds,
  fetched once at startup. Bots are left out.
- A new office_activity view adds PlayedSound, Reacted and SentEffect
  events to voice_activity, and banter now reads it, so the model sees
  lines like "Alan PlayedSound airhorn".
- The bot's own voice state is always treated as a bot, even when
  Discord leaves out the member, so it never opens the office.
- It logs a warning at startup if it lacks the Connect permission on
  the office channel.
The gateway now drops every voice-state update and voice effect from
the bot's own user ID before anything else sees them. Before, the bot
was only left out by the general bot check, which relied on Discord
sending the member with the update.
- The bot remembers whether it should be in the office and goes back
  there whenever its Discord connection is ready or resumed. Before, a
  fresh gateway session dropped it from voice until the office next
  opened, and a move tried while reconnecting was lost.
- A member's display name is no longer overwritten with their user ID
  when Discord sends an update or effect without the member.
@ChristopherPHolder
ChristopherPHolder merged commit 92ef180 into main Oct 15, 2026
4 checks passed
@ChristopherPHolder
ChristopherPHolder deleted the feat/record-voice-effects branch October 15, 2026 13:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant