Skip to content

docs: voice commands go to a resident session that speaks as the buddy - #133

Draft
longzhi wants to merge 55 commits into
mainfrom
claude/web-body-voice-assistant-7f44bc
Draft

longzhi wants to merge 55 commits into
mainfrom
claude/web-body-voice-assistant-7f44bc

Conversation

@longzhi

@longzhi longzhi commented Oct 12, 2026 •

Copy link
Copy Markdown
Collaborator

Voice commands (#132): the design, and the resident session behind them, usable from the CLI before the box can hear. Draft until accounts and the microphone are settled.

Docs

  • ADR-0015 (proposed): voice goes to the cloud for recognition, then to one long-running resident Codex session driven over the user's own codex app-server, which answers with a spoken reply in the picked Character's voice and persona. Records the rejected options (dictation, a session per command, the top task card, a cloud LLM brain, Muse's cloud, Claude Code Channels, ACP, recognition on the computer, a separate Agent voice).
  • CONTEXT.md: voice command, resident session, spoken reply; the buddy speaks in two ways; the persona rule narrows to lines; the Adapter becomes two-way.
  • ADR-0007 amendment: K0 long would become push-to-talk.

Resident session (experimental, off unless VIBEBUDDY_RESIDENT_SESSION=1)

  • daemon/src/app_server.rs, daemon/src/resident.rs: JSON-RPC client for codex app-server over stdio; one thread for good (thread/start once, thread/resume after, a new thread with a warning if resume fails); read-only sandbox, every approval declined; client-defined tools speak and list_activities; the user's MCP servers, plugins and built-in tools turned off with -c overrides.
  • The resident session's own Codex hooks are dropped before anything records them: no task cards, announcements or stats.
  • POST /v1/resident/say, GET /v1/resident, and a new vibebuddy CLI: vibebuddy say "<text>", vibebuddy resident (prints codex://threads/<id>).
  • docs/resident-session.md, including a privacy note: list_activities sends task card titles and project names to the user's Codex model provider.
  • No App UI changes, so nothing to mirror between the Mac and Linux apps yet.

Tested: cargo test --workspace and clippy pass; end to end on a Mac with codex-cli 0.162.0 against an isolated daemon (memory kept across an app-server restart, file writes refused, MCP tools unavailable).

🤖 Generated with Claude Code

ADR-0015 (proposed) records the voice design from the discussion in #132:
the cloud only hears and speaks, one resident Agent session thinks and
answers in the Character's voice and persona. CONTEXT.md gains voice
command, resident session and spoken reply, and the Adapter becomes
two-way. ADR-0007 notes that K0 long would become push-to-talk.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings October 12, 2026 14:57

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@longzhi
longzhi marked this pull request as draft October 12, 2026 15:07
longzhi and others added 25 commits October 13, 2026 15:03
Claude Code waits on Anthropic's terms for subscription login; ACP and
Channels are recorded as rejected. Records the 2026-10-13 probe of a
resident thread driven over stdio with a client-defined speak tool.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
No firmware of ours has used the box's microphone. The I2C0 scans found no
ES7210 at 0x40-0x43, so the mic is most likely on the ES8311's own ADC,
which our ES8311 sequence (esp_codec_dev's WORK_MODE_BOTH) already powers,
with samples on GPIO47 as in xiaozhi 1.9.4's atk-dnesp32s3-box config.

- core: device.mic.record (cable only) reports chip IDs at 0x18 and
  0x40-0x43, sets the ES8311 mic gain (register 0x16, as
  es8311_set_mic_gain), records, then sends per-100 ms RMS/peak levels for
  both slots and the samples as base64 lines, one per poll, with a CRC.
- device: I2S0's receiver shares the transmitter's BCLK/WS (signal
  loopback) and reads GPIO47 into PSRAM only while a capture runs; 24 kHz
  16-bit stereo, since the ES8311 has one LRCK for ADC and DAC.
- tools/mic-record.py writes a WAV and says whether a slot is live.
- docs/firmware-bringup.md: facts table with sources, how to run and read it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
vibebuddyd can now keep one long-running Codex thread for voice commands
(ADR-0015), behind VIBEBUDDY_RESIDENT_SESSION=1. It spawns the user's own
codex app-server lazily, checks the existing login, starts the thread once
(read-only, approvals on request, speak and list_activities as dynamic tools)
and resumes it on every later start. Approvals and questions are declined;
turns run one at a time and later ones queue.

The resident thread's own Codex hooks are dropped by thread id or working
directory, so it never becomes an Activity.

A new vibebuddy CLI sends a typed voice command with 'vibebuddy say' and
prints the thread's codex:// link with 'vibebuddy resident'.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The first mic build boot-looped on a stack guard panic while drawing the
look at boot. esp-hal's main stack is the internal DRAM left between the
end of .bss and the reclaimed heap; the static 24 KB RX ring and the
task's 2.4 KB local buffer took it from 47,784 to 21,032 bytes.

The ring (12 KB, 4 descriptors) is now allocated from internal heap only
while a capture runs and freed after, and the task copies straight from
the ring into the PSRAM capture, so its pool is 184 bytes. The stack is
47,576 bytes, 208 below main. A lesson records how to check this.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The resident session's codex app-server now runs with the user's plugins,
apps, MCP servers and Codex's own tool features (shell, exec, sub-agents,
web search, browser, computer use and more) switched off through -c
overrides. MCP servers are turned off one by one, named by 'codex mcp list
--json', since an override can't empty mcp_servers. The user's AGENTS.md and
hooks stay.

A thread that can't be resumed is replaced, and the log now names both the
old and the new thread id. The docs gain a privacy note on list_activities
and say the box should speak a reply as soon as speak arrives.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On the box the first capture ended in MIC ERROR STALLED with nothing to
say why. The 12 KB ring of four descriptors had to be relinked by the mic
task before the DMA reached its end, and the task only runs when the main
loop yields; once the DMA hits the end of the chain it stops for good.

The DMA now writes the whole recording into one PSRAM buffer (esp-hal's
DmaRxBuf handles the cache), so nothing depends on the main loop. Its
descriptors (~1 KB for 5 s) come from the internal heap for the capture.

Each capture also prints MIC DEBUG lines at 100 ms and at the end:
descriptors filled, I2S0 and GDMA channel 1 registers, and how often
GPIO47/21/13 changed level, so the next flash says where it stops.

Main stack: 47,472 bytes (main: 47,784).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On the box all 71 descriptors filled and the capture still came back
MIC ERROR SHORT. The I2S receiver signals an EOF every 4095 samples
(esp-hal leaves RXEOF_NUM at its 12-bit maximum, 8190 bytes), and each
EOF closes the descriptor it lands in, so about every third 4080-byte
descriptor holds 30 bytes and the buffer held about two thirds of the
samples. GDMA IN_INT_RAW 0x13 (suc_eof set) fits this.

The buffer is now 1.5x the capture plus two descriptors, and the capture
ends once the handed-back descriptors hold enough bytes. MIC DEBUG adds
the byte count and the first six descriptor lengths, so the next flash
shows whether the 4080 4080 30 pattern is the cause. The BCLK/WS level
counts are gone: those pins are outputs with the input path off, so the
input register reads 0 however fast they run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On the box the descriptors came back as 4080, 14, 4080, 14: the I2S EOF
cuts one every 4094 bytes, so a capture needs about twice its size in
descriptors. 2.5 times leaves room.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- docs/hardware.md: box 30:ED:A0:A4:0D:08 recorded 5 s on 2026-10-13. The
  mic is the ES8311's ADC, samples on GPIO47, left slot; no ES7210; about
  -31 dBFS for a close voice at 30 dB gain.
- device mic.rs and docs: RXEOF_NUM (0xFFE from esp-hal) counts bytes on
  the ESP32-S3, as ESP-IDF uses it, hence descriptors of 4080, 14, ...;
  the buffer stays at 2.5x, since lining EOFs up would need a raw register
  write racing esp-hal's that hasn't been tried on a box.
- tools/mic-record.py judges slots from the samples with DC removed and
  the first 100 ms dropped, needs a window above -70 dBFS to call one
  LIVE, and bases the overall verdict on the left slot.
- LESSONS.md: count received data by descriptor lengths, and make a
  bring-up diagnostic report its own state from the first build.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…-assistant-7f44bc

# Conflicts:
#	daemon/src/main.rs
main took 0015 (Noise) and 0016 (resumable transfers) meanwhile.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e-agent-a94da637ae9c47c0b

# Conflicts:
#	firmware-rs/core/src/lib.rs
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The computer side of docs/protocol.md "Voice commands (experimental)":

- VOICE START/DATA/END/CANCEL lines are parsed and each DATA line is
  decoded on its own from the IMA ADPCM state it carries, with a decoder
  checked nibble for nibble against firmware-core's. Lost lines become
  silence and are logged; DATA lines stay out of the log.
- voice.available goes out on every build report (after hello on each
  new link, and after a reboot) when the resident session is on and a
  speech recognizer is configured; never over the Relay.
- Speech recognition sits behind a Recognizer trait with two backends:
  the user's own OpenAI-compatible endpoint, which wins, and Cloudflare
  Workers AI's Whisper for development. Settings come from the
  environment, speech.json in the config directory (kept out of
  config.json so an older App's PUT can't wipe the key), or a dev env
  file. The language is detected unless set.
- On VOICE END: WAV -> recognizer -> voice.heard -> resident session ->
  voice.reply the moment speak is called -> voice.done, cut to fit the
  line; errors not_heard, unavailable and busy, one turn at a time.
  Resident::say_streaming hands replies over as they come; say and
  POST /v1/resident/say still collect them.

An ignored test feeds a real recording through the real recognizer and
resident session.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
While the computer offers voice (voice.available), a long K0 records until
it is let go, up to 30 s, and streams the audio as VOICE START/DATA/END/CANCEL
lines (docs/protocol.md). Without voice, K0 long still gives up the phase.

- core/voice_command.rs: the turn (24 kHz IMA ADPCM, 1152 samples a line,
  each line carrying the coder state it starts from; under 0.5 s is
  cancelled) and the overlay: LISTENING with a level meter and seconds left,
  THINKING, then what was heard and the replies, wrapped; errors; closes 15 s
  after voice.done, after a quiet minute, or on K2. Needs-input, failed and
  a phase ending close it and cancel a recording, as with the menu.
- adpcm.rs: an Encoder matching the decoder, with round-trip tests.
- buttons.rs: K0LongEnd, K0 let go after a long press.
- device/mic.rs: streaming through a circular DMA ring in PSRAM (128
  descriptors, owner checks off, so it never stops), drained by the mic task
  every 10 ms into a heap queue; MIC STREAM reports rate, drops and backlog.
  Nothing new in static RAM: main stack 42,128 bytes vs 42,656 on main.
- VOICE DATA goes over native USB and the local network only (Board::
  write_stream): the UART's 11.5 KB/s can't carry 16 KB/s and blocks.
- Microphone gain 42 dB while recording a voice command.
- Comma and apostrophe glyphs; the font still has no CJK.
- voice_preview example and docs images; docs updated (protocol, device
  menu, pomodoro, firmware bring-up, ADR-0007 amendment).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…voice-assistant-7f44bc

# Conflicts:
#	docs/protocol.md
The box sends no VOICE DATA over the CH343 bridge, so voice.available
is no longer sent there; the log says why voice stayed off (bridge or
Relay). A turn that starts while another is still being answered gets
voice.error busy right after VOICE START, as docs/protocol.md now allows,
so the box stops recording; the rest of that turn is dropped.

The voice tests now make their VOICE lines with firmware-core's own
VoiceCommand, so they check the daemon against what the box writes:
1152-sample lines, an even END count that includes lost lines, and gaps
filled to keep later audio in place.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
longzhi and others added 29 commits October 13, 2026 18:24
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each spoken reply is synthesized by the voice service
(POST {base}/v0/voice/speak, staging outside releases,
VIBEBUDDY_TTS_BASE_URL to set or switch off) behind a Synthesizer
trait, and its MP3 is forwarded to the box as it arrives, in pieces of
at most 768 base64 characters, then an end; a failed or cut-short
stream ends early with "error":"synthesis". Replies are spoken in order
on their own task so the text never waits for audio, and voice.done
follows the last reply's end. Replies over 1000 characters are split at
sentences. A dev-only table gives each Character an Edge voice, with a
Chinese voice for Chinese text.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The screen only had the 5x7 ASCII font, so voice transcripts, spoken
replies and Chinese session titles showed as placeholders.

font.rs adds 3,678 glyphs from Fusion Pixel Font 12 px (OFL 1.1): the
3,500 characters of level one of the 2013 Table of General Standard
Chinese Characters, CJK punctuation and the full-width forms. Ark Pixel
alone lacks over a hundred level-one characters at 12 px; Fusion fills
them from Cubic 11. tools/make-cjk-font.py builds the table from pinned
downloads (the font and Unihan's kTGH field): a sorted u16 index and one
packed 12x12 bitmap per glyph, 73,562 bytes, included as read-only data
so it stays in flash. The main stack is unchanged at 42,128 bytes.

Canvas::draw_str draws UTF-8 with ASCII in the old font and Chinese
centred on the same line, cuts at whole characters, and draws a dotted
box for anything the table lacks. font::wrap breaks between Chinese
characters, keeps ASCII words whole and hangs closing punctuation. Task
card titles, projects, the title line and the voice overlay use it;
text with Chinese goes to small type in the overlay. English screens are
unchanged: compare-display.sh finds all 826 frames identical, and the
voice and menu previews match byte for byte.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
voice.audio chunks for the turn the box recorded are base64-decoded and
handed to a ReplyPlayer (core spoken_reply.rs), which the audio task pulls
from whenever no announcement is queued: same I2S stream, same volume.
Each reply is buffered on the heap and decoded frame by frame with
nanomp3-core; the 22 KB decoder is built zeroed on the heap when a reply
starts and freed when nothing is left. A reply starts with 300 ms
buffered, goes quiet (not stutter) when it runs dry, and replies play in
order; a gap or out-of-order chunk drops that reply. An end with
"error":"synthesis" plays what came and logs VOICE AUDIO CUT SHORT.

Announcements and the Pomodoro chime win: the reply stops and the rest of
its turn is ignored. So do mute, a new K0 push-to-talk, K2 and the
overlay closing; the overlay stays up while a reply is still speaking.

nanomp3-core is vendored with l3_reorder marked #[inline(never)]: inlined,
LSR leaves an address the esp Xtensa backend can't select ("Cannot
select: i32 = Constant<4608>") at opt-level s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The daemon queued a whole reply's audio at once and the link's 64-frame
queue dropped most of it, so the box stopped mid-sentence: wait for room
instead. THINKING now gains dots and counts seconds, since recognition
and the resident session take ten seconds or more.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On the box a reply stopped after its first word: one voice.audio line
arrived as invalid JSON and the gap dropped the rest of the reply. UART
and USB fed one line parser, so a stray byte on the bridge could split a
line coming over USB; UART lines are now held until whole. A missing
chunk is skipped instead of ending the reply: the MP3 decoder finds the
next frame.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Codex App takes a thread's writer lock when the user opens it, and
the daemon then started a second resident thread. Replace the thread
only when its rollout is gone; otherwise fail the command and try again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The ChatGPT App's own app-server takes the writer lock of every thread
under ~/.codex it lists and keeps it, so `thread/resume` of the resident
thread failed with "already has an active writer" while the App ran.

The app-server now runs with CODEX_HOME=<state dir>/resident-codex (0700),
which nothing else lists:
- Login: resident-codex/auth.json is a symlink to the user's auth.json,
  re-made on every start. Codex 0.162.0 reads it with a plain open,
  reloads before refreshing, and saves by truncating in place, so one file
  stays shared. cli_auth_credentials_store="file" is pinned; a keychain
  login can't be shared (its entry is keyed by the Codex home's path).
- Memories, read-only: features.memories stays off, since it also starts
  the memory pipeline, which rewrites the memories folder of the home it
  runs in. The daemon puts the user's memory_summary.md in the developer
  instructions and offers a read-only search_memories tool over MEMORY.md,
  raw_memories.md and rollout_summaries/.
- The old thread is not found in the new home, so one new thread starts
  (logged), as before for a lost rollout.
- `vibebuddy resident [n]` prints the last n exchanges, read with
  thread/turns/list (GET /v1/resident/history), instead of a codex://
  link the Codex App can no longer open.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The resident session runs on gpt-6-luna at low reasoning effort, starts
with the daemon instead of on the first command, and speech synthesis
reuses one HTTPS agent.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Conversation tab, on the Mac and on Linux, while the daemon runs the
resident session: the last 20 voice commands and the buddy's spoken
replies, newest at the bottom, read again when the tab is shown and on
Refresh, with empty and error states. Both Apps ask GET /v1/resident
each time the daemon comes up and hide the tab while it is off.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Asked about tomorrow's weather, the resident session invented a forecast;
asked what just finished, list_activities had nothing, as the tracker
drops an Activity when it ends.

- The instructions forbid stating anything not seen in a tool result, a
  search result or the memories; for things about now it searches, and
  otherwise says plainly it can't find out.
- Codex's built-in web search is on (web_search="live"). Measured on a
  daemon of its own: gpt-6-luna at low answered weather, news and price
  questions from what it found, first speak in 5.6 to 7.6 s, against 11
  to 26 s on gpt-6.1-sol at low, so luna stays. MODEL and
  REASONING_EFFORT are constants.
- list_activities returns {activities, recently_ended, today}: the last
  20 ended Activities (agent, title, project, done/failed/stopped/expired,
  seconds ago), in memory only, and today's stats.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Another session's repeated needs-input closed the voice overlay a second
after the transcript appeared. While the overlay is up, needs input,
failed and done no longer take the screen or cut a reply short: the
overlay marks the first two in yellow, and the worst of what came is
announced once when the overlay closes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The voice service moved from Edge, which now refuses it, to ElevenLabs
with voices named <vendor>:<id>. The daemon takes each Character's voice
from its voice.env, so replies sound like its lines; the robot gets
Jessica, and the Doubao Characters borrow Mei until Doubao is spoken.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Xiaohe's voice is Doubao; she spoke with Mei's voice, which made her
sound like another Character. ElevenLabs' stock Jessica is used instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
voice.env gains GENDER. A Character whose voice isn't in ElevenLabs gets
the stock Sarah or George, voices of no Character; the robot keeps
Jessica.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GET /v1/resident now carries an in-memory `revision`, bumped when a
resident turn ends (voice or `say`) or the thread changes, plus `ready`
and `problem` (codex_missing / codex_logged_out). Readiness is checked
without starting the app-server or a turn: codex is looked up as a start
does and `codex login status` runs in the resident Codex home; the answer
stands for 30 s, and every app-server start counts as a check too.

The box is offered voice only while the session is on and ready and a
recognizer is configured; when readiness flips the daemon tells a booted
box on USB or the network right away, so K0 long falls back to giving up
the Pomodoro phase.

Both Apps poll GET /v1/resident every 2 s while the Conversation tab is
visible, re-read the history only when `revision` moved (every 5 s for an
older daemon), and stop when the tab or window is left. When voice isn't
ready, the tab says what's missing: install the Codex CLI (link to
OpenAI's page) or run `codex login`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A second brain for the resident session (ADR-0017): the user's own `claude` CLI, headless, authenticated only
with the user's Anthropic API key. Which brain is a setting: codex, claude, or auto (Codex when it is ready,
else Claude Code when a key is set); each keeps its own session for good.

- One long-lived `claude -p` with stream-json in and out, in a config dir of its own (resident-claude/, 0700),
  started once with --session-id and resumed with --resume; replaced only when its transcript is gone.
- The child loses every CLAUDE*/ANTHROPIC* variable of the daemon's and gets ANTHROPIC_API_KEY; the init line
  must report that key as the credential, or the process is stopped.
- Only our tools: --tools WebSearch, our speak and list_activities from an MCP server on a loopback port of its
  own behind a random token (--strict-mcp-config), dontAsk with only those allowed, the built-ins denied by name.
  --bare is not used: it takes WebSearch away.
- speak reaches voice.rs while the turn runs; the history is read from the session's transcript.
- The key and the brain setting live in resident-settings.json (0600) in the config dir, behind their own
  endpoints (PUT /v1/resident/brain, PUT/DELETE /v1/resident/anthropic-key); the key is never sent back or logged.
- GET /v1/resident reports brain, brain_setting, anthropic_key_set and the new problems claude_missing and
  anthropic_key_missing; Claude Code hooks from the resident session are dropped like Codex's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…h Apps

The Conversation tab, on the Mac and on Linux, gets "Answered by" (Automatic, Codex or Claude Code) and,
unless it is Codex, a secure field for the user's own Anthropic API key with Save, Remove and a link to the
Claude Console. The key is never shown again: the daemon only says whether it has one. The tab's not-ready
states cover Claude Code: not installed (with Anthropic's setup page) or no key yet; under Automatic, a
missing or signed-out Codex says Claude Code can answer instead. zh-Hans translations for both Apps.

Docs: the Claude Code brain in docs/resident-session.md (auth rule with the terms link, flags and why not
--bare, the MCP server, privacy: text goes to Anthropic), docs/app.md, docs/claude-adapter.md, ADR-0017's
consequences, CONTEXT.md (Brain; Answered by, Anthropic API key), and both READMEs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude asked where the user was for the weather: unlike Codex it had no
time zone, so each voice command now carries it next to the local time.
Both brains are told to write nothing after speak; Claude went on to
repeat its answer as text, taking nine more seconds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Switching from Codex (20 exchanges) to Claude Code (3) left the scroll
view past the end of the shorter history, so the tab looked empty. A new
thread now gets a new scroll view, and a plain stack is scrolled to its
end once laid out.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The resident session's instructions now carry the persona of the Character
the buddy wears (characters/<id>/persona.md, embedded at build time; the
robot gets a persona of its own) and its one language: the buddy understands
any language but always replies in its Character's, whatever earlier replies
or the user's Codex memories say. The user's form of address goes in too.

The box reports only the pack's id, so the Apps now say, when they write a
pack, which preset lends a custom Character its voice, persona and lines and
which form of address its lines carry (POST /v1/device/voice-pack?lender=&
address=). The daemon keeps that in character.json; a custom Character also
speaks in its lender's voice.

When the Character changes, the next voice command resumes the same session
with the new instructions: Codex restarts the app-server and resumes the
thread (turn/start takes no instructions), Claude Code restarts with
--resume. That command carries a context line naming the new Character,
which the history leaves out.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With the rule only in the system prompt, Haiku answered a Chinese
question in Chinese as Mei, carried along by the user's language and the
session's earlier Chinese replies.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants