Decode character references in an HTML-only inbound email

[RaphaelFakhri]

What this changes

An email trigger reads an HTML-only mail by stripping its tags, and left its character references in place, so the Bot read AT&T, we’re and Q3 numbers. extractEmailText now decodes numeric references (’, —), the five XML names and   after the tags are stripped. The decoding is one pass, so < reads as < and never becomes a tag. A name outside that list, or a reference to an invalid code point, is left as written.

Fixes #806.

Where it runs

• New state that outlives a request? None. The change is a pure string transform inside extractEmailText.
• What happens on the second replica? The same: every replica decodes the same mail to the same text.
• Anything serialised? No.
• Anything fanned out to a browser? No.
• New listener, port, or schedule? No.

Boundary and audit

• Every acting call still goes through the gateway: resolve, decide, audit, then act. Unchanged; only the text of the delivered payload differs.
• New refusals and new failures each write a row. None are added.
• Nothing new is trusted from the client that the server can resolve itself.

Changelog

• A line in CHANGELOG.md under Unreleased.

Proof

New case in server/tests/responsibility-triggers.test.ts feeds an HTML-only part with  , &, <, ", ’, — and <.

Before the change in email.ts:

cd server && bun test tests/responsibility-triggers.test.ts
(fail) inbound email via SES -> SNS > MIME text extraction decodes character references in an HTML-only part
 28 pass
 1 fail

After:

cd server && bun test tests/responsibility-triggers.test.ts
 29 pass
 0 fail

bunx biome check is clean on the three changed files, and bunx tsc --noEmit -p server passes.

[charan-rathore]

Nice fix. One missed case worth a test while you're here: the guard in decodeCharacterReference (out-of-range > 0x10ffff and surrogate range 0xd800..0xdfff leave the reference as written) has no coverage, so a refactor could drop it silently. Something like � and � staying literal in the output would pin it down.

Prepared with AI assistance (Instinct assisted).