On 16th August, one of our CI and bench chats generated 92 Task workers. When we added the parent conversation and those workers together, the cost-equivalent usage came to about $183.
+On 16th August, one NuttX CI-watch chat generated 92 Task workers. When we added the parent conversation and those workers together, the cost-equivalent usage came to $183.21.
Call it roughly $200 for one chat.
That needs a qualification. It was a combined usage estimate, not necessarily a $183 charge landing on a card. Some of the work sat within bundled product usage. Nor was every dollar caused by CI polling alone. The thread mixed build monitoring, bench work, Task fan-out and large tool results.
But the shape of the waste was clear. We had made a reasoning system behave like a sensor.
@@ -71,17 +71,20 @@Either side can arrive first. A very short build may complete before its wait registration reaches the dispatcher. A laptop may be asleep when the webhook arrives. A task may still be active when its event is delivered. Those are transport and concurrency problems, not reasons to make an agent poll. The event is retained and reconciled when the other side becomes available.
+Those are executable contracts rather than good intentions. A duplicate event is absorbed without a second wake. An event arriving after its registration expires is marked stale and cannot resume a task. An event arriving before its registration survives a dispatcher restart and is reconciled as soon as the exact registration appears. The disconnected replay sends a retained event and registration once. Five focused ordering tests exercise those cases; the live offline exercise described below tests the same design across the real tunnel.
Success events matter as much as failures. A successful image build often unblocks the next physical-board or runtime test. If success does not wake the task, someone still has to watch the build.
Event-driven delivery solved when to wake the agent. It did not by itself solve what to put into context.
A raw BitBake, Soong or Ninja log can contain megabytes of routine progress. Pulling all of it into a chat recreates much of the cost in a different form. It can also bury the useful failure line.
The harness now reduces logs programmatically before model reasoning begins. For BitBake, the first ERROR: record is significant even when unrelated tasks continue afterwards. For Android, we look for bounded, actionable Ninja, Soong, compiler, lpmake or avbtool failures. The extractor sanitises the first useful signal and leaves the ordinary log outside model context.
The limits are explicit. A webhook body is capped at 128 KiB. The origin may read a build log once, locally, up to 64 MiB, but the reducer can pass on no more than a 1,000-character specific error, 12 changed paths and a 3,000-character state summary. The continuation accepts no more than eight allowlisted evidence values and renders a task prompt of at most 3,000 characters. Raw logs, authorisation material and unsanitised secrets are rejected. Forty-five fresh tests cover the reducers, fail-fast rules, webhooks and Jev advisory boundary.
Where we control the build process, the same rules can fail fast. There is little value in allowing hours of dependent work to continue after a decisive failure if the harness can stop safely, wake the task and begin a repair.
We are also testing Jev through a Preloop adapter as an observe-only semantic sensor. It can help classify a failure, judge whether the first error appears actionable and suggest the cheapest next proof.
It is deliberately not in the critical wake path.
The deterministic event and bounded failure envelope are stored and routed first. Jev is supplemental. It cannot mark CI green, authorise a change, retry a build or weaken compiler, test, hash or human approval gates. If Jev or Preloop is unavailable, the task still wakes. A later explicit event may make one deferred advisory attempt; there is no watcher checking when the watcher is available.
This distinction matters. Event-driven should not mean handing control to a probabilistic component. It means using deterministic events to decide when reasoning is worth paying for.
+The real Foundries failure gave us a useful example. The retained advisory was 2,713 bytes, including the deterministic envelope, rather than a raw build log. Jev used 1,133 input and 206 output tokens. It confidently classified the failure lane, but marked the question of whether the first error was actionable as ambiguous. That is useful advice precisely because it stayed advice. The task had already been routed, the CI result remained failed and every execution authority flag remained false.
The expected saving is not mysterious:
The aim is narrower and more defensible: do not spend reasoning tokens on waiting.
At the time of this draft, the Foundries kiosk and Android FRDM lanes provide the reference implementation. A real Foundries failure has already traversed the webhook, durable outbox, private tunnel and exact-task continuation path. That is useful evidence, but it is not yet a claim that every external wait in the harness is event-driven.
+The disconnected path is no longer theoretical. We stopped the private tunnel, registered an exact wait and retained a synthetic success event on the origin without a delivery receipt. Reconnecting ran one bounded replay, wrote the receipt and resumed the owning task once. A later reconnect accepted no new event and created no second turn. That proves the transport behaviour; it does not turn a synthetic event into evidence of a real successful build.
+The exercise also found a less glamorous fault. Nine old remote replay helpers had survived earlier tunnel disconnects. They were not spending model tokens, but they were still accumulating watchers in a design intended to avoid them. We changed the helper to terminate with its owning SSH process and proved the live count moved from one while connected, to zero on disconnect, and back to one after recovery. Event-driven design includes cleaning up the transport that waits for events.
+The scale of the old pattern is now easier to see. Before we paused the kiosk heartbeat, its task had recorded 811 completed turns and 668,928,917 locally counted tokens. The Android task had recorded 797 completed turns and 831,302,773 tokens. In both cases 98.9% of input was cached. Those are whole-task counters, not a measure of polling alone, and they are not an invoice. They are a baseline for the before-and-after comparison, not a cost-saving claim.
+The first exact after-measurement added another caveat. Foundries build 2969 woke its owning task once, exactly as designed. That continuation turn still recorded 1,830,847 local tokens across 12 model responses because the task it resumed was already very large. Of its 1,824,786 input tokens, 98.1% were cached. Again, that is not a bill. It is evidence that event delivery and context hygiene solve different problems. Webhooks remove empty waiting turns; they do not make an accumulated conversation cheap to reload.
+The next design question is therefore how to keep the exact owning task small enough to resume economically - for example, by reaching a deliberate task boundary before a long wait - without losing the correlation, evidence or human context that made the task useful.
Before publishing this as a completed journey, I want evidence that: