Skip to content

feat(cost-diagnostics): content-free tool census and compaction counter - #1095

Open
philmerrell wants to merge 2 commits into
developfrom
feature/cost-diagnostics-census
Open

feat(cost-diagnostics): content-free tool census and compaction counter#1095
philmerrell wants to merge 2 commits into
developfrom
feature/cost-diagnostics-census

Conversation

@philmerrell

Copy link
Copy Markdown
Contributor

Summary

PR 3 of 3. The one signal the admin session profile could not derive from existing rows: which tools a conversation called, how often, and how often they failed — plus how many times compaction actually fired. Recorded out of band at turn end, never in the conversation, so the cacheable prefix is untouched. Independent of the other two PRs (the API tolerates the attributes' absence).

  • ToolCensusHook tallies tool name → {calls, errors} per model call using the same cycle counter AgentStatusHook uses; non-drained, per-turn only. The coordinator reads each call's tally at turn end and attaches it to that call's C# row as toolCalls (the turnAgentId mechanism).
  • _bump_session_aggregates ADDs toolCallCount / toolErrorCount in the same UpdateItem as totalCost.
  • _save_compaction_state(record_event=True) ADDs a monotonic compactionCount when a checkpoint is actually cut — the persisted compaction map is last-write-wins and cannot count occurrences; a top-level ADD never moves backwards even with two Agents on one session.
  • COST_DIAGNOSTICS_ENABLED — default on, =false kill switch (unset/empty ⇒ on). With it off nothing is written, so the profile reads "not tracked" rather than an honest-looking 0.

Deploy notes

🏗️ backend.yml only. Additive attributes on rows the turn already writes — no table, no index, no backfill. Sessions that predate this read "not tracked" in the admin profile. Forks need to do nothing; the census appears as data accrues.

Verification

  • uv run python -m pytest tests/ — full suite green (16 new: hook attribution/reset/non-drained reads/kill switch, moto persistence on C# + S#, coordinator attach seam, compaction counter + kill switch).

🤖 Generated with Claude Code

philmerrell and others added 2 commits September 13, 2026 13:24
The one signal the admin session profile could not derive from existing
rows: which tools a conversation called, how often, and how often they
failed. Recorded out of band at turn end, never in the conversation, so the
cacheable prefix is untouched.

- ToolCensusHook tallies tool name -> {calls, errors} per model call using
  the same cycle counter AgentStatusHook uses; non-drained and per-turn only.
  The stream coordinator reads each call's tally at turn end and attaches it
  to that call's C# cost row as `toolCalls` (the turnAgentId mechanism).
- _bump_session_aggregates ADDs `toolCallCount` / `toolErrorCount` on the
  session row in the same UpdateItem as totalCost.
- _save_compaction_state(record_event=True) ADDs a monotonic
  `compactionCount` when a checkpoint is actually cut — the persisted
  `compaction` map is last-write-wins and cannot count occurrences.
- COST_DIAGNOSTICS_ENABLED, default on with a `=false` kill switch. With it
  off nothing is written, so the profile reads "not tracked" rather than 0.

Additive attributes only — no table, no index, no backfill; sessions that
predate this read "not tracked" in the admin profile.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…; match the save stub's signature

- apis/shared/sessions/metadata.py now reads the COST_DIAGNOSTICS_ENABLED
  kill switch, which makes apis/shared/feature_flags.py reachable from the
  scheduled-runs Lambdas' import graph. COPY it into the image and list it in
  the build manifest so the content-hash tag notices future changes (the
  supply-chain import-closure guard caught this).
- test_compaction_stability's _save stub mirrors the real signature, which
  gained record_event.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant