Skip to content

Add always-on build telemetry delivery - #1099

Open
anth-volk wants to merge 5 commits into
mainfrom
codex/always-on-telemetry-emitter
Open

anth-volk wants to merge 5 commits into
mainfrom
codex/always-on-telemetry-emitter

Conversation

@anth-volk

@anth-volk anth-volk commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • start a local telemetry emitter service automatically for supported US and UK dataset builds
  • keep network access, Hugging Face authentication, retries, process sampling, and the bounded SQLite queue outside the build process
  • use HF_TOKEN, HUGGINGFACE_TOKEN, or the cached Hugging Face login automatically; no telemetry configuration is required for PolicyEngine members
  • exchange only the Hugging Face credential for one short-lived collector session credential, then register runs and upload events with that credential
  • use one fixed production collector origin with no production destination override; retain only an explicit loopback option for development
  • permanently keep a run local when no Hugging Face credential exists or membership is rejected, including across process restarts, while retaining transient failures for retry
  • report stage transitions, graph-node and batch progress, calibration progress, resource use, heartbeats, completion, and classified failures without allowing collector availability to fail a build

The hosted collector and dashboard integration are in PolicyEngine/calibration-diagnostics#199.

Verification

  • focused telemetry suite: 17 tests passed
  • affected US release and UK build suites: passed with two expected skips
  • Ruff check and format on changed Python files: passed
  • repository diff check: passed

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant