feat(telemetry): expose LLM token usage coverage

[ehz0ah]

Description

Expose whether the LLM token subtotal covers every successful VLM response in an operation.

OpenAI and LiteLLM currently skip token aggregation when a successful response has no usage object. This makes missing usage look like an explicitly reported zero. If an earlier response reported usage, a later response without usage leaves a valid but partial subtotal that looks complete.

This change adds an optional summary.tokens.llm.coverage object. It reports complete, partial, or missing with successful, reported, and missing response counts. Numeric token totals keep their existing meaning.

Human Involvement

• A human participated in the implementation or review loop
• This PR was generated entirely by AI agents without human participation in the loop

Related Issue

The same machine-readable accounting ambiguity is described in openai/codex#54055. This PR implements a focused OpenViking contract. It does not copy external code or claim provider billing finality.

Type of Change

• Bug fix (non-breaking change that fixes an issue)
• New feature (non-breaking change that adds functionality)
• Breaking change (fix or feature that would cause existing functionality not to work as expected)
• Documentation update
• Refactoring (no functional changes)
• Performance improvement
• Test update

Changes Made

• Count successful VLM responses and responses that include a usage object.
• Expose coverage under summary.tokens.llm.coverage without changing numeric totals.
• Cover OpenAI, LiteLLM, and the separate VolcEngine media Responses path.
• Keep coverage absent when no successful VLM response occurred.

Example:

{
  "tokens": {
    "llm": {
      "input": 100,
      "output": 10,
      "total": 110,
      "coverage": {
        "status": "partial",
        "successful_response_count": 2,
        "reported_response_count": 1,
        "missing_report_count": 1
      }
    }
  }
}

Testing

• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• I have tested this on the following platforms:
  • Linux
  • macOS
  • Windows

Commands run:

PYTHONPATH=. /private/tmp/openviking-pass880-venv/bin/pytest tests/models/vlm/test_token_usage_coverage.py tests/test_telemetry_runtime.py --disable-warnings -ra
/private/tmp/openviking-pass880-venv/bin/ruff format --check openviking/telemetry/operation.py openviking/models/vlm/base.py openviking/models/vlm/backends/openai_vlm.py openviking/models/vlm/backends/litellm_vlm.py openviking/models/vlm/backends/volcengine_media.py tests/models/vlm/test_token_usage_coverage.py
/private/tmp/openviking-pass880-venv/bin/ruff check openviking/telemetry/operation.py openviking/models/vlm/base.py openviking/models/vlm/backends/openai_vlm.py openviking/models/vlm/backends/litellm_vlm.py openviking/models/vlm/backends/volcengine_media.py tests/models/vlm/test_token_usage_coverage.py
git diff --check 46e15a248863831e34c4602dc96112b0b120aae2..HEAD

The focused run passed 35 tests. Two byte-identical fixed-state probe runs passed 41 source and behavior checks. Missing versus explicit zero and partial versus complete improved from 0/2 distinguishable provider families on the base to 3/3 instrumented provider paths.

A broader regression run passed 162 tests and failed 17. The same 17 exact cases fail on the unchanged base. They are existing Ollama num_ctx expectation mismatches and one existing OpenAI response usage-shape mismatch.

Checklist

• My code follows the project's coding style
• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• Any dependent changes have been merged and published

Screenshots (if applicable)

Not applicable.

Additional Notes

The scope is observed successful VLM response coverage for one operation. It does not cover failed attempts, retries, replay identity, delegated threads, cumulative session usage, or provider billing finality. There are no dependencies, configuration changes, stored-data changes, or migrations.