Skip to content

docs(cost): the card cost fallback is always unknown in-tree — say so and warn about require_cost #1085

Closed

Description

@HugoXK

Problem

docs/cost.md:71-73 tells a reader that when the caller does not pass
MergeContext.diff_cost, the gate prices the diff from Cost.from_cards — "the
usage the loop already records per working diff". No producer in the package
writes that usage onto a card, so the fallback is always Cost(), unknown.

CostPolicy's card fallback reads EvidenceCard.cost_tokens /
cost_wallclock (cost.py:167-168), and neither producer site sets either field
(evolution.py:4362-4381, async_evolve.py:998-1012); grep -rn "cost_tokens" --include=*.py agentdescent/ returns defaults, readers and JSON round-trips only
(trajectory.py:161-162,214-215,238-239,484-485 are Trajectory, not a card).

Consequence on a normal in-tree run: from_cards() returns Cost(), so
require_cost=True defers every candidate with category="cost-unknown" — a
run configured that way can never commit — and the
reference_quality_per_cost / min_gain_per_1k_tokens branches never fire
because they only run on a measured cost.

Probe (run-1114-0904/probe-cost-card-fallback.py, cards built the way the
producers build them):

Shape Measured
producer-shaped card tokens/wallclock 0/0.0; from_cards().known=False; cost_efficiency=0.10 -> None
require_cost=True accept=False, defer=True, category=cost-unknown
reference_quality_per_cost=0.05 accept=True, status='unknown'
min_gain_per_1k_tokens=1.0 accept=True, status='unknown'
control, cost_tokens=900 status='improved', 0.111 per 1k
control, cost_tokens=5000 accept=False, category=cost-regression

Scope

The page and the module docstrings ship as one change — folded from #1099, because a corrected page beside a docstring that still says the opposite is the same contradiction one file over.

  • agentdescent/cost.py:155-161 (Cost.from_cards): replace "The loop already records one proposing rollout's usage on each card, so the diff's price is the sum over the cards that justify it." — no producer sets cost_tokens/cost_wallclock (evolution.py:4362-4381, async_evolve.py:998-1012; evolvable.py:237-238 are defaults), so the honest sentence is that the fallback is unknown in-tree and a caller wanting a priced diff passes MergeContext.diff_cost = Cost.from_usage(usage).

  • agentdescent/cost.py:35-39 (module docstring): the first sentence ("the cost the contributing EvidenceCards recorded, which is what the loop already collects") is the false one; the second sentence, that aggregator._decide does not fill diff_cost so the card fallback is what every in-tree caller takes, is correct and stays.

  • Rewrite the two sentences at docs/cost.md:71-73: in-tree, no producer fills
    the card cost, so the fallback is unknown, and a caller that wants the gate to
    price a diff must pass MergeContext.diff_cost = Cost.from_usage(usage).

  • Add the warning that require_cost=True defers every candidate on a run whose
    cards carry no cost.

  • One changelog fragment. No behaviour change and no test surface: the only
    code in this PR is two docstrings in agentdescent/cost.py — no branch, no
    signature.

Tier: T2 — agentdescent/** is a T2 path (the docs alone would be T0).
Danyang's click, so a PASS at the tip first.

The real fix — recording the proposing rollout's tokens on the card at the two
producer sites — is out of scope here: the Usage is one shared Meter across
workers, so per-rollout attribution is racy and that is a T3 change in
evolution.py.

Acceptance (added for the folded docstrings)

  • grep -n "already records one proposing rollout" agentdescent/cost.py → no
    match; the module docstring's fallback sentence says the fallback is unknown
    in-tree and names Cost.from_usage as the caller's route.
  • grep -n "already record for one working diff" docs/cost.md → no match;
    the §1 summary says the card sum is Cost() in-tree and points at §2's
    fallback and require_cost warning.
  • python -m pytest -q tests/test_cost_per_accepted.py tests/test_cost_aware_selection.py
    → green; python -m tools.gen_api_docs --check → up to date.

Found by: AgentDescent Product (09:04Z audit at 0c9e936d).

Activity

  1. added
    enhancementNew feature or request
    area:evolveThe evolve loop: evolution, async_evolve, aggregator, population, policies, strategies
    area:docsDocumentation site (docs/, mkdocs.yml) and README
    on Nov 14, 2026
  2. self-assigned this
    on Nov 14, 2026
  3. HugoXK commented on Nov 14, 2026

    @HugoXK
    CollaboratorAuthor

    Assigned to AgentDescent Dev-4, benched behind #951's PR #1086. Its own PR opens after a merge, not now — the ready queue is at its cap.

    Scope stays docs-only: docs/cost.md:71-73 rewritten, the require_cost warning, one fragment.

  4. HugoXK commented on Nov 15, 2026

    @HugoXK
    CollaboratorAuthor

    Claim: carrier Dev-1, branch agent/1085-cost-card-fallback cut from live main 09492927f614b5303e54cc3f87b0bb809929e336.

    Branch only for now — #1097 holds this repo's open-PR slot, so the Fixes #1085 PR opens when that one frees.

    Scope widened once, on record, and no further: the page and the module that states the same fallback ship together — the docs/cost.md fallback sentences plus a require_cost=True warning, the two agentdescent/cost.py docstrings that repeat the claim (Cost.from_cards, module docstring), one changelog fragment, and the acceptance row. That code half is the #1099 fold recorded there and approved by Chief; it is why the tier is now T2 (agentdescent/**). It is docstrings only — no signature, branch or import change — and still not the two producer sites (evolution.py:4362-4381, async_evolve.py:998-1012): recording the proposing rollout's tokens on the card remains the separate real fix this issue deliberately leaves out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area:docsDocumentation site (docs/, mkdocs.yml) and READMEarea:evolveThe evolve loop: evolution, async_evolve, aggregator, population, policies, strategiesenhancementNew feature or request

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions