Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 14 additions & 9 deletions agentdescent/cost.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,11 +33,14 @@

Placement of the price is caller-supplied by design.
:class:`~agentdescent.policies.MergeContext` carries ``diff_cost`` when a
caller supplies one, and the policy falls back to the cost the
contributing
:class:`~agentdescent.evolvable.EvidenceCard`\\ s recorded, which is what the
loop already collects. ``aggregator._decide`` does not fill ``diff_cost`` yet,
so the card fallback is what every in-tree caller takes today.
caller supplies one, and a caller that wants the gate to price a diff passes
``Cost.from_usage(usage)`` there. The policy otherwise falls back to the cost
the contributing
:class:`~agentdescent.evolvable.EvidenceCard`\\ s recorded -- and in-tree no
producer writes ``cost_tokens``/``cost_wallclock`` onto a card, so that
fallback is unknown, never a free diff. ``aggregator._decide`` does not fill
``diff_cost`` yet, so the card fallback is what every in-tree caller takes
today.
``agentdescent.promotion.PromotionRules.require_cost``
is the matching fail-closed bar on the promotion side.

Expand Down Expand Up @@ -155,10 +158,12 @@ def from_usage(cls, usage: Any, *, usd: Optional[float] = None) -> "Cost":
def from_cards(cls, cards: Iterable[Any]) -> "Cost":
"""The cost the contributing evidence cards recorded for this diff.

The loop already records one proposing rollout's usage on each card, so
the diff's price is the sum over the cards that justify it. A card with
no measurement contributes nothing; a run where no card measured
anything yields ``Cost()`` -- unknown, not free.
In-tree this is always unknown: no producer writes ``cost_tokens`` or
``cost_wallclock`` onto a card (``evolution.py`` and ``async_evolve.py``
build cards without them), so the sum is empty and this returns
``Cost()`` -- unknown, not free. A caller that wants the gate to price a
diff passes the measurement as ``MergeContext.diff_cost =
Cost.from_usage(usage)`` instead of relying on the cards.
"""
tokens = 0
wallclock = 0.0
Expand Down
8 changes: 8 additions & 0 deletions changelog.d/1085-cost-card-fallback.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
Changed
`docs/cost.md` and the `Cost.from_cards` / module docstrings now state what the
card fallback actually returns in-tree: no producer writes
`cost_tokens`/`cost_wallclock` onto an `EvidenceCard`, so `Cost.from_cards`
yields `Cost()` — unknown, not a free diff — and a caller that wants the gate to
price a diff passes `MergeContext.diff_cost = Cost.from_usage(usage)`. The page
also warns that `require_cost=True` defers every candidate on a run whose cards
carry no cost (#1085).
27 changes: 24 additions & 3 deletions docs/cost.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,10 @@ print(cost_status(cost_efficiency(0.10, cost), 0.05)) # improved/neutral/…
free"); an unmeasured field is not. `Cost().known` is `False`, and every derived
value (`cost_efficiency`, the report's per-accepted fields) stays `None` rather
than pretending the diff was free. `Cost.from_cards` sums what the contributing
[evidence cards](data-model.md) already record for one working diff.
[evidence cards](data-model.md) record for one working diff — and in-tree that
is nothing, since no producer writes a cost onto a card, so it returns `Cost()`
as well. [The acceptance option](#2-the-acceptance-option) has the fallback and
what `require_cost` does with it.

`usd` is whatever the caller measured. There is no provider price table and no
billing in this repository: an unpriced run leaves `usd=None` and still reports
Expand Down Expand Up @@ -69,8 +72,26 @@ block (with `status`) is on `provenance["cost"]`. The decision that a cost
quality gain is real, it just did not pay for itself.

The cost the policy reads is `MergeContext.diff_cost` when the caller measured
one; otherwise it is `Cost.from_cards` over the cards that justify the diff —
the usage the loop already records per working diff.
one. Otherwise it is `Cost.from_cards` over the cards that justify the diff, and
**in-tree that fallback is always unknown**: no producer writes
`cost_tokens`/`cost_wallclock` onto an `EvidenceCard` — `evolution.py` and
`async_evolve.py` build cards without them — so the sum is empty and
`from_cards` returns `Cost()`, not a free diff. A caller that wants the gate to
price a diff passes the measurement itself:

```python
from agentdescent import Cost

ctx.diff_cost = Cost.from_usage(usage) # the caller's own metered Usage
```

!!! warning "`require_cost=True` on a run whose cards carry no cost"

Every candidate the inner gate accepted is then deferred, with
`category="cost-unknown"`: that run cannot commit until the caller prices
the diff. The `reference_quality_per_cost` and `min_gain_per_1k_tokens`
branches are read only from a measured cost, so neither fires on the card
fallback.

## 3. The report

Expand Down
Loading