diff --git a/agentdescent/cost.py b/agentdescent/cost.py index 39fd21d8..1c2426c0 100644 --- a/agentdescent/cost.py +++ b/agentdescent/cost.py @@ -33,11 +33,14 @@ Placement of the price is caller-supplied by design. :class:`~agentdescent.policies.MergeContext` carries ``diff_cost`` when a -caller supplies one, and the policy falls back to the cost the -contributing -:class:`~agentdescent.evolvable.EvidenceCard`\\ s recorded, which is what the -loop already collects. ``aggregator._decide`` does not fill ``diff_cost`` yet, -so the card fallback is what every in-tree caller takes today. +caller supplies one, and a caller that wants the gate to price a diff passes +``Cost.from_usage(usage)`` there. The policy otherwise falls back to the cost +the contributing +:class:`~agentdescent.evolvable.EvidenceCard`\\ s recorded -- and in-tree no +producer writes ``cost_tokens``/``cost_wallclock`` onto a card, so that +fallback is unknown, never a free diff. ``aggregator._decide`` does not fill +``diff_cost`` yet, so the card fallback is what every in-tree caller takes +today. ``agentdescent.promotion.PromotionRules.require_cost`` is the matching fail-closed bar on the promotion side. @@ -155,10 +158,12 @@ def from_usage(cls, usage: Any, *, usd: Optional[float] = None) -> "Cost": def from_cards(cls, cards: Iterable[Any]) -> "Cost": """The cost the contributing evidence cards recorded for this diff. - The loop already records one proposing rollout's usage on each card, so - the diff's price is the sum over the cards that justify it. A card with - no measurement contributes nothing; a run where no card measured - anything yields ``Cost()`` -- unknown, not free. + In-tree this is always unknown: no producer writes ``cost_tokens`` or + ``cost_wallclock`` onto a card (``evolution.py`` and ``async_evolve.py`` + build cards without them), so the sum is empty and this returns + ``Cost()`` -- unknown, not free. A caller that wants the gate to price a + diff passes the measurement as ``MergeContext.diff_cost = + Cost.from_usage(usage)`` instead of relying on the cards. """ tokens = 0 wallclock = 0.0 diff --git a/changelog.d/1085-cost-card-fallback.md b/changelog.d/1085-cost-card-fallback.md new file mode 100644 index 00000000..d2e5780c --- /dev/null +++ b/changelog.d/1085-cost-card-fallback.md @@ -0,0 +1,8 @@ +Changed +`docs/cost.md` and the `Cost.from_cards` / module docstrings now state what the +card fallback actually returns in-tree: no producer writes +`cost_tokens`/`cost_wallclock` onto an `EvidenceCard`, so `Cost.from_cards` +yields `Cost()` — unknown, not a free diff — and a caller that wants the gate to +price a diff passes `MergeContext.diff_cost = Cost.from_usage(usage)`. The page +also warns that `require_cost=True` defers every candidate on a run whose cards +carry no cost (#1085). diff --git a/docs/cost.md b/docs/cost.md index 63ed54f9..15e3d2ec 100644 --- a/docs/cost.md +++ b/docs/cost.md @@ -31,7 +31,10 @@ print(cost_status(cost_efficiency(0.10, cost), 0.05)) # improved/neutral/… free"); an unmeasured field is not. `Cost().known` is `False`, and every derived value (`cost_efficiency`, the report's per-accepted fields) stays `None` rather than pretending the diff was free. `Cost.from_cards` sums what the contributing -[evidence cards](data-model.md) already record for one working diff. +[evidence cards](data-model.md) record for one working diff — and in-tree that +is nothing, since no producer writes a cost onto a card, so it returns `Cost()` +as well. [The acceptance option](#2-the-acceptance-option) has the fallback and +what `require_cost` does with it. `usd` is whatever the caller measured. There is no provider price table and no billing in this repository: an unpriced run leaves `usd=None` and still reports @@ -69,8 +72,26 @@ block (with `status`) is on `provenance["cost"]`. The decision that a cost quality gain is real, it just did not pay for itself. The cost the policy reads is `MergeContext.diff_cost` when the caller measured -one; otherwise it is `Cost.from_cards` over the cards that justify the diff — -the usage the loop already records per working diff. +one. Otherwise it is `Cost.from_cards` over the cards that justify the diff, and +**in-tree that fallback is always unknown**: no producer writes +`cost_tokens`/`cost_wallclock` onto an `EvidenceCard` — `evolution.py` and +`async_evolve.py` build cards without them — so the sum is empty and +`from_cards` returns `Cost()`, not a free diff. A caller that wants the gate to +price a diff passes the measurement itself: + +```python +from agentdescent import Cost + +ctx.diff_cost = Cost.from_usage(usage) # the caller's own metered Usage +``` + +!!! warning "`require_cost=True` on a run whose cards carry no cost" + + Every candidate the inner gate accepted is then deferred, with + `category="cost-unknown"`: that run cannot commit until the caller prices + the diff. The `reference_quality_per_cost` and `min_gain_per_1k_tokens` + branches are read only from a measured cost, so neither fires on the card + fallback. ## 3. The report