Skip to content

fix(pricing): verify Anthropic batch rows; retract unverified Codex GA claim - #262

Merged
ayn-builds merged 8 commits into
awslabs:mainfrom
herosjourney:fix/opus46-sonnet46-batch-and-codex-pricing
Sep 15, 2026
Merged

ayn-builds merged 8 commits into
awslabs:mainfrom
herosjourney:fix/opus46-sonnet46-batch-and-codex-pricing

Conversation

@herosjourney

@herosjourney herosjourney commented Sep 2, 2026 •

Copy link
Copy Markdown
Contributor

Problem

A routine audit of the Anthropic batch & prompt-cache table (pricing-cache.md) and the OpenAI-on-Bedrock reference (openai-on-bedrock.md) surfaced two pricing facts that were recorded without a citation to a model-support table, so nothing would catch them drifting from reality:

  1. Unverified batch rows presented as fact. The batch table lists rates for Fable 5, Sonnet 5, and Opus 4.8 with no distinction from the rest of the table. None of those three appear on the official batch-inference-supported-models table — their rows were a 50%-of-on-demand projection, not a confirmed SKU. An estimate that builds a batch-discount TCO on one of these three models would be citing a rate that may not exist.

  2. A fabricated GA claim for Codex on Bedrock. openai-on-bedrock.md stated "Codex on Bedrock is GA with pay-per-token pricing." Codex does not appear on the OpenAI model card index, and no Codex rate exists anywhere in this plugin — not on the pricing page, not in pricing-cache.md, not in bedrock_pricing.py's static table. Any coding-agent workload routed through this claim would hit a pricing dead end with no guidance on what to do instead.

Solution

1. Verify the batch table row-by-row and mark what isn't confirmed (pricing-cache.md).

  • Checked all nine rows against the batch-inference-supported-models table. Opus 4.6, Sonnet 4.6, Opus 4.5, Sonnet 4.5, and Haiku 4.5 are listed there (via CRIS profiles), and their cached batch rates match the pricing page's 50%-of-on-demand discount exactly — confirmed correct.
  • Fable 5, Sonnet 5, and Opus 4.8 are not on that table. Marked their batch cells † with a footnote: unverified, do not build a batch-discount TCO on them, price on-demand instead.
  • Added a note against relying on the AWS Pricing Calculator to cross-check this table — its Bedrock batch/cache coverage lags the docs and pricing page for several of these models, so it isn't a reliable second source here. The batch-inference-supported-models table and pricing page remain the sources of truth.

2. Retract the Codex claim (openai-on-bedrock.md).

Rewrote the paragraph: re-check the model card index on refresh; if Codex is still absent, route coding-agent workloads to a priced cross-family target (GPT-5.6 Sol / Terra fit the coding-agent tier) or report pricing_source: "unverified" — never attach a fabricated rate to an estimate.

Both files mirrored to advisor/plugins/aws-startup-advisor/ per the drift-gate convention (both copies were byte-identical before this change).

Verification

  • mise run drift:check — OK (advisor vendored copies synced, 270 identical / 25 allowlisted)
  • mise run shared:check, mise run lint:model-ids — OK
  • mise run lint:md, dprint check — clean on changed files
  • mise run pricing:staleness — pre-existing warnings only (global CRIS GPT-5.6 rows, untouched here)
  • Batch support verified against the AWS docs batch-inference-supported-models table (2026-09-02), not estimated

Not in scope

  • The pre-existing pricing:staleness drift warnings on the global.openai.gpt-5.6-* STATIC_FALLBACK rows vs. the cache rate card
  • Claude Opus 5 appears on the docs batch table but isn't in the pricing cache at all yet — separate refresh

Type of Change

  • Bug fix
  • New plugin/power/tool
  • Enhancement to existing content
  • Documentation update
  • Guardrail/CI update

…ified Codex-on-Bedrock GA claim

- Batch & prompt cache table: Opus 4.6 / Sonnet 4.6 / Opus 4.5 / Sonnet 4.5 /
  Haiku 4.5 batch support verified 2026-09-02 against the Bedrock
  batch-inference-supported docs table (50% of on-demand, matching existing
  rows). Fable 5, Sonnet 5, and Opus 4.8 are NOT on that table - their batch
  cells are now marked unverified with an explicit no-batch-discount-TCO rule.
- Warn against cross-checking these rows with the AWS Pricing Calculator,
  which carries no batch metered IDs for Opus 4.6 / Sonnet 4.6 and collapses
  other batch/cache SKUs (external SUP-BLOCKER report).
- openai-on-bedrock.md: retract 'Codex on Bedrock is GA' - Codex is absent
  from the OpenAI model card index and has no rate anywhere in this plugin;
  route coding-agent workloads to a priced cross-family target or report
  pricing_source: unverified instead of fabricating a rate.
- Advisor vendored copies synced (drift:check OK).
@herosjourney
herosjourney requested review from a team as code owners September 2, 2026 15:06

@ayn-builds ayn-builds left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both changed files are byte-identical across advisor/ and migrate/ at ee781f7, so no drift.

The substance here is right and I'd like it to land. Retracting the Codex claim with instructions for what to do instead beats deleting it, and the batch banner is the right shape - :386 already carried an ad hoc version of this exact warning ("confirm batch availability per model (Opus 4.7 batch was not listed on the global cross-region table when this row was added)"), so promoting it to a marker plus footnote is a real improvement. The calculator caveat is a genuinely useful thing to have written down.

Inline notes. The first two are both "the fix doesn't reach where it needs to":

  1. The Codex retraction doesn't reach design-refs/ai-openai-to-bedrock.md:257, which still says "Codex on Bedrock (GA)" with pay-per-token. Design routes there directly, and openai-on-bedrock.md:13-15 explicitly says that file defers to this one.
  2. The † marker has no consumer. design-ai.md:189 and estimate-ai.md:140 still apply 50% to "any model," and the unverified gate at estimate-ai.md:126 is scoped to OpenAI rows, so it won't cover these even if the cells are relabelled.
  3. Adjacent rather than yours: Sonnet 5's batch and cache cells derive from the $2/$10 intro rate that ended Aug 31. The cache's Last updated: 2026-08-24 predates that, so the row was fine when written - but the new banner is dated 2026-09-02 and now certifies those cells as correct, and the 30-day staleness rule won't fire until Sep 23.

Plus two nits (the ninth table row isn't named in the banner, and † already means something else at ai-anthropic-to-bedrock.md:18) and one question about whether the CRIS premium affects the confirmed batch rates.

All line edits in files this PR already touches, except the design-ref row and the two batch-discount tables.

… unverified gate, Sonnet 5 rollover

Addresses ayn-builds review on PR awslabs#262:

- Codex retraction now reaches design-refs/ai-openai-to-bedrock.md (row no
  longer claims 'GA / pay-per-token'; points to shared/openai-on-bedrock.md)
  and Codex is added to the 'Not on Bedrock' list, so Design (which routes to
  that file directly) no longer reads the stale GA claim.
- Batch unverified marker now has a consumer: switched glyph from the
  colliding daggerto double-dagger, footnote uses the _unverified_ vocabulary,
  the two batch-discount rows (design-ai.md, estimate-ai.md) now carve out
  confirmed-batch models only, and the unverified gate in estimate-ai.md is
  widened beyond OpenAI to cover any _unverified_ row (incl. Anthropic batch).
- Sonnet 5 intro pricing (/) ended Aug 31; on-demand row, prose, and the
  batch/cache cells updated to the /-derived values (1.50/7.50 batch,
  3.75/6.00/0.30 cache) — the file states the rollover date so this isn't
  drift the staleness rule would catch in time.
- Banner: added Sonnet 4.5 LC (9th row) to the confirmed list and a CRIS-vs-
  N.Virginia batch-rate clause.

All 5 files kept byte-identical across advisor/ and migrate/ (drift: 270
identical). build lint/frontmatter/drift, markdownlint, dprint all clean.
@herosjourney

Copy link
Copy Markdown
Contributor Author

Thanks @ayn-builds — all addressed in 08b521b.

1. Codex retraction reaches the design-ref. design-refs/ai-openai-to-bedrock.md row is now Not verified on Bedrock — see shared/openai-on-bedrock.md / Do not price; re-check the OpenAI model card index first, and Codex is added to the Not on Bedrock list in openai-on-bedrock.md. Design no longer has a path to the stale GA claim.

2. The batch marker now has a consumer.

  • Switched the glyph to ‡ (the † collision with ai-anthropic-to-bedrock.md:18 is gone).
  • Footnote now uses the _unverified_ vocabulary the gate keys on.
  • Added the model qualifier to both batch-discount rows (design-ai.md, estimate-ai.md): confirmed-batch models only.
  • Widened the unverified gate in estimate-ai.md beyond OpenAI so it covers any _unverified_ row, including the Anthropic batch cells.

3. Sonnet 5 rollover. The intro $2/$10 ended Aug 31, so the on-demand row, the prose, and the batch/cache cells now use the $3/$15-derived values (1.50 / 7.50 batch, 3.75 / 6.00 / 0.30 cache — matching Sonnet 4.6). You were right it was missed, not deferred.

Nits. Added Sonnet 4.5 — LC (the 9th row) to the confirmed list in the banner.

CRIS question. Good catch — rather than re-derive, I added a clause to the banner: the confirmed rows are 50% of the N. Virginia on-demand price; where batch is only reachable via a CRIS profile listing above N. Virginia (e.g. Sonnet 4.6 in Ohio, ≈10% higher), the effective batch rate is ≈10% above these cells, and you should re-derive from the profile's on-demand rate for a region-specific quote.

Verification: mise run build lint/frontmatter/drift all OK, drift:check 270 identical / 25 allowlisted (no new suppressions), markdownlint 0 errors, dprint clean. All 5 files remain byte-identical across advisor/ and migrate/.

Logan Kleier added 2 commits September 12, 2026 11:16
…atch-and-codex-pricing

# Conflicts:
#	advisor/plugins/aws-startup-advisor/skills/gcp-to-aws/references/phases/estimate/estimate-ai.md
#	migrate/plugins/migration-to-aws/skills/gcp-to-aws/references/phases/estimate/estimate-ai.md
Resolve pricing-cache.md (both plugin copies) against ee92c83: Anthropic made
Sonnet 5's $2/$10 launch rate the standard price and cancelled the Sep 1
increase to $3/$15 (Anthropic pricing page; Bedrock pricing page read
2026-09-03 on main). Take main's on-demand narrative and quick-reference row,
and re-derive this branch's Sonnet 5 batch/cache row from $2/$10 instead of
$3/$15: batch 1.00/5.00 (‡, still not on the batch-supported table), 5m cache
write 2.50, 1h cache write 4.00, cache read 0.20. Batch-verification notes and
‡ markers from this branch are kept unchanged.
@herosjourney

Copy link
Copy Markdown
Contributor Author

Conflicts resolved at cf5589f (merge of origin/main into this branch).

The conflict in both copies of pricing-cache.md was factual, not textual. This branch assumed Claude Sonnet 5's scheduled Sep 1 increase to $3/$15 took effect and derived its batch/cache row from that. main (ee92c83, Sep 3) records that Anthropic made the $2/$10 launch rate the standard price and cancelled the increase. Checked against the primary source before choosing: Anthropic's pricing page says the $2/$10 rate "is now the standard price" and the $3/$15 increase "will not occur".

Resolution, identical in both plugin copies:

  • Took main's Sonnet 5 on-demand narrative and $2/$10 quick-reference row.
  • Kept this branch's batch-verification work unchanged (2026-09-02 docs-table check, ‡ markers, CRIS and Pricing Calculator caveats, _unverified_ gate footnote).
  • Re-derived the Sonnet 5 batch/cache row from $2/$10 using the same multipliers as every other row (50% / 1.25× / 2× / 0.1×): batch 1.00 ‡ / 5.00 ‡, cache write 2.50 / 4.00, cache read 0.20. The pre-merge 1.50 / 7.50 / 3.75 / 6.00 / 0.30 would have overstated Sonnet 5 batch and cache by 50%.

Verified after the merge: twins byte-identical; mise run lint exit 0; drift:check 275 identical / 25 allowlisted (unchanged); shared:check OK; node suites pass; test_bedrock_pricing.py 21/21 (uv run --group dev pytest); sweep for any remaining "$3/$15 Sonnet 5" claim across both trees returned nothing.

Note for reviewers: the PR description still calls the unverified marker †; the files use ‡.

@ayn-builds
ayn-builds merged commit 57708ee into awslabs:main Sep 15, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants