Skip to content

fix(bedrock): refresh model lifecycle statuses and adopt the 45-day Legacy policy - #307

Merged
leon1418 merged 5 commits into
awslabs:mainfrom
herosjourney:fix/bedrock-lifecycle-refresh-sep2026
Sep 22, 2026
Merged

leon1418 merged 5 commits into
awslabs:mainfrom
herosjourney:fix/bedrock-lifecycle-refresh-sep2026

Conversation

@herosjourney

@herosjourney herosjourney commented Sep 18, 2026 •

Copy link
Copy Markdown
Contributor

Problem

The Bedrock lifecycle table in both plugin trees is a snapshot dated Aug 24, 2026. Two things have gone stale since:

  1. Three models have passed EOL and three more crossed into the 90-day exclusion zone. The plugin would still recommend, or price, models that are either already gone or will be unusable before a migration reaches production.
  2. AWS split the lifecycle policy on 2026-09-07. Models launched on or after that date are governed by their model card and may get only a 45-day Legacy period instead of six months, and they never appear in the pre-2026-09-07 Legacy table at all. Previously, a candidate's absence from the table was indistinguishable from "Active" — which is now an unsafe inference.

Changes

Applied identically to migrate/plugins/migration-to-aws/ and advisor/plugins/aws-startup-advisor/.

skills/gcp-to-aws/references/shared/ai-model-lifecycle.md

Documents both policies and links the two authoritative pages (the original lifecycle page now covers only post-2026-09-07 models; pre-2026-09-07 models moved to model-lifecycle-legacy.html).

Table changes, recomputing days_to_eol as of 2026-09-21:

Model EOL Was Now
Claude 3 Haiku 2026-09-10 excluded Removed (past EOL)
Nova Premier v1 2026-09-14 excluded Removed (past EOL)
Nova Sonic v1 2026-09-14 excluded Removed (past EOL)
Nova Canvas v1 2026-09-30 excluded excluded (9 days)
Nova Reel v1 2026-09-30 excluded excluded (9 days)
Claude Sonnet 4 2026-10-14 excluded excluded (23 days)
Jamba 1.5 Large / Mini 2026-11-26 legacy excluded (66 days)
Marengo Embed v2.7 2026-11-30 legacy excluded (70 days)
Claude Opus 4.1 2027-01-08 legacy legacy (109 days) — the only row still outside the zone

Two behavioral rules follow from this:

  • The Nova Canvas carve-out that let it appear "as a Legacy fallback annotation" is dropped — it is inside the exclusion zone, so it must not appear at all.
  • Refresh Cadence now requires a GetFoundationModel / ListFoundationModels check on modelLifecycle.status for any candidate absent from the table, and requires reporting the lifecycle as unverified when neither the API nor the model card is reachable, rather than inferring active from a pricing cache Status column.

skills/gcp-to-aws/references/shared/pricing-cache.md

Restates the affected Status values, drops the EOL'd Nova Premier row from the web-grounding table, and adds a header note that the Status column is a dated snapshot rather than a substitute for the runtime lifecycle check.

Rates for EOL'd models are deliberately retained — they still explain the cost delta to a user already on one. Last updated is not bumped, because no prices were re-verified; only lifecycle statuses changed.

tools/model-id-lint.py

Adds patterns for the three newly-EOL'd IDs so CI catches them as migration targets. Each appears only in the two catalog files today, which are allowlisted.

The other past-EOL IDs (Llama 3.2 / 3.1 405B, Command R / R+, Claude 3 Sonnet, Claude 3.5 Sonnet, Titan Image Generator v2) are deliberately not covered: llm-to-bedrock names them as migration source models in resolve-source-model and run-source-model-baseline, which is a valid use. Extending the lint to source-model references needs a source-vs-target distinction and belongs in a separate change.

Verification

mise run lint and mise run fmt:check both pass, including:

  • lint:model-ids — OK for both plugins
  • drift:check — OK (281 identical, 27 allowlisted across 6 skill trees)
  • shared:check — OK for both plugins
  • lint:md — 0 errors across 896 files

…egacy policy

The lifecycle table was a snapshot dated Aug 24, 2026. Three models have
since passed EOL and three more crossed into the 90-day exclusion zone,
so the plugin would still have recommended or priced models that are
either gone or unusable by the time a migration reaches production.

AWS also split the lifecycle policy on 2026-09-07. Models launched on or
after that date are governed by their model card and may get only a
45-day Legacy period instead of six months, and they never appear in the
pre-2026-09-07 Legacy table at all. A candidate's absence from that table
was previously indistinguishable from "Active", which is now wrong.

Lifecycle doc:
- Document both policies and their authoritative sources.
- Past EOL, moved to Removed: Claude 3 Haiku (2026-09-10), Nova Premier
  v1 and Nova Sonic v1 (2026-09-14).
- legacy -> excluded: Jamba 1.5 Large/Mini (2026-11-26) and Marengo Embed
  v2.7 (2026-11-30) are now inside the 90-day zone. Nova Canvas v1 and
  Nova Reel v1 (2026-09-30) too, so drop the "may appear as a Legacy
  fallback annotation" carve-out for Nova Canvas.
- Require a GetFoundationModel / ListFoundationModels check on
  modelLifecycle.status for any candidate not in the table, and require
  saying "unverified" rather than inferring active from a pricing cache.

Pricing caches: restate the affected Status values, drop the EOL'd Nova
Premier row from the web-grounding table, and state in the header that
the Status column is a dated snapshot rather than a substitute for the
runtime lifecycle check.

model-id-lint.py: cover the three newly-EOL'd IDs so CI catches them as
targets. Each appears only in the two catalog files today, which are
allowlisted. The other past-EOL IDs are deliberately left uncovered:
llm-to-bedrock names them as migration *source* models, which is valid.

Applied identically to the migrate and advisor plugin trees; drift:check,
shared:check, lint:model-ids, and the full lint + fmt:check suites pass.

@leon1418 leon1418 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[🤖 AI review 🤖]

Reviewed a13a42beb1238662102a80addb83049e5c274353 against b30cb39be02d2eae78cb8c482f4018435d4dc988. Requesting changes for one P2 lifecycle-guard regression described inline.

The refreshed dates and launch-date/45-day policy match the current AWS lifecycle documentation. The three changed mirror pairs are byte-identical. Each full Agent Advisor Python suite reports 321 passed / 1 failed; both base suites pass all 322. Each scoped preflight/IAM/pricing suite passes 63 tests. Model-ID lint, cross-plugin drift, Markdown lint, formatting, and diff whitespace checks pass. Additional local probes reproduce the Removed-section guard gap and verify date arithmetic plus bare/profile lint and static-pricing behavior.

GitHub currently reports all nine checks successful; that result does not cover the failing Agent Advisor suites. No live AWS inference or deployment was performed.

Logan Kleier added 2 commits September 18, 2026 20:54
…ection

Moving Nova Sonic v1 (and two other retired models) out of the
pipe-delimited legacy/excluded table into the bulleted **Removed**
list broke _legacy_or_excluded_rows() in test_scoring.py: it only
matched lines starting with "|" and containing "legacy" or
"excluded", so once a model crosses fully past EOL into the Removed
section it silently stops being visible to the drift guard entirely
-- even though the file's own text says Removed models are a
stronger violation than "excluded" ("Never recommend or invoke a
model listed in Removed").

test_drift_mechanism_actually_fires_on_a_known_legacy_model asserted
Nova Sonic v1 was still visible in a matched row; once it moved to
Removed, that assertion failed in both plugin trees -- the reviewer's
report matched exactly (322 -> 321 passed, one new failure).

Extended _legacy_or_excluded_rows() to also capture bulleted entries
between the "**Removed" heading and the following blockquote (the
"AWS page lag" note that closes the section), so a model that ages
out of the table into Removed keeps being caught by
test_no_pool_model_is_legacy_or_excluded. Repointed the table
self-proof at Claude Opus 4.1 (still legacy in the table as of this
file's current content) and added a second, Removed-specific
self-proof anchored on Nova Sonic v1, so the two matching paths (table
vs bulleted Removed section) each have their own proof that the
matcher actually fires rather than vacuously passing.

Verified: reproduced the reviewer's adversarial probe (a pool key
mapped to a Removed model's id fragment) and confirmed it's now
rejected -- it passed on the pre-fix code. Confirmed none of the
lifecycle file's other bulleted lists (policy prose, exclusion-zone
rules, etc.) are swallowed by the Removed-section scan. Full suite:
323/323 passing in both plugin trees (up from 322, the new
Removed-specific self-proof test). fixtures:assert (8 asserters),
cross-plugin-drift (281 identical, 27 allowlisted), git diff --check
all clean.
leon1418
leon1418 previously approved these changes Sep 21, 2026
Logan Kleier added 2 commits September 21, 2026 16:11
The lifecycle table was computed as of 2026-09-17; 4 days on, the "as of" stamps
and the static Days-to-EOL snapshot column were stale. Recomputed
days_to_eol = EOL - 2026-09-21 for every row and bumped the four date stamps
(section header, Notes, Removed header, AWS-page-lag note).

No row changed classification: Nova Canvas/Reel (9d), Claude Sonnet 4 (23d),
Jamba 1.5 Large/Mini (66d), Marengo Embed v2.7 (70d) stay excluded; Claude
Opus 4.1 (109d) stays legacy. Nothing new crossed the 90-day zone or hit EOL
in the interval, so no model moved between the live table and Removed. The
Notes line's "crossed into the zone since the last refresh" wording is
softened to "are inside the zone" since that crossing is no longer this
refresh's news. pricing-cache.md Last-updated is intentionally NOT touched
(no prices re-verified — same rationale as the original PR).

Mirrored byte-identical across both plugin trees. drift:check OK,
lint:model-ids OK, lint:md 0 errors, fmt:check clean.
@herosjourney

Copy link
Copy Markdown
Contributor Author

Recommendation: fold this fix into #304 and close this PR as superseded

Short version: the 45-day-policy content here is correct and needs to land, but this PR edits a path that #304 is about to relocate \u2014 so merging it on the current path would set up a silent clobber. The cleaner move is to port this content into #304 and close this.

Why. #304 introduces a canonical AI lifecycle file at skills/shared/ai/ai-model-lifecycle.md (vendored into each skill via shared:sync), and deletes the gcp-to-aws/references/shared/ai-model-lifecycle.md path this PR edits. Because the two PRs touch different paths, git raises no merge conflict \u2014 whichever lands second silently wins:\n- If #304 lands after this, #304's canonical file (which still carries the stale \u201cLegacy = minimum 6 months\u201d text, with no 45-day / 2026-09-07 policy) silently reappears on main and is fanned out byte-identically into 4 vendored copies, all blessed by shared:check.\n- If this lands after #304, its diff targets a path that no longer exists.\n\nTiming makes the call easy. #304 is ~7 days out; this EOL fix has a ~10-14 day window \u2014 so #304 carries the fix to main with slack, and there\u2019s no urgency reason to merge this early on the soon-to-be-obsolete path.\n\nPlan:\n1. Port this PR\u2019s corrected content \u2014 the 45-day / 2026-09-07 split policy, model-card governance, and the EOL\u2019d rows \u2014 into #304\u2019s canonical skills/shared/ai/ai-model-lifecycle.md, then shared:sync so the vendored copies inherit it. (#304 currently creates this file in both the advisor and migrate trees; if shared:check enforces cross-tree byte-identity, the port has to touch both to keep CI green, even though the migrate tree is being deprecated.)\n2. Verify #304\u2019s canonical file carries the 45-day policy \u2014 not the stale 6-month text \u2014 before #304 merges (tracked as a blocking checklist item on #304).\n3. Close this PR as superseded once #304\u2019s branch carries the content.\n\nDo not close this PR until the port is committed on #304\u2019s branch \u2014 closing early + #304 merging with its current stale file = the fix is lost silently, with CI defending the wrong content. The two actions must happen together.\n

@leon1418 leon1418 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[🤖 AI review 🤖]

Re-reviewed d2473bb4418d588d94200d03ccb3e660c653e618 against ec681ba5d49b039dbf1ef9d87d3304205b824f3d using startups-hybrid-v1: OCR delegation, one fresh independent general reviewer, repository contract review, and the completed evidence gate. All eight changed files were covered, including the four Markdown files omitted by OCR selection. No remaining substantive findings.

The September 21 lifecycle snapshot and 45-day policy match the current primary AWS documentation. The bounded Removed-list guard still captures all ten entries and excludes unrelated bullets. Both Agent Advisor suites pass 323 tests; the scoped preflight/IAM/pricing suites pass 63 tests per tree. The independent reviewer also ran both 323-test suites. Lifecycle/model-ID probes, model-ID lint, drift, Markdown lint, formatting, and whitespace checks pass. GitHub reports nine successful checks; no live AWS invocation or deployment was performed.

The author's proposal to carry this change through #304 remains a separate landing decision. Before closing #307 as superseded, verify that the corrected policy and lifecycle guard are preserved in the replacement paths and consumers.

@leon1418
leon1418 merged commit 39ca3d4 into awslabs:main Sep 22, 2026
9 checks passed
icarthick added a commit to icarthick/startups that referenced this pull request Sep 22, 2026
…l ai-model-lifecycle.md (PR awslabs#304 merge blocker)

herosjourney's blocking checklist (PR awslabs#304): awslabs#307 is being folded into awslabs#304 and
closed as superseded. Because this PR relocates ai-model-lifecycle.md into the
canonical skills/shared/ai/ tree (deleting gcp-to-aws/references/shared/...), no
merge conflict fires — so if awslabs#304 merged with the stale canonical, shared:check
would keep the WRONG content ("Legacy minimum 6 months before EOL", no 45-day
policy) byte-identical across all 6 copies. This carries awslabs#307's fix in.

Two-way merge (herosjourney's ready-to-apply content, comment 5770573792):
kept awslabs#307's FACTS + this PR's canonical FRAMING.
- From awslabs#307: the 45-day / 2026-09-07 model-card policy split and the
  "don't apply the 6-month assumption post-2026-09-07" warning; both reference
  links; EOL'd rows (Claude 3 Haiku, Nova Premier v1, Nova Sonic v1) moved from
  the live table into Removed; restricted / Covered-Model handling; the
  GetFoundationModel unverified-fallback guidance; dates refreshed to Sep 21.
- From this PR: the canonical blockquote header (source-agnostic + edit-here-
  then-shared:sync) and the shared-location wording.

Applied to BOTH tree canonicals (advisor + migrate), then shared:sync propagated
to all vendored copies. All 6 ai-model-lifecycle.md files are byte-identical.

Verified: shared:check OK (both trees), drift:check OK (397 identical),
lint:model-ids OK, fmt:check clean, lint:md 0 errors. Only the 6 lifecycle files
touched. Stale "minimum 6 months before EOL" gone; 45-day policy present in every
copy; Haiku/Premier/Sonic in Removed.

awslabs#307 to be closed as superseded only after this lands.
icarthick added a commit to icarthick/startups that referenced this pull request Sep 22, 2026
…ng-cache + phase-status conflicts)

main advanced 39 commits (incl. awslabs#307 Bedrock lifecycle refresh merged as 39ca3d4,
awslabs#291 heroku decision gate, awslabs#310/awslabs#314 report + plan-writer). Conflict resolution:

- ai-model-lifecycle.md (gcp vendored, both trees): took OURS — same awslabs#307 facts,
  but at this PR's correct vendored/canonical path with the canonical header
  (main edited the old gcp-to-aws/references/shared/ path this PR deletes).
- pricing-cache.md (gcp shared, both trees): hybrid — kept main's FRESHER lifecycle
  facts (Nova Canvas/Reel excluded; Nova Sonic past-EOL) but rewrote the path
  references from shared/ai-model-lifecycle.md -> vendored/ai/ai-model-lifecycle.md
  (this PR relocated the file; shared/ path would dangle).
- phase-status.schema.json (shared + heroku vendored, both trees): took THEIRS
  (main's newer run_mode description prose); run_mode key preserved.
- model-id-lint.py (both trees): updated the Haiku/Premier/Sonic allowlist paths
  from the deleted skills/gcp-to-aws/references/shared/ai-model-lifecycle.md to the
  canonical skills/shared/ai/ai-model-lifecycle.md (vendored copies inherit via
  canonicalize()) — main's linter didn't know this PR moved the file.

Post-merge shared:sync re-propagated the shared phase-status schema into the azure
vendored copy (canonical-staleness trap).

Verified green both trees: shared:check OK, drift:check OK (398 identical),
frontmatter 7 files, asserters 15 PASS, 92 validator unit tests pass, mise run build
all 16 tasks clean (lint:model-ids OK, fmt clean, lint:md 0 errors, all security
scanners clean). All six ai-model-lifecycle.md copies byte-identical.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants