fix(bedrock): refresh model lifecycle statuses and adopt the 45-day Legacy policy - #307
Conversation
…egacy policy The lifecycle table was a snapshot dated Aug 24, 2026. Three models have since passed EOL and three more crossed into the 90-day exclusion zone, so the plugin would still have recommended or priced models that are either gone or unusable by the time a migration reaches production. AWS also split the lifecycle policy on 2026-09-07. Models launched on or after that date are governed by their model card and may get only a 45-day Legacy period instead of six months, and they never appear in the pre-2026-09-07 Legacy table at all. A candidate's absence from that table was previously indistinguishable from "Active", which is now wrong. Lifecycle doc: - Document both policies and their authoritative sources. - Past EOL, moved to Removed: Claude 3 Haiku (2026-09-10), Nova Premier v1 and Nova Sonic v1 (2026-09-14). - legacy -> excluded: Jamba 1.5 Large/Mini (2026-11-26) and Marengo Embed v2.7 (2026-11-30) are now inside the 90-day zone. Nova Canvas v1 and Nova Reel v1 (2026-09-30) too, so drop the "may appear as a Legacy fallback annotation" carve-out for Nova Canvas. - Require a GetFoundationModel / ListFoundationModels check on modelLifecycle.status for any candidate not in the table, and require saying "unverified" rather than inferring active from a pricing cache. Pricing caches: restate the affected Status values, drop the EOL'd Nova Premier row from the web-grounding table, and state in the header that the Status column is a dated snapshot rather than a substitute for the runtime lifecycle check. model-id-lint.py: cover the three newly-EOL'd IDs so CI catches them as targets. Each appears only in the two catalog files today, which are allowlisted. The other past-EOL IDs are deliberately left uncovered: llm-to-bedrock names them as migration *source* models, which is valid. Applied identically to the migrate and advisor plugin trees; drift:check, shared:check, lint:model-ids, and the full lint + fmt:check suites pass.
leon1418
left a comment
There was a problem hiding this comment.
[🤖 AI review 🤖]
Reviewed a13a42beb1238662102a80addb83049e5c274353 against b30cb39be02d2eae78cb8c482f4018435d4dc988. Requesting changes for one P2 lifecycle-guard regression described inline.
The refreshed dates and launch-date/45-day policy match the current AWS lifecycle documentation. The three changed mirror pairs are byte-identical. Each full Agent Advisor Python suite reports 321 passed / 1 failed; both base suites pass all 322. Each scoped preflight/IAM/pricing suite passes 63 tests. Model-ID lint, cross-plugin drift, Markdown lint, formatting, and diff whitespace checks pass. Additional local probes reproduce the Removed-section guard gap and verify date arithmetic plus bare/profile lint and static-pricing behavior.
GitHub currently reports all nine checks successful; that result does not cover the failing Agent Advisor suites. No live AWS inference or deployment was performed.
…ection
Moving Nova Sonic v1 (and two other retired models) out of the
pipe-delimited legacy/excluded table into the bulleted **Removed**
list broke _legacy_or_excluded_rows() in test_scoring.py: it only
matched lines starting with "|" and containing "legacy" or
"excluded", so once a model crosses fully past EOL into the Removed
section it silently stops being visible to the drift guard entirely
-- even though the file's own text says Removed models are a
stronger violation than "excluded" ("Never recommend or invoke a
model listed in Removed").
test_drift_mechanism_actually_fires_on_a_known_legacy_model asserted
Nova Sonic v1 was still visible in a matched row; once it moved to
Removed, that assertion failed in both plugin trees -- the reviewer's
report matched exactly (322 -> 321 passed, one new failure).
Extended _legacy_or_excluded_rows() to also capture bulleted entries
between the "**Removed" heading and the following blockquote (the
"AWS page lag" note that closes the section), so a model that ages
out of the table into Removed keeps being caught by
test_no_pool_model_is_legacy_or_excluded. Repointed the table
self-proof at Claude Opus 4.1 (still legacy in the table as of this
file's current content) and added a second, Removed-specific
self-proof anchored on Nova Sonic v1, so the two matching paths (table
vs bulleted Removed section) each have their own proof that the
matcher actually fires rather than vacuously passing.
Verified: reproduced the reviewer's adversarial probe (a pool key
mapped to a Removed model's id fragment) and confirmed it's now
rejected -- it passed on the pre-fix code. Confirmed none of the
lifecycle file's other bulleted lists (policy prose, exclusion-zone
rules, etc.) are swallowed by the Removed-section scan. Full suite:
323/323 passing in both plugin trees (up from 322, the new
Removed-specific self-proof test). fixtures:assert (8 asserters),
cross-plugin-drift (281 identical, 27 allowlisted), git diff --check
all clean.
The lifecycle table was computed as of 2026-09-17; 4 days on, the "as of" stamps and the static Days-to-EOL snapshot column were stale. Recomputed days_to_eol = EOL - 2026-09-21 for every row and bumped the four date stamps (section header, Notes, Removed header, AWS-page-lag note). No row changed classification: Nova Canvas/Reel (9d), Claude Sonnet 4 (23d), Jamba 1.5 Large/Mini (66d), Marengo Embed v2.7 (70d) stay excluded; Claude Opus 4.1 (109d) stays legacy. Nothing new crossed the 90-day zone or hit EOL in the interval, so no model moved between the live table and Removed. The Notes line's "crossed into the zone since the last refresh" wording is softened to "are inside the zone" since that crossing is no longer this refresh's news. pricing-cache.md Last-updated is intentionally NOT touched (no prices re-verified — same rationale as the original PR). Mirrored byte-identical across both plugin trees. drift:check OK, lint:model-ids OK, lint:md 0 errors, fmt:check clean.
Recommendation: fold this fix into #304 and close this PR as supersededShort version: the 45-day-policy content here is correct and needs to land, but this PR edits a path that #304 is about to relocate \u2014 so merging it on the current path would set up a silent clobber. The cleaner move is to port this content into #304 and close this. Why. #304 introduces a canonical AI lifecycle file at |
leon1418
left a comment
There was a problem hiding this comment.
[🤖 AI review 🤖]
Re-reviewed d2473bb4418d588d94200d03ccb3e660c653e618 against ec681ba5d49b039dbf1ef9d87d3304205b824f3d using startups-hybrid-v1: OCR delegation, one fresh independent general reviewer, repository contract review, and the completed evidence gate. All eight changed files were covered, including the four Markdown files omitted by OCR selection. No remaining substantive findings.
The September 21 lifecycle snapshot and 45-day policy match the current primary AWS documentation. The bounded Removed-list guard still captures all ten entries and excludes unrelated bullets. Both Agent Advisor suites pass 323 tests; the scoped preflight/IAM/pricing suites pass 63 tests per tree. The independent reviewer also ran both 323-test suites. Lifecycle/model-ID probes, model-ID lint, drift, Markdown lint, formatting, and whitespace checks pass. GitHub reports nine successful checks; no live AWS invocation or deployment was performed.
The author's proposal to carry this change through #304 remains a separate landing decision. Before closing #307 as superseded, verify that the corrected policy and lifecycle guard are preserved in the replacement paths and consumers.
…l ai-model-lifecycle.md (PR awslabs#304 merge blocker) herosjourney's blocking checklist (PR awslabs#304): awslabs#307 is being folded into awslabs#304 and closed as superseded. Because this PR relocates ai-model-lifecycle.md into the canonical skills/shared/ai/ tree (deleting gcp-to-aws/references/shared/...), no merge conflict fires — so if awslabs#304 merged with the stale canonical, shared:check would keep the WRONG content ("Legacy minimum 6 months before EOL", no 45-day policy) byte-identical across all 6 copies. This carries awslabs#307's fix in. Two-way merge (herosjourney's ready-to-apply content, comment 5770573792): kept awslabs#307's FACTS + this PR's canonical FRAMING. - From awslabs#307: the 45-day / 2026-09-07 model-card policy split and the "don't apply the 6-month assumption post-2026-09-07" warning; both reference links; EOL'd rows (Claude 3 Haiku, Nova Premier v1, Nova Sonic v1) moved from the live table into Removed; restricted / Covered-Model handling; the GetFoundationModel unverified-fallback guidance; dates refreshed to Sep 21. - From this PR: the canonical blockquote header (source-agnostic + edit-here- then-shared:sync) and the shared-location wording. Applied to BOTH tree canonicals (advisor + migrate), then shared:sync propagated to all vendored copies. All 6 ai-model-lifecycle.md files are byte-identical. Verified: shared:check OK (both trees), drift:check OK (397 identical), lint:model-ids OK, fmt:check clean, lint:md 0 errors. Only the 6 lifecycle files touched. Stale "minimum 6 months before EOL" gone; 45-day policy present in every copy; Haiku/Premier/Sonic in Removed. awslabs#307 to be closed as superseded only after this lands.
…ng-cache + phase-status conflicts) main advanced 39 commits (incl. awslabs#307 Bedrock lifecycle refresh merged as 39ca3d4, awslabs#291 heroku decision gate, awslabs#310/awslabs#314 report + plan-writer). Conflict resolution: - ai-model-lifecycle.md (gcp vendored, both trees): took OURS — same awslabs#307 facts, but at this PR's correct vendored/canonical path with the canonical header (main edited the old gcp-to-aws/references/shared/ path this PR deletes). - pricing-cache.md (gcp shared, both trees): hybrid — kept main's FRESHER lifecycle facts (Nova Canvas/Reel excluded; Nova Sonic past-EOL) but rewrote the path references from shared/ai-model-lifecycle.md -> vendored/ai/ai-model-lifecycle.md (this PR relocated the file; shared/ path would dangle). - phase-status.schema.json (shared + heroku vendored, both trees): took THEIRS (main's newer run_mode description prose); run_mode key preserved. - model-id-lint.py (both trees): updated the Haiku/Premier/Sonic allowlist paths from the deleted skills/gcp-to-aws/references/shared/ai-model-lifecycle.md to the canonical skills/shared/ai/ai-model-lifecycle.md (vendored copies inherit via canonicalize()) — main's linter didn't know this PR moved the file. Post-merge shared:sync re-propagated the shared phase-status schema into the azure vendored copy (canonical-staleness trap). Verified green both trees: shared:check OK, drift:check OK (398 identical), frontmatter 7 files, asserters 15 PASS, 92 validator unit tests pass, mise run build all 16 tasks clean (lint:model-ids OK, fmt clean, lint:md 0 errors, all security scanners clean). All six ai-model-lifecycle.md copies byte-identical.
Problem
The Bedrock lifecycle table in both plugin trees is a snapshot dated Aug 24, 2026. Two things have gone stale since:
Changes
Applied identically to
migrate/plugins/migration-to-aws/andadvisor/plugins/aws-startup-advisor/.skills/gcp-to-aws/references/shared/ai-model-lifecycle.mdDocuments both policies and links the two authoritative pages (the original lifecycle page now covers only post-2026-09-07 models; pre-2026-09-07 models moved to
model-lifecycle-legacy.html).Table changes, recomputing
days_to_eolas of 2026-09-21:Two behavioral rules follow from this:
GetFoundationModel/ListFoundationModelscheck onmodelLifecycle.statusfor any candidate absent from the table, and requires reporting the lifecycle as unverified when neither the API nor the model card is reachable, rather than inferringactivefrom a pricing cacheStatuscolumn.skills/gcp-to-aws/references/shared/pricing-cache.mdRestates the affected
Statusvalues, drops the EOL'd Nova Premier row from the web-grounding table, and adds a header note that theStatuscolumn is a dated snapshot rather than a substitute for the runtime lifecycle check.Rates for EOL'd models are deliberately retained — they still explain the cost delta to a user already on one.
Last updatedis not bumped, because no prices were re-verified; only lifecycle statuses changed.tools/model-id-lint.pyAdds patterns for the three newly-EOL'd IDs so CI catches them as migration targets. Each appears only in the two catalog files today, which are allowlisted.
The other past-EOL IDs (Llama 3.2 / 3.1 405B, Command R / R+, Claude 3 Sonnet, Claude 3.5 Sonnet, Titan Image Generator v2) are deliberately not covered:
llm-to-bedrocknames them as migration source models inresolve-source-modelandrun-source-model-baseline, which is a valid use. Extending the lint to source-model references needs a source-vs-target distinction and belongs in a separate change.Verification
mise run lintandmise run fmt:checkboth pass, including:lint:model-ids— OK for both pluginsdrift:check— OK (281 identical, 27 allowlisted across 6 skill trees)shared:check— OK for both pluginslint:md— 0 errors across 896 files