docs(advisor): refresh README/AGENTS/setup for recent capabilities - #301
Conversation
…rges Doc-only. Reflects capability changes merged into aws-startup-advisor since mid-August that the top-level docs never picked up, plus two pre-existing inaccuracies found in the same pass: - gcp-to-aws: OpenAI sources now land on the same GPT model on Bedrock when one exists (awslabs#210, awslabs#250, awslabs#237), not just a Claude/Nova cross-family swap. Discover can pull real spend from the OpenAI Admin API, consent-gated (awslabs#236). GCP Document AI / Vision / Speech-to-Text are detected and routed to AWS traditional-AI services (awslabs#240, awslabs#241). - Both migration skills: commitment-discount guidance (shared RI/SP eligibility matrix, three-state model, dedicated report section — awslabs#266, awslabs#270, awslabs#272, awslabs#277) was undocumented anywhere in the top-level docs. - heroku-to-aws: Generate emits `baseline.tf`, an account security baseline, alongside the app Terraform (awslabs#261). - setup.md: agent-advisor was missing Lambda MicroVMs; prompt-library counts were wrong (30 prompts / 4 agents named "Migration" -> actually 29 prompts / 5 agents, none called "Migration") and README's agent list was incomplete. Pre-existing, not tied to a specific recent PR. - knowledge-base-for-startups: bumped the skill's own "Last updated" date to 2026-09-09 to match the offers refresh in awslabs#281 (the date drives its 6-month staleness nudge to users). Verified: markdownlint clean. No skill/schema/tool files touched.
Address review nits on awslabs#301: note Config/Security Hub when compliance is declared, attribute baseline.tf to both migration skills, and say OpenRouter/LiteLLM apply only when the underlying model is OpenAI. Co-authored-by: Cursor <cursoragent@cursor.com>
|
Pushed a follow-up for the review nits:
|
leon1418
left a comment
There was a problem hiding this comment.
[🤖 AI review 🤖]
Reviewed e3dd985 against the committed discovery, estimate, and generation contracts. Three documentation claims need correction; details are inline.
All nine GitHub checks and a local dprint check of the four changed files pass. This review validates the documentation against source contracts; it does not claim a deployed migration test.
…ust IaC Design maps a target instance class whether Discover's source was Terraform/IaC or live discovery (e.g. Cloud SQL settings.tier via discover-live.md); estimate-infra.md's /month sizing rule keys off that mapped instance class, not the discovery source. The prior wording said 'only when IaC supplies an instance class', which would tell an agent to withhold a valid dollar-sized RI/SP estimate from a user who ran live discovery instead of providing Terraform.
|
Pushed a second follow-up (
All three of your comments checked out on verification; thanks for catching that I'd only closed two of three. |
leon1418
left a comment
There was a problem hiding this comment.
[🤖 AI review 🤖]
Completed the follow-up review of c039bed against base dd7096b. The database-sizing finding is fixed in README and AGENTS. Two P2 documentation issues remain in the original threads: the unrestricted baseline.tf guarantee and the broader-than-implemented Config/Security Hub compliance condition. Please correct both claims consistently in README, AGENTS, and setup before merging.
Validation: changed-file dprint check --incremental=false, git diff --check, and cross-plugin drift pass (275 identical, 25 allowlisted). I compared 33 relevant source pairs, including the applicable allowlisted differences. All nine current GitHub checks pass. This is a source-contract review; no migration or deployment was executed.
Two documentation-accuracy findings across README.md, AGENTS.md, and
setup.md:
1. baseline.tf is scoped to gcp-to-aws's infrastructure-generation
route, not unconditional. generate.md only dispatches
generate-artifacts-infra.md (which is the only place baseline.tf is
written) when generation-infra.json AND aws-design.json both
exist. AI-only runs instead emit bedrock_monitoring.tf
(generate-artifacts-ai.md Step 3F, "always" for AI paths, but never
baseline.tf); billing-only runs emit skeleton Terraform. All three
docs stated the account security baseline unconditionally for
gcp-to-aws, which overpromises the AI-only and billing-only routes.
heroku-to-aws has no such caveat -- it has a single route, and
baseline.tf really is unconditional there -- so only the gcp-to-aws
lines needed the route qualifier; the heroku-to-aws lines just
needed the compliance-list fix below.
2. Name the four compliance frameworks that gate Config + Security
Hub, instead of "when compliance is declared." Both generators
(gcp-to-aws generate-artifacts-infra.md Step 1.5, heroku-to-aws
generate-terraform.md Step 1.5) enable the Config/Security Hub
compliance-conditional section only when preferences.json.compliance
contains soc2, pci, hipaa, or fedramp. gcp-to-aws's own Step 5
self-check explicitly requires baseline.tf to contain NO
aws_config_*/aws_securityhub_* resources when compliance is
empty, absent, or GDPR-only. The prior wording ("when compliance is
declared") told a GDPR-only user they'd get controls the generator
will not emit.
Verified both claims by reading generate.md's route-dispatch
conditions, generate-artifacts-infra.md's Step 1.5 + Step 5
self-check, generate-artifacts-ai.md's Step 3F, and
generate-terraform.md's Step 1.5/6, in the advisor plugin tree (the
only tree these three docs live in). No executable test covers prose
accuracy in these files; verification is fmt/lint/drift-check plus a
grep sweep confirming no other instance of either vague phrasing
remains in the three files. dprint check, markdownlint (0 errors),
cross-plugin-drift (281 identical, 27 allowlisted), fixtures:assert
(8 asserters) all green.
leon1418
left a comment
There was a problem hiding this comment.
[🤖 AI review 🤖]
Completed the current-head review of 75fa6d7 against ade57ad. The three original substantive findings are addressed: AI-only/billing-only runs no longer promise the account baseline; Config/Security Hub use the four implemented compliance frameworks; and database sizing still accepts a Design-mapped class from IaC or live discovery above the $50/month threshold. No new substantive finding from the full documentation diff and relevant producer/consumer review.
Local changed-file formatting, patch-whitespace, and cross-plugin drift checks pass (281 identical, 27 allowlisted); 38 relevant mirror pairs were inspected, including applicable allowlisted differences. All nine current GitHub checks pass. No migration or deployment was executed. This COMMENT records verification; human approval and the remaining conversation/review-state requirements are still outstanding.
leon1418
left a comment
There was a problem hiding this comment.
[🤖 AI review 🤖]
Completed the startups-hybrid-v1 review of 362623c against ec681ba: actual OCR delegation, a fresh independent general reviewer, the repository review, and explicit candidate verification.
One verified P2 remains in the existing compliance thread: GCP Clarify produces design_constraints.compliance.value, while the baseline generator reads root compliance. The mismatch predates this PR, but the new automatic Config/Security Hub guarantee is not ensured for the standard producer output. Qualify that GCP claim or align the handoff before merging. The earlier route-scope and live database-sizing corrections remain intact.
All four changed Markdown files were reviewed despite OCR excluding their extension. Local formatting, whitespace and cross-plugin drift checks pass; the independent source probes and all nine current GitHub checks pass. No live migration or deployment was executed.
…ndoff Generate adds those controls only when root preferences.json.compliance contains soc2, pci, hipaa, or fedramp. Clarify stores the Q2 answer at design_constraints.compliance.value and leaves the root field unset, so a standard Clarify answer takes the base-only baseline. Heroku still keys off global.compliance, which Clarify does write. Co-authored-by: Cursor <cursoragent@cursor.com>
|
Qualified the GCP Config/Security Hub claim in Confirmed the handoff the review described: Clarify Step 5 writes
The producer/consumer mismatch itself is pre-existing and still unfixed; this only stops the docs from guaranteeing controls the current handoff does not emit. |
leon1418
left a comment
There was a problem hiding this comment.
[🤖 AI review 🤖]
Completed the hybrid/convergence follow-up at c5830aa (base ec681ba). F1 is fixed: the GCP compliance limitation is now accurately qualified. No generator repair is required for that documentation fix.
One P2 documentation issue remains, missed in my prior review, not introduced by this fix: billing-only RI/SP and dedicated-section promises exceed that route's estimator/report contract. The consolidated correction is limited to README, AGENTS, and setup:
- Do not promise commitment options or a dedicated RI/SP section for no-CUD billing-only inputs.
- Describe active-CUD billing-only guidance as conditional percentage comparison, without implying instance sizing or optimization rows.
- Keep infrastructure/Heroku guidance and the valid compliance qualification intact.
OCR delegation, both independent impact maps, candidate verification and the convergence gate are complete. Local formatting, whitespace, drift and source-contract checks pass; all nine current CI checks pass. No migration or deployment was executed.
…efresh-sep-2026 # Conflicts: # advisor/README.md # advisor/plugins/aws-startup-advisor/setup.md
…overy Bicep, ARM template, and live az CLI discovery are not yet built (their extractors are planned follow-ups; a Bicep/ARM/live-only workspace halts rather than guessing). Stop advertising them as supported discovery sources and change the phase count from 7 to 6 across README, AGENTS, setup, and the azure-to-aws SKILL description (plus the Migration-mode default line that contradicted the fix).
…nly RI/SP - setup.md: add contextual-offers-for-startups to the install-confirmation list and 'what these skills do', bump 10 -> 11 installed skills (jkzietz). - README/AGENTS/setup: qualify billing-only commitment guidance — a no-CUD billing-only run gets neither the dedicated RI/SP section nor percentage options; commitment guidance appears only as a conditional CUD-vs-AWS comparison when the billing export shows active commitments (leon1418 P2).
setup.md install-success message and README intro said 'GCP or Heroku'; Azure is now supported, so both now read 'GCP, Azure, or Heroku'.
Status: all review findings addressed at
|
| Finding | Reviewer | Fixed in | Notes |
|---|---|---|---|
[P2] Scope baseline.tf to the infrastructure-generation route |
leon1418 | 75fa6d7d |
AI-only/billing-only runs no longer promise the account baseline. Reviewer replied "Verified… addressed." |
| [P2] Name the four compliance frameworks (soc2/pci/hipaa/fedramp) | leon1418 | 75fa6d7d |
Replaced "when compliance is declared" across README/AGENTS/setup. Reviewer replied "addressed." |
| [P2] Allow DB sizing from live discovery, not just IaC | leon1418 | e8d23ba5 |
$50/mo rule keys off a Design-mapped instance class from IaC or live discovery. Reviewer replied "addressed." |
| [F1] Qualify the GCP Q2→Generate compliance handoff | leon1418 | c5830aa9 |
Documented that Clarify writes design_constraints.compliance.value, not root compliance. Reviewer replied "fixes F1." |
| [P2] Qualify billing-only commitment / RI-SP guidance | leon1418 | 6837b50a |
A no-CUD billing-only run gets neither the dedicated RI/SP section nor percentage options — only a conditional CUD-vs-AWS comparison when the export shows active CUDs. Corrected in README, AGENTS, and the GCP setup summary. |
| There are 11 skills in the plugin now | jkzietz | 6837b50a |
Added contextual-offers-for-startups to the install-confirmation list and "What these skills do"; count 10 → 11. |
Also addressed while here (Azure had shipped to main but the docs lagged):
- Corrected
azure-to-awsfrom 7 phases → 6 phases and from "Terraform/Bicep/ARM +azcapture" to Terraform (azurerm_*) + app code — Bicep, ARM, and liveazdiscovery are called out as not-yet-available follow-ups (491f7333). - Included Azure in the "GCP or Heroku" migration-source phrasing in README and setup (
a9df5bbe).
No open comments remain. @leon1418 — the last automated review predates the billing-only fix; a re-review against a9df5bbe would confirm. Reviewers, please resolve the threads if they look good to you.
Problem
advisor/README.md,advisor/AGENTS.md, andsetup.mdwere last touched Sep 3 for MCP-provisioning wording, but several capabilities merged since mid-August were never reflected there: same-model OpenAI-on-Bedrock routing (#210, #250, #237), OpenAI Admin API discovery (#236), GCP Document AI/Vision/Speech-to-Text detection (#240, #241), Savings Plan/RI eligibility + report section (#266, #270, #272, #277), and account-securitybaseline.tfon both migration skills (#261 and gcp Generate). Same pass also found pre-existing inaccuracies insetup.md(wrong prompt/agent counts, missing Lambda MicroVMs) and a stale "Last updated" date on the knowledge-base skill that drives its own staleness nudge to users.Separately, the docs over-stated the
azure-to-awsskill: they described it as a 7-phase workflow that discovers from Terraform/Bicep/ARM plus a liveazCLI capture. In reality only Terraform (azurerm_*) + application-code discovery is built today — the Bicep, ARM-template, and live-azextractors are planned follow-ups (a Bicep/ARM/live-only workspace halts rather than guessing), and the backbone is 6 phases, consistent with the siblinggcp-to-aws/heroku-to-awsskills.Solution
Doc-only update to
advisor/README.md,advisor/AGENTS.md,advisor/plugins/aws-startup-advisor/setup.md, andknowledge-base-for-startups/SKILL.md'sLast updateddate. No skill, schema, or tool files touched — nothing here changes behavior, only what the docs claim about existing behavior.Follow-up on review:
baseline.tfnow notes Config/Security Hub when compliance is declared and is attributed to bothgcp-to-awsandheroku-to-aws; OpenRouter/LiteLLM wording is limited to OpenAI models that have a Bedrock same-model target.Azure correction: changed
azure-to-awsfrom 7 phases to 6 phases and from "Terraform/Bicep/ARM +azcapture" to Terraform (azurerm_*) + app code (billing exports as fallback) across README, AGENTS, setup, and theazure-to-awsSKILL description — with Bicep, ARM templates, and liveazcapture explicitly called out as not-yet-available follow-ups. Also corrected the SKILL body'sMigration modedefault line, which still claimed "IaC (Terraform/Bicep/ARM) fully supported" and would otherwise have contradicted the fix. Branch brought up to date withmain.Type of Change
Team Folder
advisor/migrate/solution-architecture/Checklist
mise run buildlocally and it passes — doc-only change; randprint checkon the touched files instead (no code/schema touched)By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.