Skip to content

docs(advisor): refresh README/AGENTS/setup for recent capabilities - #301

Merged
leon1418 merged 15 commits into
awslabs:mainfrom
herosjourney:docs/advisor-readme-refresh-sep-2026
Sep 24, 2026
Merged

leon1418 merged 15 commits into
awslabs:mainfrom
herosjourney:docs/advisor-readme-refresh-sep-2026

Conversation

@herosjourney

@herosjourney herosjourney commented Sep 15, 2026 •

Copy link
Copy Markdown
Contributor

Problem

advisor/README.md, advisor/AGENTS.md, and setup.md were last touched Sep 3 for MCP-provisioning wording, but several capabilities merged since mid-August were never reflected there: same-model OpenAI-on-Bedrock routing (#210, #250, #237), OpenAI Admin API discovery (#236), GCP Document AI/Vision/Speech-to-Text detection (#240, #241), Savings Plan/RI eligibility + report section (#266, #270, #272, #277), and account-security baseline.tf on both migration skills (#261 and gcp Generate). Same pass also found pre-existing inaccuracies in setup.md (wrong prompt/agent counts, missing Lambda MicroVMs) and a stale "Last updated" date on the knowledge-base skill that drives its own staleness nudge to users.

Separately, the docs over-stated the azure-to-aws skill: they described it as a 7-phase workflow that discovers from Terraform/Bicep/ARM plus a live az CLI capture. In reality only Terraform (azurerm_*) + application-code discovery is built today — the Bicep, ARM-template, and live-az extractors are planned follow-ups (a Bicep/ARM/live-only workspace halts rather than guessing), and the backbone is 6 phases, consistent with the sibling gcp-to-aws / heroku-to-aws skills.

Solution

Doc-only update to advisor/README.md, advisor/AGENTS.md, advisor/plugins/aws-startup-advisor/setup.md, and knowledge-base-for-startups/SKILL.md's Last updated date. No skill, schema, or tool files touched — nothing here changes behavior, only what the docs claim about existing behavior.

Follow-up on review: baseline.tf now notes Config/Security Hub when compliance is declared and is attributed to both gcp-to-aws and heroku-to-aws; OpenRouter/LiteLLM wording is limited to OpenAI models that have a Bedrock same-model target.

Azure correction: changed azure-to-aws from 7 phases to 6 phases and from "Terraform/Bicep/ARM + az capture" to Terraform (azurerm_*) + app code (billing exports as fallback) across README, AGENTS, setup, and the azure-to-aws SKILL description — with Bicep, ARM templates, and live az capture explicitly called out as not-yet-available follow-ups. Also corrected the SKILL body's Migration mode default line, which still claimed "IaC (Terraform/Bicep/ARM) fully supported" and would otherwise have contradicted the fix. Branch brought up to date with main.

Type of Change

  • New plugin/power/tool
  • Bug fix
  • Enhancement to existing content
  • Documentation update
  • Guardrail/CI update

Team Folder

  • advisor/
  • migrate/
  • solution-architecture/
  • Other: ___

Checklist

  • I have read the ../CONTRIBUTING.md guidelines
  • My changes do not include hardcoded secrets, credentials, or internal-only content
  • I have run mise run build locally and it passes — doc-only change; ran dprint check on the touched files instead (no code/schema touched)
  • I have updated documentation if needed
  • My changes are scoped to my team's folder only

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

…rges

Doc-only. Reflects capability changes merged into aws-startup-advisor since
mid-August that the top-level docs never picked up, plus two pre-existing
inaccuracies found in the same pass:

- gcp-to-aws: OpenAI sources now land on the same GPT model on Bedrock when
  one exists (awslabs#210, awslabs#250, awslabs#237), not just a Claude/Nova cross-family swap.
  Discover can pull real spend from the OpenAI Admin API, consent-gated
  (awslabs#236). GCP Document AI / Vision / Speech-to-Text are detected and routed
  to AWS traditional-AI services (awslabs#240, awslabs#241).
- Both migration skills: commitment-discount guidance (shared RI/SP
  eligibility matrix, three-state model, dedicated report section — awslabs#266,
  awslabs#270, awslabs#272, awslabs#277) was undocumented anywhere in the top-level docs.
- heroku-to-aws: Generate emits `baseline.tf`, an account security baseline,
  alongside the app Terraform (awslabs#261).
- setup.md: agent-advisor was missing Lambda MicroVMs; prompt-library counts
  were wrong (30 prompts / 4 agents named "Migration" -> actually 29 prompts
  / 5 agents, none called "Migration") and README's agent list was
  incomplete. Pre-existing, not tied to a specific recent PR.
- knowledge-base-for-startups: bumped the skill's own "Last updated" date to
  2026-09-09 to match the offers refresh in awslabs#281 (the date drives its
  6-month staleness nudge to users).

Verified: markdownlint clean. No skill/schema/tool files touched.
@herosjourney
herosjourney requested a review from a team as a code owner September 15, 2026 17:56
Address review nits on awslabs#301: note Config/Security Hub when compliance
is declared, attribute baseline.tf to both migration skills, and say
OpenRouter/LiteLLM apply only when the underlying model is OpenAI.

Co-authored-by: Cursor <cursoragent@cursor.com>
@herosjourney

Copy link
Copy Markdown
Contributor Author

Pushed a follow-up for the review nits:

  • baseline.tf: note Config + Security Hub when compliance is declared; attribute the baseline to both gcp-to-aws and heroku-to-aws (gcp Generate already emits it).
  • Same-model wording: OpenRouter/LiteLLM parenthetical now says OpenAI models reached via those gateways, not “models” in general.
  • Checklist: clarified that this is doc-only / dprint check on touched files rather than a full mise run build claim.

@leon1418 leon1418 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[🤖 AI review 🤖]

Reviewed e3dd985 against the committed discovery, estimate, and generation contracts. Three documentation claims need correction; details are inline.

All nine GitHub checks and a local dprint check of the four changed files pass. This review validates the documentation against source contracts; it does not claim a deployed migration test.

Comment thread advisor/README.md Outdated
Comment thread advisor/README.md Outdated
Comment thread advisor/AGENTS.md Outdated
…ust IaC

Design maps a target instance class whether Discover's source was
Terraform/IaC or live discovery (e.g. Cloud SQL settings.tier via
discover-live.md); estimate-infra.md's /month sizing rule keys off
that mapped instance class, not the discovery source. The prior wording
said 'only when IaC supplies an instance class', which would tell an
agent to withhold a valid dollar-sized RI/SP estimate from a user who
ran live discovery instead of providing Terraform.
@herosjourney

Copy link
Copy Markdown
Contributor Author

Pushed a second follow-up (e8d23ba5) for the third nit, which my prior "pushed a follow-up" comment missed:

  • Database RI/SP sizing: the "only when IaC supplies an instance class" wording was wrong — estimate-infra.md's $50/month rule keys off Design having mapped a target instance class, and Design gets that from live discovery (Cloud SQL settings.tier via discover-live.md) just as much as from Terraform. Fixed in both AGENTS.md and README.md — README carried the identical narrower framing that wasn't flagged inline but needed the same fix.

All three of your comments checked out on verification; thanks for catching that I'd only closed two of three.

@leon1418 leon1418 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[🤖 AI review 🤖]

Completed the follow-up review of c039bed against base dd7096b. The database-sizing finding is fixed in README and AGENTS. Two P2 documentation issues remain in the original threads: the unrestricted baseline.tf guarantee and the broader-than-implemented Config/Security Hub compliance condition. Please correct both claims consistently in README, AGENTS, and setup before merging.

Validation: changed-file dprint check --incremental=false, git diff --check, and cross-plugin drift pass (275 identical, 25 allowlisted). I compared 33 relevant source pairs, including the applicable allowlisted differences. All nine current GitHub checks pass. This is a source-contract review; no migration or deployment was executed.

Logan Kleier added 2 commits September 18, 2026 20:43
Two documentation-accuracy findings across README.md, AGENTS.md, and
setup.md:

1. baseline.tf is scoped to gcp-to-aws's infrastructure-generation
   route, not unconditional. generate.md only dispatches
   generate-artifacts-infra.md (which is the only place baseline.tf is
   written) when generation-infra.json AND aws-design.json both
   exist. AI-only runs instead emit bedrock_monitoring.tf
   (generate-artifacts-ai.md Step 3F, "always" for AI paths, but never
   baseline.tf); billing-only runs emit skeleton Terraform. All three
   docs stated the account security baseline unconditionally for
   gcp-to-aws, which overpromises the AI-only and billing-only routes.
   heroku-to-aws has no such caveat -- it has a single route, and
   baseline.tf really is unconditional there -- so only the gcp-to-aws
   lines needed the route qualifier; the heroku-to-aws lines just
   needed the compliance-list fix below.

2. Name the four compliance frameworks that gate Config + Security
   Hub, instead of "when compliance is declared." Both generators
   (gcp-to-aws generate-artifacts-infra.md Step 1.5, heroku-to-aws
   generate-terraform.md Step 1.5) enable the Config/Security Hub
   compliance-conditional section only when preferences.json.compliance
   contains soc2, pci, hipaa, or fedramp. gcp-to-aws's own Step 5
   self-check explicitly requires baseline.tf to contain NO
   aws_config_*/aws_securityhub_* resources when compliance is
   empty, absent, or GDPR-only. The prior wording ("when compliance is
   declared") told a GDPR-only user they'd get controls the generator
   will not emit.

Verified both claims by reading generate.md's route-dispatch
conditions, generate-artifacts-infra.md's Step 1.5 + Step 5
self-check, generate-artifacts-ai.md's Step 3F, and
generate-terraform.md's Step 1.5/6, in the advisor plugin tree (the
only tree these three docs live in). No executable test covers prose
accuracy in these files; verification is fmt/lint/drift-check plus a
grep sweep confirming no other instance of either vague phrasing
remains in the three files. dprint check, markdownlint (0 errors),
cross-plugin-drift (281 identical, 27 allowlisted), fixtures:assert
(8 asserters) all green.

@leon1418 leon1418 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[🤖 AI review 🤖]

Completed the current-head review of 75fa6d7 against ade57ad. The three original substantive findings are addressed: AI-only/billing-only runs no longer promise the account baseline; Config/Security Hub use the four implemented compliance frameworks; and database sizing still accepts a Design-mapped class from IaC or live discovery above the $50/month threshold. No new substantive finding from the full documentation diff and relevant producer/consumer review.

Local changed-file formatting, patch-whitespace, and cross-plugin drift checks pass (281 identical, 27 allowlisted); 38 relevant mirror pairs were inspected, including applicable allowlisted differences. All nine current GitHub checks pass. No migration or deployment was executed. This COMMENT records verification; human approval and the remaining conversation/review-state requirements are still outstanding.

@leon1418 leon1418 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[🤖 AI review 🤖]

Completed the startups-hybrid-v1 review of 362623c against ec681ba: actual OCR delegation, a fresh independent general reviewer, the repository review, and explicit candidate verification.

One verified P2 remains in the existing compliance thread: GCP Clarify produces design_constraints.compliance.value, while the baseline generator reads root compliance. The mismatch predates this PR, but the new automatic Config/Security Hub guarantee is not ensured for the standard producer output. Qualify that GCP claim or align the handoff before merging. The earlier route-scope and live database-sizing corrections remain intact.

All four changed Markdown files were reviewed despite OCR excluding their extension. Local formatting, whitespace and cross-plugin drift checks pass; the independent source probes and all nine current GitHub checks pass. No live migration or deployment was executed.

…ndoff

Generate adds those controls only when root preferences.json.compliance
contains soc2, pci, hipaa, or fedramp. Clarify stores the Q2 answer at
design_constraints.compliance.value and leaves the root field unset, so
a standard Clarify answer takes the base-only baseline. Heroku still
keys off global.compliance, which Clarify does write.

Co-authored-by: Cursor <cursoragent@cursor.com>
@herosjourney

Copy link
Copy Markdown
Contributor Author

Qualified the GCP Config/Security Hub claim in c5830aa9 instead of changing the generator.

Confirmed the handoff the review described: Clarify Step 5 writes design_constraints.compliance.value (the example is ["hipaa"] with no root compliance), and generate-artifacts-infra.md Step 1.5 reads root preferences.json.compliance, treating absent/empty as no framework. A standard Clarify answer therefore takes the base-only baseline.tf branch.

README.md, AGENTS.md, and setup.md now say those controls are added only when that root field contains soc2, pci, hipaa, or fedramp, and that a standard Clarify answer does not set it. Heroku is unchanged in behavior and now names global.compliance, which Clarify does write.

The producer/consumer mismatch itself is pre-existing and still unfixed; this only stops the docs from guaranteeing controls the current handoff does not emit. dprint check and markdownlint-cli2 on the three files are clean.

@leon1418 leon1418 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[🤖 AI review 🤖]

Completed the hybrid/convergence follow-up at c5830aa (base ec681ba). F1 is fixed: the GCP compliance limitation is now accurately qualified. No generator repair is required for that documentation fix.

One P2 documentation issue remains, missed in my prior review, not introduced by this fix: billing-only RI/SP and dedicated-section promises exceed that route's estimator/report contract. The consolidated correction is limited to README, AGENTS, and setup:

  • Do not promise commitment options or a dedicated RI/SP section for no-CUD billing-only inputs.
  • Describe active-CUD billing-only guidance as conditional percentage comparison, without implying instance sizing or optimization rows.
  • Keep infrastructure/Heroku guidance and the valid compliance qualification intact.

OCR delegation, both independent impact maps, candidate verification and the convergence gate are complete. Local formatting, whitespace, drift and source-contract checks pass; all nine current CI checks pass. No migration or deployment was executed.

Comment thread advisor/README.md Outdated
Logan Kleier added 3 commits September 23, 2026 14:36
…efresh-sep-2026

# Conflicts:
#	advisor/README.md
#	advisor/plugins/aws-startup-advisor/setup.md
…overy

Bicep, ARM template, and live az CLI discovery are not yet built
(their extractors are planned follow-ups; a Bicep/ARM/live-only
workspace halts rather than guessing). Stop advertising them as
supported discovery sources and change the phase count from 7 to 6
across README, AGENTS, setup, and the azure-to-aws SKILL description
(plus the Migration-mode default line that contradicted the fix).
Comment thread advisor/plugins/aws-startup-advisor/setup.md Outdated
Logan Kleier and others added 3 commits September 24, 2026 08:27
…nly RI/SP

- setup.md: add contextual-offers-for-startups to the install-confirmation
  list and 'what these skills do', bump 10 -> 11 installed skills (jkzietz).
- README/AGENTS/setup: qualify billing-only commitment guidance — a no-CUD
  billing-only run gets neither the dedicated RI/SP section nor percentage
  options; commitment guidance appears only as a conditional CUD-vs-AWS
  comparison when the billing export shows active commitments (leon1418 P2).
setup.md install-success message and README intro said 'GCP or Heroku';
Azure is now supported, so both now read 'GCP, Azure, or Heroku'.
@herosjourney

Copy link
Copy Markdown
Contributor Author

Status: all review findings addressed at a9df5bbe

Recap of every raised thread and where it was fixed. All are addressed in code and have a reply on the thread; the threads still show "unresolved" only because they're marked outdated (the referenced lines moved) and haven't been clicked-resolved.

Finding Reviewer Fixed in Notes
[P2] Scope baseline.tf to the infrastructure-generation route leon1418 75fa6d7d AI-only/billing-only runs no longer promise the account baseline. Reviewer replied "Verified… addressed."
[P2] Name the four compliance frameworks (soc2/pci/hipaa/fedramp) leon1418 75fa6d7d Replaced "when compliance is declared" across README/AGENTS/setup. Reviewer replied "addressed."
[P2] Allow DB sizing from live discovery, not just IaC leon1418 e8d23ba5 $50/mo rule keys off a Design-mapped instance class from IaC or live discovery. Reviewer replied "addressed."
[F1] Qualify the GCP Q2→Generate compliance handoff leon1418 c5830aa9 Documented that Clarify writes design_constraints.compliance.value, not root compliance. Reviewer replied "fixes F1."
[P2] Qualify billing-only commitment / RI-SP guidance leon1418 6837b50a A no-CUD billing-only run gets neither the dedicated RI/SP section nor percentage options — only a conditional CUD-vs-AWS comparison when the export shows active CUDs. Corrected in README, AGENTS, and the GCP setup summary.
There are 11 skills in the plugin now jkzietz 6837b50a Added contextual-offers-for-startups to the install-confirmation list and "What these skills do"; count 10 → 11.

Also addressed while here (Azure had shipped to main but the docs lagged):

  • Corrected azure-to-aws from 7 phases → 6 phases and from "Terraform/Bicep/ARM + az capture" to Terraform (azurerm_*) + app code — Bicep, ARM, and live az discovery are called out as not-yet-available follow-ups (491f7333).
  • Included Azure in the "GCP or Heroku" migration-source phrasing in README and setup (a9df5bbe).

No open comments remain. @leon1418 — the last automated review predates the billing-only fix; a re-review against a9df5bbe would confirm. Reviewers, please resolve the threads if they look good to you.

@leon1418
leon1418 merged commit cb74315 into awslabs:main Sep 24, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants