Skip to content

docs(migrate): scope baseline.tf and compliance controls accurately - #311

Closed
herosjourney wants to merge 4 commits into
awslabs:mainfrom
herosjourney:docs/migrate-readme-baseline-scope
Closed

herosjourney wants to merge 4 commits into
awslabs:mainfrom
herosjourney:docs/migrate-readme-baseline-scope

Conversation

@herosjourney

Copy link
Copy Markdown
Contributor

Summary

Follow-up to PR #301, which fixed the same two documentation-accuracy issues in advisor/README.md, advisor/AGENTS.md, and advisor/plugins/aws-startup-advisor/setup.md but did not touch migrate/README.md (out of that PR's scope). This PR applies the identical fixes there.

  1. Scope baseline.tf to the infrastructure-generation route for gcp-to-aws. generate.md only dispatches generate-artifacts-infra.md (the only file that writes baseline.tf) when generation-infra.json AND aws-design.json both exist. AI-only runs emit bedrock_monitoring.tf instead; billing-only runs emit skeleton Terraform. The "What You Get" comparison table said baseline.tf is "always emitted by both gcp-to-aws and heroku-to-aws Generate" — true for heroku-to-aws (single route), not true for gcp-to-aws's AI-only or billing-only routes.

  2. Name the four compliance frameworks that gate Config + Security Hub. Both generators (gcp-to-aws generate-artifacts-infra.md Step 1.5, heroku-to-aws generate-terraform.md Step 1.5) enable the Config/Security Hub compliance-conditional section only when preferences.json.compliance contains soc2, pci, hipaa, or fedramp; gcp-to-aws's own Step 5 self-check requires their absence for GDPR-only compliance. The prior text didn't mention compliance at all.

Verified

  • Checked the same generator files as PR docs(advisor): refresh README/AGENTS/setup for recent capabilities #301 confirmed the exact gating logic (identical content across migrate/advisor trees per cross-plugin-drift).
  • dprint fmt (table column realignment after the longer cell text), dprint check, markdownlint (0 errors), git diff --check all clean.
  • No executable test covers prose accuracy in this file.

What was tested

Documentation-only change (one file, migrate/README.md). No code paths affected.

Same two documentation-accuracy fixes as PR awslabs#301 (advisor/README.md,
advisor/AGENTS.md, advisor/plugins/aws-startup-advisor/setup.md),
applied to migrate/README.md, which was not part of that PR's scope:

1. baseline.tf is scoped to gcp-to-aws's infrastructure-generation
   route, not unconditional. generate.md only dispatches
   generate-artifacts-infra.md (the only file that writes baseline.tf)
   when generation-infra.json AND aws-design.json both exist. AI-only
   runs emit bedrock_monitoring.tf instead (generate-artifacts-ai.md
   Step 3F); billing-only runs emit skeleton Terraform. The
   "What You Get" comparison table said baseline.tf is "always emitted
   by both gcp-to-aws and heroku-to-aws Generate" -- true for
   heroku-to-aws (single route), not true for gcp-to-aws's AI-only or
   billing-only routes.

2. Name the four compliance frameworks that gate Config + Security
   Hub. Both generators (gcp-to-aws generate-artifacts-infra.md
   Step 1.5, heroku-to-aws generate-terraform.md Step 1.5) enable the
   Config/Security Hub compliance-conditional section only when
   preferences.json.compliance contains soc2, pci, hipaa, or fedramp;
   gcp-to-aws's own Step 5 self-check requires their absence for
   GDPR-only compliance. The "Generates production-ready Terraform"
   bullet and the comparison table row didn't mention compliance at
   all, so a reader had no way to know Config/Security Hub aren't
   unconditional either.

Verified against the same generator files as PR awslabs#301 (identical
content across the migrate/advisor trees per cross-plugin-drift).
dprint fmt (table column realignment after the longer cell text),
markdownlint (0 errors), git diff --check all clean.
@herosjourney
herosjourney requested a review from a team as a code owner September 19, 2026 03:50
Self-review follow-up: the accuracy fix in the previous commit made
the "Security baseline" comparison-table row roughly 3x longer than
every other row in the same table, hurting scannability. Re-verified
both underlying claims are unchanged and still accurate against
generate.md's route-dispatch conditions and
generate-artifacts-infra.md's Step 1.5/Step 5 self-check, then
tightened the wording (dropped redundant phrasing, switched "Config
and Security Hub when compliance includes X, Y, or Z" to the more
compact "Config/Security Hub for X/Y/Z/W" already used elsewhere in
this PR) without losing any of the corrected accuracy. Also matched
the "Generates production-ready Terraform" bullet to the same
shorter phrasing for consistency. Confirmed no other spot in
migrate/README.md or migrate/docs/ repeats either now-corrected
claim. dprint fmt (table realignment), markdownlint, git diff --check
all clean.

@leon1418 leon1418 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[🤖 AI review 🤖]

Reviewed 3e232a9bb7a5384e3544a20bfbe5e43f9691b04a; no substantive findings. The revised README matches the generation contracts: GCP's infrastructure route emits baseline.tf, its AI-only and billing-only routes do not, and Heroku Generate requires it. Both generators gate Config/Security Hub on soc2, pci, hipaa, or fedramp, excluding GDPR-only compliance. I checked the routing, emission rules, and completion checks in both plugin trees; the eight relevant generator/orchestration files are byte-identical.

Validation: README formatting, Markdown lint (896 files, 0 errors), cross-plugin drift (281 identical, 27 allowlisted), and git diff --check passed locally. All nine reported GitHub CI checks passed. This is a documentation/source-contract review; no cloud deployment was performed.

Approval is recommended for a human reviewer to submit; required code-owner approval is still outstanding.

@leon1418 leon1418 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[🤖 AI review 🤖]

Reassessed 3e232a9bb7a5384e3544a20bfbe5e43f9691b04a with OCR delegation, an isolated independent reviewer, and repository contract verification. I missed the GCP compliance data-shape mismatch in my earlier review of this same head. This replaces my earlier clean recommendation.

One P2 correction: qualify or remove the new GCP Config/Security Hub assurance at both migrate/README.md:17 and :115 until the producer/consumer handoff supports it. Clarify emits design_constraints.compliance.value, while GCP Estimate and baseline Generate read top-level compliance. A normal declared-framework answer therefore selects the empty-compliance branch under the documented reads. The source bug predates this PR; this PR newly adds the unsupported README assurance.

If retaining the assurance through an implementation fix, align the Estimate and Generate reads/defaults in both plugin trees and verify canonical wrapped framework answers, negative cases, and consistent cost/control/report outputs. The Heroku assurance and ordinary baseline route distinction are supported; no unrelated improvements are required.

Validation: exact producer-shaped offline probes, independent static-policy execution, formatting, Markdown lint, mirror drift, and diff whitespace checks. The evidence gate passed. No end-to-end migration or cloud deployment was performed.

Comment thread migrate/README.md Outdated

- **Maps your GCP resources to AWS equivalents** — Cloud Run → Fargate, Cloud SQL → RDS or Aurora (based on availability requirements), GKE → EKS, Cloud Storage → S3, VPC → VPC, and more
- **Generates production-ready Terraform** — `vpc.tf`, `compute.tf`, `database.tf`, `security.tf`, `baseline.tf` with account-wide security controls (GuardDuty, CloudTrail, IMDSv2, ECR scanning), and a full `terraform/README.md`
- **Generates production-ready Terraform** — `vpc.tf`, `compute.tf`, `database.tf`, `security.tf`, `baseline.tf` with account-wide security controls (GuardDuty, CloudTrail, IMDSv2, ECR scanning, plus Config/Security Hub for soc2/pci/hipaa/fedramp), and a full `terraform/README.md`

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[🤖 AI review 🤖] [P2] Qualify the GCP controls promise until compliance reaches the consumers

For a normal HIPAA answer, gcp-to-aws/references/phases/clarify/clarify.md:606-620 writes design_constraints.compliance.value: ["hipaa"]. But generate-artifacts-infra.md:156 reads preferences.json.compliance, treats its absence as empty, and consequently skips the Config/Security Hub section at line 190. estimate-infra.md:140 uses the same flat lookup for the compliance cost entry. The current producer example therefore does not satisfy this new assurance; both plugin copies have the same mismatch.

Please qualify/remove the GCP assurance here and in the Security baseline row (line 115), or first align the consumers with the canonical wrapped value and verify declared/empty/GDPR-only cases. Heroku's global.compliance handoff is supported. This underlying source bug predates the PR and was missed in my earlier review.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed at 92bdb9c by taking the "qualify/remove" option — this is a docs-accuracy PR, so the README should not assert a control path the code does not support, and folding the underlying handoff fix into it would be out of scope.

Verified the shape mismatch you flagged, in both trees:

  • Clarify writes the declared frameworks to preferences.json.design_constraints.compliance.value (clarify-assemble.md: the compliance object is {value: [...], chosen_by: ...} nested under design_constraints).
  • gcp-to-aws Estimate (estimate-infra.md) and baseline Generate (generate-artifacts-infra.md Steps 1.5/3/6) read top-level preferences.json.compliance. A normal declared-framework answer lands in the empty branch, so Config/Security Hub is not emitted for GCP.
  • heroku-to-aws is consistent — Clarify writes and generate-terraform.md Step 0 reads preferences.json.global.compliance — so its assurance is supported.

Changes: dropped the Config/Security Hub claim from the GCP infrastructure bullet (:17) and scoped it to heroku-to-aws in the comparison table (:115). The baseline.tf core controls and the infra-route-only distinction are unchanged. The GCP read/write shape mismatch predates this PR and is a separate code fix (align Estimate + baseline Generate to read design_constraints.compliance.value in both plugin trees, with canonical-wrapped / negative-case coverage) — tracked as follow-up, not bundled here. dprint fmt + markdownlint clean.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[🤖 AI review 🤖]

Verified at 92bdb9c8ca36c9635dfe1238fd382a2f145d1172: this finding is fixed through the documentation-only option. The infrastructure bullet no longer promises GCP Config/Security Hub controls, and the comparison row explicitly attributes that conditional assurance to heroku-to-aws. I rechecked the producer/consumer contracts in both plugin trees; the underlying GCP field-shape gap remains separate and is not represented as repaired here.

After OCR-assisted review, a fresh independent review, and final candidate verification, I found no remaining substantive issue in this PR. Source/route checks, formatting, Markdown lint, mirror drift, and diff whitespace checks passed locally; all nine current CI checks passed. No migration or deployment was run.

Approval is recommended for the human reviewer to submit. This reply confirms the documentation correction; it does not dismiss the earlier blocking review or resolve the thread.

herosjourney and others added 2 commits September 22, 2026 22:02
The prior revision of this PR claimed gcp-to-aws's baseline.tf emits
Config/Security Hub controls for soc2/pci/hipaa/fedramp. That assurance is not
supported by the current producer/consumer handoff: Clarify writes the declared
frameworks to `preferences.json.design_constraints.compliance.value`, while
gcp-to-aws Estimate (estimate-infra.md) and baseline Generate
(generate-artifacts-infra.md Steps 1.5/3/6) read top-level
`preferences.json.compliance`. A normal declared-framework answer therefore lands
in the empty-compliance branch, and the Config/Security Hub section is not emitted
for GCP — so the README must not promise it.

- Line 17 (GCP infrastructure bullet): remove the Config/Security Hub claim.
- Line 115 (comparison table): keep the baseline.tf core controls and the
  infra-route-only distinction; scope the Config/Security Hub compliance controls
  to heroku-to-aws, which IS consistent (Clarify writes and Generate reads
  `preferences.json.global.compliance`).

Docs-only accuracy fix (the underlying GCP read/write shape mismatch predates this
PR and is tracked separately). dprint fmt (table realign), markdownlint 0 errors,
git diff --check clean.
@jkzietz

jkzietz commented Sep 25, 2026

Copy link
Copy Markdown
Contributor

Heads up: the aws-startup-advisor plugin has a new home

The plugin now lives in the Agent Toolkit for AWS, merged to main earlier today:
https://github.com/aws/agent-toolkit-for-aws/tree/main/plugins/aws-startup-advisor

We are beginning plans to deprecate this repository, so that's the copy to build on going forward — changes landed here won't reach customers once distribution repoints. Please re-open this PR against aws/agent-toolkit-for-aws.

Porting your diff: paths move from advisor/plugins/aws-startup-advisor/… to plugins/aws-startup-advisor/…. Two differences to expect:

  • The plugin declares a single MCP server there — aws-mcp, the unified AWS MCP Server — and pricing is cache-only, with no live lookup.
  • The destination enforces markdownlint's default rule set, which this repo does not. MD036 (emphasis used as a heading) and MD059 (descriptive link text) are the two that usually need fixing.

Happy to help with the move if anything doesn't map cleanly.

@herosjourney

Copy link
Copy Markdown
Contributor Author

Ported to aws/agent-toolkit-for-aws#342 per the move to the new plugin home. Paths remapped from advisor/plugins/aws-startup-advisor/… to plugins/aws-startup-advisor/…; the port passes the destination's markdownlint default ruleset and tools/validate.py. Closing this one in favor of the ported PR, since this repo is being deprecated.

herosjourney added a commit to herosjourney/agent-toolkit-for-aws that referenced this pull request Sep 28, 2026
…the migration Generate bullet (aws#342)

Ported from the intent of awslabs/startups#311 (and its advisor/README
counterpart aws#301). The rewritten plugin README's Generate bullet dropped
the security-baseline detail entirely. Restore it with accurate scoping:
baseline.tf is emitted on the infrastructure route by heroku-to-aws and
gcp-to-aws, Config/Security Hub are gated on soc2/pci/hipaa/fedramp, and
gcp-to-aws's AI-only and billing-only routes skip the baseline. Reflects
the post-aws#341 behavior where the declared GCP compliance set now reaches
the generator.

Co-authored-by: Logan Kleier <lkleier@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants