feat(heroku-to-aws): emit baseline.tf account security baseline in Generate - #261
Conversation
…nerate
Closes the largest security-parity gap between the two migration paths:
gcp-to-aws has always emitted baseline.tf (account-wide controls), while
heroku-to-aws users got Terraform with no GuardDuty, no CloudTrail, no
IMDSv2 default, no budget alerts. This ports the gcp-to-aws Step 1.5
baseline spec into heroku-to-aws's DSL-based Generate phase, translated
into the DSL's native idiom (frontmatter contracts + assembler gates)
rather than pasted as prose orchestration.
generate-terraform.md (new Step 1.5, both plugin trees):
- Compliance normalization: heroku stores preferences.global.compliance
as a scalar ("none"|"soc2"|"hipaa"|"pci") or user array; the
baseline spec normalizes to an array before the retention mapping.
Unrecognized frameworks get conservative 365-day retention plus a
generation-warnings.json entry.
- Same always-on resource set as gcp-to-aws: 3x alternate contacts,
IAM password policy, S3 account PAB, EBS default encryption, Access
Analyzer, IMDSv2 account default, CloudTrail + hardened log bucket,
monthly budget (max(50, ceil(total.mid * 1.2)) from
estimation-infra.json with fallback), GuardDuty.
- Same compliance-conditional section (Config + Security Hub + FSBP;
PCI DSS subscription only for pci; explicitly no NIST 800-53).
- Item 9 rider re-scoped for heroku's compute paths: IMDSv2
metadata_options on the EKS self-managed launch template
(generate-eks.md); Fargate and EB emit no launch templates.
Step 2 gains the three fill-once contact email variables with
placeholder-rejecting validation blocks (plan fails loudly on
TODO/example.com); Step 11's tfvars.example lists them; Step 12 gains
a baseline self-check row.
generate.md frontmatter: terraform/baseline.tf added to _produces and
_check_file_exists; three new _assert postconditions (resource
inventory, compliance-conditional exactness, contact variables) —
enforced fail-closed by the assembler per INTERPRETER.md gate
protocol, making these checks stronger than gcp-to-aws's prose
checklist equivalents.
generate-docs.md: MIGRATION_GUIDE Phase 1 explains the baseline,
the contact tfvars, and the two opt-outs (delete baseline.tf; delete
the compliance-conditional block); README artifact table lists
baseline.tf.
READMEs: the security-baseline row now correctly covers both paths,
and drops 'ECR scanning' from the always-on list (heroku-to-aws
generates no ECR repositories; scan-on-push applies where ECR exists).
Intentionally out of scope (PR 2): tf-best-practices authoring-posture
invocation and the policy gate with validation-report.json.
Verification: markdownlint-cli2 0 errors; dprint clean; cross-plugin
drift OK (both trees byte-identical); frontmatter validator OK on both
heroku-to-aws trees; tests/heroku-eb-runtime-settings.test.ts 11/12
(the 1 failure is a pre-existing environmental Terraform version pin —
1.13.x expected, 1.15.2 installed — and fails identically on a clean
tree).
… and honest opt-out Five must-fixes from review, all first-user apply/plan failures: 1. Opt-out was incomplete: deleting baseline.tf leaves three defaultless contact variables that still fail plan even unreferenced. The guide, assembler output, README artifact row, Step 1.5 intro, and emission rules now document the real two-step opt-out (delete the file AND remove the variables). 2. Budget read the gcp path (projected_costs.breakdown.total.mid), which doesn't exist in heroku's estimation schema — nearly every run would have silently fallen to the $50 floor. Pinned to projected_costs.aws_monthly_balanced, the key heroku's own Estimate postconditions assert. 3. AWSConfigRole -> arn:aws:iam::aws:policy/service-role/AWS_ConfigRole (the no-underscore name is deprecated and fails apply). Fixed in the heroku spec AND at the gcp source it was copied from, both trees. 4. Validation error messages pointed at a 'MIGRATION_GUIDE.md fill-in checklist' that doesn't exist in heroku's guide; they now point at Phase 1's 'Security baseline contacts' block, which now carries that heading. 5. aws_account_alternate_contact requires name/title/phone_number, not just email_address. Spec now pins name/title per contact type and a clearly-commented placeholder phone. Plus four should-fixes: - 'unknown' compliance normalizes to [] like 'none' (an unconfirmed answer is not a framework); none/unknown entries dropped from arrays. - Unrecognized-framework note moved from generation-warnings.json (whose entries are service-shaped and feed the every-service-accounted-for gate) to the baseline.tf header comment. - Golden HCL block for the four shapes agents invent wrong: full alternate-contact fields, CloudTrail bucket policy (SourceArn-scoped), Config trust policy + current managed-policy ARN, lifecycle filter. - Report next-steps spec gains one bullet: baseline.tf ships enabled and three contact emails must be set before plan (the plugin README already claims the report covers the security baseline). Structural: the golden HCL and EKS rider are bold paragraphs after list item 8 rather than list items — dprint dedents fenced blocks inside two-digit ordered items, which split the list and tripped MD029. Verification: markdownlint 0 errors; dprint clean; drift OK (all touched files byte-identical across trees); frontmatter validator OK both trees; heroku EB test 11/12 (unchanged pre-existing environmental Terraform version-pin failure).
Review addressed —
|
The golden bucket policy pinned aws:SourceArn to
trail/${var.project_name}-baseline while item 5 never named the trail —
an agent naming it anything else produces a bucket policy that rejects
CloudTrail's ACL check at create time. 'Match them exactly' made the
incomplete snippet authoritative, so the gap was guaranteed to ship.
- Item 5 now pins name = "${var.project_name}-baseline" and
depends_on = [aws_s3_bucket_policy.cloudtrail_logs] on
aws_cloudtrail.baseline (CloudTrail validates the bucket policy at
create time).
- Golden HCL completes the chain: aws:SourceArn condition added to the
PutObject statement (was on GetBucketAcl only), plus the
aws_s3_bucket_policy attachment and the aws_cloudtrail resource
itself, with the name/SourceArn coupling called out in comments.
Nits from the same review:
- Item 8: dropped the '(or in tfvars if the user prefers)' phone clause
(no phone variable exists); merged the two alternate-contact comment
bullets into one.
- Item 3 now says what item 1 promised: unrecognized compliance values
get a named header line with the conservative 365-day retention.
Both plugin trees, byte-identical. markdownlint 0, dprint clean, drift
OK, frontmatter validator OK, EB test 11/12 (pre-existing env failure).
Round-2 review addressed —
|
leon1418
left a comment
There was a problem hiding this comment.
[🤖 AI review 🤖]
Review of PR #261 — feat(heroku-to-aws): emit baseline.tf account security baseline in Generate
Head: ead24bd80f9b1656a283a9eb7fc0002147e65284 | 16 files | +641/−75
What this change does
Ports the gcp-to-aws Step 1.5 baseline.tf security baseline (account alternate contacts, IAM password policy, S3 account public access block, EBS default encryption, Access Analyzer, IMDSv2 account default, CloudTrail + hardened log bucket, budget alerts, GuardDuty, plus compliance-conditional Config + Security Hub) into heroku-to-aws's Generate phase. Translates it into heroku's DSL idiom — fail-closed _assert postconditions and _check_file_exists gates in frontmatter rather than prose checklists the agent is trusted to follow. Also adds IMDSv2 enforcement on EKS self-managed launch templates, placeholder-rejecting contact email variable validations, MIGRATION_GUIDE opt-out docs, README accuracy fix (ECR scanning claim scoped to where ECR repos actually exist), and fixes the deprecated AWSConfigRole → AWS_ConfigRole policy name at its source in gcp-to-aws.
Findings
0 mandatory blockers. 0 Nits.
Design is sound:
- The port is not a blind copy — it adapts to heroku-to-aws's DSL architecture (interpreter-enforced postconditions vs advisory prose), which is the right approach. The three new
_assertpostconditions ingenerate.mdfrontmatter cover: (1) always-on resource inventory, (2) compliance-conditional section presence/absence, (3) contact variables with no-default + placeholder-rejecting validation. All three fail_halt_and_inform. - Compliance normalization handles the scalar/array polymorphism correctly;
"unknown"treated as"none"(conservative — no framework assumed) with unrecognized values getting 365-day conservative retention + file-header comment (correctly NOT ingeneration-warnings.jsonwhose entries are service-shaped). - Budget pinned to
projected_costs.aws_monthly_balanced— the same key Estimate postconditions assert — with sensible $50 floor fallback. strcontainsusage in validation blocks is compatible with the existingrequired_version = ">= 1.5.0"constraint.- EKS IMDSv2 rider at hop limit 1 (stricter than account default 2) is deliberately documented and correct for plugin-owned templates.
- Golden HCL covers the three shapes agents historically get wrong: alternate contacts (all 4 required fields), CloudTrail bucket policy (SourceArn-scoped with depends_on), and Config role (current managed policy ARN). Trail name pinned to match SourceArn — the coupling that causes create-time failures when improvised.
- Two-step opt-out (delete file AND remove variables) is documented in MIGRATION_GUIDE Phase 1 with clear explanation of why both steps are needed.
- The deprecated
AWSConfigRole→AWS_ConfigRolefix in gcp-to-aws is a genuine bug fix (the old name failsterraform apply).
Validation evidence
- Cross-plugin parity: All 7 heroku-to-aws file pairs (advisor ↔ migrate) verified byte-identical. The 1 gcp-to-aws file pair also identical. No drift allowlist entries added or modified.
- Frontmatter:
generate.mdYAML parsed successfully — 14 postcondition items, all structurally valid with_on_failure: _halt_and_inform. - README updates: Only the 2 migrate-side READMEs updated; advisor README has no security baseline capability row (confirmed — no counterpart needed).
- CI: No checks reported yet on this branch.
- Approvals: 0/0.
- Mergeable: MERGEABLE (GitHub).
CLEAN. Recommend merge after CI goes green and maintainer approval.
Ports the gcp-to-aws two-touchpoint pattern into heroku-to-aws, translated into the DSL idiom: a before-write authoring-posture invoke (Step 0) and an after-write policy gate (Step 12) that writes validation-report.json with policy_status, with a fail-closed policy_status _assert in generate.md _postconditions so POLICY_FAIL blocks Generate. Completes the policy-gate half of P1-A (baseline.tf half landed in awslabs#261). - tf-best-practices/SKILL.md: consumers note now includes heroku-to-aws - generate-terraform.md: Step 0 authoring posture + Step 12 policy gate + validation-report.json in _contributes - generate.md: validation-report.json in _produces/_check_file_exists + policy_status _assert - generate-assemble.md: policy gate is a required (not optional) completion check - 2 Heroku-shaped fixtures (POLICY_OK + POLICY_FAIL) + 2 tests; fixture count 23->25 - migrate README scope-note corrected - policy script unchanged (source-agnostic); EB false-positive check ruled out exemptions Both plugin trees updated byte-identically.
Ports the gcp-to-aws two-touchpoint pattern into heroku-to-aws, translated into the DSL idiom: a before-write authoring-posture invoke (Step 0) and an after-write policy gate (Step 12) that writes validation-report.json with policy_status, with a fail-closed policy_status _assert in generate.md _postconditions so POLICY_FAIL blocks Generate. Completes the policy-gate half of P1-A (baseline.tf half landed in awslabs#261). - tf-best-practices/SKILL.md: consumers note now includes heroku-to-aws - generate-terraform.md: Step 0 authoring posture + Step 12 policy gate + validation-report.json in _contributes - generate.md: validation-report.json in _produces/_check_file_exists + policy_status _assert - generate-assemble.md: policy gate is a required (not optional) completion check - 2 Heroku-shaped fixtures (POLICY_OK + POLICY_FAIL) + 2 tests; fixture count 23->25 - migrate README scope-note corrected - policy script unchanged (source-agnostic); EB false-positive check ruled out exemptions Both plugin trees updated byte-identically.
…rges Doc-only. Reflects capability changes merged into aws-startup-advisor since mid-August that the top-level docs never picked up, plus two pre-existing inaccuracies found in the same pass: - gcp-to-aws: OpenAI sources now land on the same GPT model on Bedrock when one exists (#210, #250, #237), not just a Claude/Nova cross-family swap. Discover can pull real spend from the OpenAI Admin API, consent-gated (#236). GCP Document AI / Vision / Speech-to-Text are detected and routed to AWS traditional-AI services (#240, #241). - Both migration skills: commitment-discount guidance (shared RI/SP eligibility matrix, three-state model, dedicated report section — #266, #270, #272, #277) was undocumented anywhere in the top-level docs. - heroku-to-aws: Generate emits `baseline.tf`, an account security baseline, alongside the app Terraform (#261). - setup.md: agent-advisor was missing Lambda MicroVMs; prompt-library counts were wrong (30 prompts / 4 agents named "Migration" -> actually 29 prompts / 5 agents, none called "Migration") and README's agent list was incomplete. Pre-existing, not tied to a specific recent PR. - knowledge-base-for-startups: bumped the skill's own "Last updated" date to 2026-09-09 to match the offers refresh in #281 (the date drives its 6-month staleness nudge to users). Verified: markdownlint clean. No skill/schema/tool files touched.
Problem
gcp-to-awsGenerate has always emittedbaseline.tf— account-wide security controls (alternate contacts, IAM password policy, S3 account public access block, EBS default encryption, Access Analyzer, IMDSv2 account default, CloudTrail with a hardened log bucket, a monthly AWS Budget with alerts, GuardDuty, plus compliance-conditional Config + Security Hub).heroku-to-awsnever has: a Heroku user today gets Terraform with none of those controls. This is the largest security-parity gap between the two migration paths, and it's the gap the READMEs have had to caveat since #229.What this does
Ports the
gcp-to-awsStep 1.5 baseline spec intoheroku-to-aws's Generate phase — translated into the DSL's native idiom, not pasted as prose. The two skills are architected differently: gcp-to-aws is prose-orchestrated (imperative steps the agent follows), heroku-to-aws is DSL-based (frontmatter contracts enforced fail-closed by the vendored interpreter + assembler gates). So the port lands in three layers:generate-artifacts-infra.mdgenerate-terraform.md+_contributesentry_assertpostconditions ingenerate.mdfrontmatter, enforced by the assembler perINTERPRETER.mdgate protocolterraform/baseline.tfin_produces+_check_file_existsmetadata_optionson the EKS self-managed launch template ingenerate-eks.md(Fargate/EB emit no launch templates)Because the checks are interpreter-enforced postconditions rather than prose the agent is trusted to follow, the ported version is fail-closed where the original is advisory.
Adaptations (not blind copies)
preferences.global.complianceas a scalar ("none"|"soc2"|"hipaa"|"pci") or user-specified array (Clarify Q2 option E); gcp uses an array. New item 0 normalizes to an array before the retention mapping, treating"unknown"like"none"(an unconfirmed answer is not a framework). Unrecognized frameworks (possible via option E) get conservative 365-day retention plus a named note in thebaseline.tfheader comment (notgeneration-warnings.json, whose entries are service-shaped and feed the every-service-accounted-for gate).operations_email/billing_email/security_emaildidn't exist in heroku'svariables.tfstep. Added with no defaults and placeholder-rejectingvalidationblocks (the gcp pattern —terraform planfails loudly onTODO/example.cominstead of silently making them the account's security contacts), plusterraform.tfvars.exampleentries.baseline.tfAND remove the three contact variables, which have no defaults and failplaneven unreferenced; partial opt-out deletes just the compliance-conditional block. README artifact table listsbaseline.tf; the report next-steps spec gains a one-bullet baseline mention.aws_account_alternate_contactrequired fields (name/title/phone_number, not just email), the SourceArn-scoped CloudTrail bucket policy with itsaws_s3_bucket_policyattachment and theaws_cloudtrailresource itself (trail name pinned to match the SourceArn;depends_onthe attachment since CloudTrail validates the bucket policy at create time), the Config trust policy with the currentAWS_ConfigRolemanaged-policy ARN, and the lifecyclefilter {}.projected_costs.aws_monthly_balanced— the key heroku's own Estimate postconditions assert — with a $50 floor fallback.heroku-to-awsgenerates no ECR repositories. Now reads "ECR scan-on-push where ECR repositories are generated," and covers both paths.Scope
In: baseline.tf emission, contact variables, DSL postconditions, EKS IMDSv2 rider, golden HCL, docs. Both plugin trees updated byte-identically (
advisor/+migrate/). Also fixes the deprecatedAWSConfigRolemanaged-policy name at its source ingcp-to-aws's spec (both trees) — this PR inherited the bug from there, and it is the same apply failure on gcp's compliance path.Out (follow-up PR):
tf-best-practicesauthoring-posture invocation and the policy gate withvalidation-report.json— the "Consumers (v1): gcp-to-aws only" note intf-best-practices/SKILL.mdis intentionally untouched until that lands.Interaction with #229: that open PR hedges the README security-baseline row with "not yet emitted by heroku-to-aws (planned)". This PR makes the unhedged claim true. Whichever merges second will have a trivial README conflict to resolve in favor of this PR's wording.
Verification
markdownlint-cli2: 0 errors (repo config)dprint check: cleancross-plugin-drift.ts: OK — all five touched generate files byte-identical across both plugin trees (none are drift-allowlisted, so CI enforces this permanently)heroku-to-awstrees (7 phase files each)tests/heroku-eb-runtime-settings.test.ts: 11/12 pass; the 1 failure is a pre-existing environmental Terraform version-pin assertion (expects 1.13.x, machine has 1.15.2) and fails identically on a clean checkout — not introduced by this change