Skip to content

feat(heroku-to-aws): emit baseline.tf account security baseline in Generate - #261

Merged
leon1418 merged 3 commits into
awslabs:mainfrom
herosjourney:feat/heroku-baseline-tf
Sep 2, 2026
Merged

leon1418 merged 3 commits into
awslabs:mainfrom
herosjourney:feat/heroku-baseline-tf

Conversation

@herosjourney

@herosjourney herosjourney commented Sep 2, 2026 •

Copy link
Copy Markdown
Contributor

Problem

gcp-to-aws Generate has always emitted baseline.tf — account-wide security controls (alternate contacts, IAM password policy, S3 account public access block, EBS default encryption, Access Analyzer, IMDSv2 account default, CloudTrail with a hardened log bucket, a monthly AWS Budget with alerts, GuardDuty, plus compliance-conditional Config + Security Hub). heroku-to-aws never has: a Heroku user today gets Terraform with none of those controls. This is the largest security-parity gap between the two migration paths, and it's the gap the READMEs have had to caveat since #229.

What this does

Ports the gcp-to-aws Step 1.5 baseline spec into heroku-to-aws's Generate phase — translated into the DSL's native idiom, not pasted as prose. The two skills are architected differently: gcp-to-aws is prose-orchestrated (imperative steps the agent follows), heroku-to-aws is DSL-based (frontmatter contracts enforced fail-closed by the vendored interpreter + assembler gates). So the port lands in three layers:

gcp-to-aws (prose) heroku-to-aws (this PR)
Step 1.5 emission spec in generate-artifacts-infra.md New Step 1.5 in generate-terraform.md + _contributes entry
Step 5 prose checklist (~12 baseline assertions) Three _assert postconditions in generate.md frontmatter, enforced by the assembler per INTERPRETER.md gate protocol
Phase Completion prose gate ("baseline.tf MUST exist") terraform/baseline.tf in _produces + _check_file_exists
compute.tf IMDSv2 launch-template rider metadata_options on the EKS self-managed launch template in generate-eks.md (Fargate/EB emit no launch templates)

Because the checks are interpreter-enforced postconditions rather than prose the agent is trusted to follow, the ported version is fail-closed where the original is advisory.

Adaptations (not blind copies)

  • Compliance shape: heroku stores preferences.global.compliance as a scalar ("none"|"soc2"|"hipaa"|"pci") or user-specified array (Clarify Q2 option E); gcp uses an array. New item 0 normalizes to an array before the retention mapping, treating "unknown" like "none" (an unconfirmed answer is not a framework). Unrecognized frameworks (possible via option E) get conservative 365-day retention plus a named note in the baseline.tf header comment (not generation-warnings.json, whose entries are service-shaped and feed the every-service-accounted-for gate).
  • Contact variables: operations_email/billing_email/security_email didn't exist in heroku's variables.tf step. Added with no defaults and placeholder-rejecting validation blocks (the gcp pattern — terraform plan fails loudly on TODO/example.com instead of silently making them the account's security contacts), plus terraform.tfvars.example entries.
  • Docs: MIGRATION_GUIDE Phase 1 template ("Security baseline contacts" — the target of the validation error messages) explains the baseline, the three contact tfvars, both opt-outs, and the CloudTrail/Config/Security Hub collision warnings. Full opt-out is two steps — delete baseline.tf AND remove the three contact variables, which have no defaults and fail plan even unreferenced; partial opt-out deletes just the compliance-conditional block. README artifact table lists baseline.tf; the report next-steps spec gains a one-bullet baseline mention.
  • Golden HCL: a reference block for the shapes agents invent wrong — full aws_account_alternate_contact required fields (name/title/phone_number, not just email), the SourceArn-scoped CloudTrail bucket policy with its aws_s3_bucket_policy attachment and the aws_cloudtrail resource itself (trail name pinned to match the SourceArn; depends_on the attachment since CloudTrail validates the bucket policy at create time), the Config trust policy with the current AWS_ConfigRole managed-policy ARN, and the lifecycle filter {}.
  • Budget source: pinned to projected_costs.aws_monthly_balanced — the key heroku's own Estimate postconditions assert — with a $50 floor fallback.
  • README accuracy fix: the security-baseline capability row claimed "ECR scanning" unconditionally — heroku-to-aws generates no ECR repositories. Now reads "ECR scan-on-push where ECR repositories are generated," and covers both paths.

Scope

In: baseline.tf emission, contact variables, DSL postconditions, EKS IMDSv2 rider, golden HCL, docs. Both plugin trees updated byte-identically (advisor/ + migrate/). Also fixes the deprecated AWSConfigRole managed-policy name at its source in gcp-to-aws's spec (both trees) — this PR inherited the bug from there, and it is the same apply failure on gcp's compliance path.
Out (follow-up PR): tf-best-practices authoring-posture invocation and the policy gate with validation-report.json — the "Consumers (v1): gcp-to-aws only" note in tf-best-practices/SKILL.md is intentionally untouched until that lands.

Interaction with #229: that open PR hedges the README security-baseline row with "not yet emitted by heroku-to-aws (planned)". This PR makes the unhedged claim true. Whichever merges second will have a trivial README conflict to resolve in favor of this PR's wording.

Verification

  • markdownlint-cli2: 0 errors (repo config)
  • dprint check: clean
  • cross-plugin-drift.ts: OK — all five touched generate files byte-identical across both plugin trees (none are drift-allowlisted, so CI enforces this permanently)
  • frontmatter validator: OK on both heroku-to-aws trees (7 phase files each)
  • tests/heroku-eb-runtime-settings.test.ts: 11/12 pass; the 1 failure is a pre-existing environmental Terraform version-pin assertion (expects 1.13.x, machine has 1.15.2) and fails identically on a clean checkout — not introduced by this change

…nerate

Closes the largest security-parity gap between the two migration paths:
gcp-to-aws has always emitted baseline.tf (account-wide controls), while
heroku-to-aws users got Terraform with no GuardDuty, no CloudTrail, no
IMDSv2 default, no budget alerts. This ports the gcp-to-aws Step 1.5
baseline spec into heroku-to-aws's DSL-based Generate phase, translated
into the DSL's native idiom (frontmatter contracts + assembler gates)
rather than pasted as prose orchestration.

generate-terraform.md (new Step 1.5, both plugin trees):
- Compliance normalization: heroku stores preferences.global.compliance
  as a scalar ("none"|"soc2"|"hipaa"|"pci") or user array; the
  baseline spec normalizes to an array before the retention mapping.
  Unrecognized frameworks get conservative 365-day retention plus a
  generation-warnings.json entry.
- Same always-on resource set as gcp-to-aws: 3x alternate contacts,
  IAM password policy, S3 account PAB, EBS default encryption, Access
  Analyzer, IMDSv2 account default, CloudTrail + hardened log bucket,
  monthly budget (max(50, ceil(total.mid * 1.2)) from
  estimation-infra.json with  fallback), GuardDuty.
- Same compliance-conditional section (Config + Security Hub + FSBP;
  PCI DSS subscription only for pci; explicitly no NIST 800-53).
- Item 9 rider re-scoped for heroku's compute paths: IMDSv2
  metadata_options on the EKS self-managed launch template
  (generate-eks.md); Fargate and EB emit no launch templates.

Step 2 gains the three fill-once contact email variables with
placeholder-rejecting validation blocks (plan fails loudly on
TODO/example.com); Step 11's tfvars.example lists them; Step 12 gains
a baseline self-check row.

generate.md frontmatter: terraform/baseline.tf added to _produces and
_check_file_exists; three new _assert postconditions (resource
inventory, compliance-conditional exactness, contact variables) —
enforced fail-closed by the assembler per INTERPRETER.md gate
protocol, making these checks stronger than gcp-to-aws's prose
checklist equivalents.

generate-docs.md: MIGRATION_GUIDE Phase 1 explains the baseline,
the contact tfvars, and the two opt-outs (delete baseline.tf; delete
the compliance-conditional block); README artifact table lists
baseline.tf.

READMEs: the security-baseline row now correctly covers both paths,
and drops 'ECR scanning' from the always-on list (heroku-to-aws
generates no ECR repositories; scan-on-push applies where ECR exists).

Intentionally out of scope (PR 2): tf-best-practices authoring-posture
invocation and the policy gate with validation-report.json.

Verification: markdownlint-cli2 0 errors; dprint clean; cross-plugin
drift OK (both trees byte-identical); frontmatter validator OK on both
heroku-to-aws trees; tests/heroku-eb-runtime-settings.test.ts 11/12
(the 1 failure is a pre-existing environmental Terraform version pin —
1.13.x expected, 1.15.2 installed — and fails identically on a clean
tree).
@herosjourney
herosjourney requested review from a team as code owners September 2, 2026 04:10
… and honest opt-out

Five must-fixes from review, all first-user apply/plan failures:

1. Opt-out was incomplete: deleting baseline.tf leaves three defaultless
   contact variables that still fail plan even unreferenced. The guide,
   assembler output, README artifact row, Step 1.5 intro, and emission
   rules now document the real two-step opt-out (delete the file AND
   remove the variables).
2. Budget read the gcp path (projected_costs.breakdown.total.mid), which
   doesn't exist in heroku's estimation schema — nearly every run would
   have silently fallen to the $50 floor. Pinned to
   projected_costs.aws_monthly_balanced, the key heroku's own Estimate
   postconditions assert.
3. AWSConfigRole -> arn:aws:iam::aws:policy/service-role/AWS_ConfigRole
   (the no-underscore name is deprecated and fails apply). Fixed in the
   heroku spec AND at the gcp source it was copied from, both trees.
4. Validation error messages pointed at a 'MIGRATION_GUIDE.md fill-in
   checklist' that doesn't exist in heroku's guide; they now point at
   Phase 1's 'Security baseline contacts' block, which now carries that
   heading.
5. aws_account_alternate_contact requires name/title/phone_number, not
   just email_address. Spec now pins name/title per contact type and a
   clearly-commented placeholder phone.

Plus four should-fixes:
- 'unknown' compliance normalizes to [] like 'none' (an unconfirmed
  answer is not a framework); none/unknown entries dropped from arrays.
- Unrecognized-framework note moved from generation-warnings.json (whose
  entries are service-shaped and feed the every-service-accounted-for
  gate) to the baseline.tf header comment.
- Golden HCL block for the four shapes agents invent wrong: full
  alternate-contact fields, CloudTrail bucket policy (SourceArn-scoped),
  Config trust policy + current managed-policy ARN, lifecycle filter.
- Report next-steps spec gains one bullet: baseline.tf ships enabled and
  three contact emails must be set before plan (the plugin README
  already claims the report covers the security baseline).

Structural: the golden HCL and EKS rider are bold paragraphs after list
item 8 rather than list items — dprint dedents fenced blocks inside
two-digit ordered items, which split the list and tripped MD029.

Verification: markdownlint 0 errors; dprint clean; drift OK (all
touched files byte-identical across trees); frontmatter validator OK
both trees; heroku EB test 11/12 (unchanged pre-existing environmental
Terraform version-pin failure).
@herosjourney

Copy link
Copy Markdown
Contributor Author

Review addressed — 3bd5082

All five must-fixes plus the four should-fixes, one commit, both plugin trees.

Must-fixes (all were first-user plan/apply failures):

  1. Opt-out — the two-step reality (delete baseline.tf AND remove the three contact variables, which have no defaults and fail plan even unreferenced) is now documented in MIGRATION_GUIDE Phase 1, and every place that said "just delete the file" (assembler output, README artifact row, Step 1.5 intro, emission rules) now points there instead of repeating the broken claim.
  2. Budget path — pinned to projected_costs.aws_monthly_balanced, the key this skill's own Estimate postconditions assert is a positive number. The gcp-shaped breakdown.total.mid path would have sent nearly every run to the $50 floor.
  3. AWS_ConfigRole — fixed with the full ARN and an explicit note that the no-underscore name is deprecated and fails apply. One correction to the review's supporting claim: there is no "working GCP file in this repo" with the correct name — the gcp spec (generate-artifacts-infra.md) carries the identical AWSConfigRole bug, which is where this PR copied it from. Fixed at the source too, both trees, since it's the same apply failure on gcp's compliance path.
  4. Error-message target — the guide's Phase 1 contact block now carries the heading "Security baseline contacts" and all three error_messages and variable descriptions point at it. No "fill-in checklist" references remain.
  5. Alternate contacts — spec now names all four required arguments per contact type (pinned name/title, placeholder phone_number with an update-post-apply comment), and the golden HCL shows the full shape.

Should-fixes:

  • "unknown" normalizes to [] like "none"; both are dropped from user-specified arrays.
  • The unrecognized-framework note moved to the baseline.tf header comment — agreed that generation-warnings.json is service-shaped and a compliance note there would feed a junk row into the every-service-accounted-for gate.
  • Golden HCL block added for the four invent-prone shapes: full alternate-contact fields, SourceArn-scoped CloudTrail bucket policy, Config trust policy + current managed-policy ARN, lifecycle filter {}.
  • Report next-steps spec gains the one-bullet baseline mention (the plugin README already claims the report covers the security baseline, so the claim and the report now match).

Not added, per the review: Estimate pricing GuardDuty/CloudTrail/Config so the budget input includes the baseline itself — real gap, separate follow-up.

Verification re-run after changes: markdownlint 0 errors, dprint clean, cross-plugin drift OK (all touched files byte-identical across trees), frontmatter validator OK on both trees, heroku EB test 11/12 with the same pre-existing environmental Terraform version-pin failure as before.

The golden bucket policy pinned aws:SourceArn to
trail/${var.project_name}-baseline while item 5 never named the trail —
an agent naming it anything else produces a bucket policy that rejects
CloudTrail's ACL check at create time. 'Match them exactly' made the
incomplete snippet authoritative, so the gap was guaranteed to ship.

- Item 5 now pins name = "${var.project_name}-baseline" and
  depends_on = [aws_s3_bucket_policy.cloudtrail_logs] on
  aws_cloudtrail.baseline (CloudTrail validates the bucket policy at
  create time).
- Golden HCL completes the chain: aws:SourceArn condition added to the
  PutObject statement (was on GetBucketAcl only), plus the
  aws_s3_bucket_policy attachment and the aws_cloudtrail resource
  itself, with the name/SourceArn coupling called out in comments.

Nits from the same review:
- Item 8: dropped the '(or in tfvars if the user prefers)' phone clause
  (no phone variable exists); merged the two alternate-contact comment
  bullets into one.
- Item 3 now says what item 1 promised: unrecognized compliance values
  get a named header line with the conservative 365-day retention.

Both plugin trees, byte-identical. markdownlint 0, dprint clean, drift
OK, frontmatter validator OK, EB test 11/12 (pre-existing env failure).
@herosjourney

Copy link
Copy Markdown
Contributor Author

Round-2 review addressed — ead24bd

Must-fix (CloudTrail name/policy chain): the golden HCL now carries the complete, internally consistent chain:

  • Item 5's aws_cloudtrail.baseline bullet pins name = "${var.project_name}-baseline" (matching the bucket policy's aws:SourceArn) and depends_on = [aws_s3_bucket_policy.cloudtrail_logs].
  • The golden block adds the aws:SourceArn condition to the PutObject statement (it was on GetBucketAcl only), the aws_s3_bucket_policy attachment, and the aws_cloudtrail resource itself — with the name/SourceArn coupling called out in comments on both sides.
  • The "match them exactly" intro now flags the coupling explicitly: identifiers may vary, the trail name and SourceArn must agree.

The meta-point was taken: an authoritative-but-incomplete snippet is worse than none. The chain is now closed end to end — policy document → attachment → trail, with the create-time-validation ordering explicit.

Nits: phantom "(or in tfvars if the user prefers)" phone clause dropped; the two alternate-contact comment bullets merged into one; item 3 now states what item 1 promised (unrecognized frameworks get a named header line with the conservative 365-day retention). The README "appendices" claim is left alone as pre-existing, per the review.

Not added, per the review: policy gate (PR 2), Estimate line items for baseline services, #229 rebase.

Both trees byte-identical; markdownlint 0 errors, dprint clean, drift OK, frontmatter validator OK, EB test 11/12 with the same pre-existing environmental Terraform version-pin failure.

@leon1418 leon1418 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[🤖 AI review 🤖]

Review of PR #261 — feat(heroku-to-aws): emit baseline.tf account security baseline in Generate
Head: ead24bd80f9b1656a283a9eb7fc0002147e65284 | 16 files | +641/−75

What this change does

Ports the gcp-to-aws Step 1.5 baseline.tf security baseline (account alternate contacts, IAM password policy, S3 account public access block, EBS default encryption, Access Analyzer, IMDSv2 account default, CloudTrail + hardened log bucket, budget alerts, GuardDuty, plus compliance-conditional Config + Security Hub) into heroku-to-aws's Generate phase. Translates it into heroku's DSL idiom — fail-closed _assert postconditions and _check_file_exists gates in frontmatter rather than prose checklists the agent is trusted to follow. Also adds IMDSv2 enforcement on EKS self-managed launch templates, placeholder-rejecting contact email variable validations, MIGRATION_GUIDE opt-out docs, README accuracy fix (ECR scanning claim scoped to where ECR repos actually exist), and fixes the deprecated AWSConfigRole → AWS_ConfigRole policy name at its source in gcp-to-aws.

Findings

0 mandatory blockers. 0 Nits.

Design is sound:

  • The port is not a blind copy — it adapts to heroku-to-aws's DSL architecture (interpreter-enforced postconditions vs advisory prose), which is the right approach. The three new _assert postconditions in generate.md frontmatter cover: (1) always-on resource inventory, (2) compliance-conditional section presence/absence, (3) contact variables with no-default + placeholder-rejecting validation. All three fail _halt_and_inform.
  • Compliance normalization handles the scalar/array polymorphism correctly; "unknown" treated as "none" (conservative — no framework assumed) with unrecognized values getting 365-day conservative retention + file-header comment (correctly NOT in generation-warnings.json whose entries are service-shaped).
  • Budget pinned to projected_costs.aws_monthly_balanced — the same key Estimate postconditions assert — with sensible $50 floor fallback.
  • strcontains usage in validation blocks is compatible with the existing required_version = ">= 1.5.0" constraint.
  • EKS IMDSv2 rider at hop limit 1 (stricter than account default 2) is deliberately documented and correct for plugin-owned templates.
  • Golden HCL covers the three shapes agents historically get wrong: alternate contacts (all 4 required fields), CloudTrail bucket policy (SourceArn-scoped with depends_on), and Config role (current managed policy ARN). Trail name pinned to match SourceArn — the coupling that causes create-time failures when improvised.
  • Two-step opt-out (delete file AND remove variables) is documented in MIGRATION_GUIDE Phase 1 with clear explanation of why both steps are needed.
  • The deprecated AWSConfigRole → AWS_ConfigRole fix in gcp-to-aws is a genuine bug fix (the old name fails terraform apply).

Validation evidence

  • Cross-plugin parity: All 7 heroku-to-aws file pairs (advisor ↔ migrate) verified byte-identical. The 1 gcp-to-aws file pair also identical. No drift allowlist entries added or modified.
  • Frontmatter: generate.md YAML parsed successfully — 14 postcondition items, all structurally valid with _on_failure: _halt_and_inform.
  • README updates: Only the 2 migrate-side READMEs updated; advisor README has no security baseline capability row (confirmed — no counterpart needed).
  • CI: No checks reported yet on this branch.
  • Approvals: 0/0.
  • Mergeable: MERGEABLE (GitHub).

CLEAN. Recommend merge after CI goes green and maintainer approval.

@leon1418
leon1418 merged commit 5c4b316 into awslabs:main Sep 2, 2026
8 checks passed
herosjourney pushed a commit to herosjourney/startups that referenced this pull request Sep 12, 2026
Ports the gcp-to-aws two-touchpoint pattern into heroku-to-aws, translated
into the DSL idiom: a before-write authoring-posture invoke (Step 0) and an
after-write policy gate (Step 12) that writes validation-report.json with
policy_status, with a fail-closed policy_status _assert in generate.md
_postconditions so POLICY_FAIL blocks Generate. Completes the policy-gate half
of P1-A (baseline.tf half landed in awslabs#261).

- tf-best-practices/SKILL.md: consumers note now includes heroku-to-aws
- generate-terraform.md: Step 0 authoring posture + Step 12 policy gate + validation-report.json in _contributes
- generate.md: validation-report.json in _produces/_check_file_exists + policy_status _assert
- generate-assemble.md: policy gate is a required (not optional) completion check
- 2 Heroku-shaped fixtures (POLICY_OK + POLICY_FAIL) + 2 tests; fixture count 23->25
- migrate README scope-note corrected
- policy script unchanged (source-agnostic); EB false-positive check ruled out exemptions

Both plugin trees updated byte-identically.
herosjourney pushed a commit to herosjourney/startups that referenced this pull request Sep 12, 2026
Ports the gcp-to-aws two-touchpoint pattern into heroku-to-aws, translated
into the DSL idiom: a before-write authoring-posture invoke (Step 0) and an
after-write policy gate (Step 12) that writes validation-report.json with
policy_status, with a fail-closed policy_status _assert in generate.md
_postconditions so POLICY_FAIL blocks Generate. Completes the policy-gate half
of P1-A (baseline.tf half landed in awslabs#261).

- tf-best-practices/SKILL.md: consumers note now includes heroku-to-aws
- generate-terraform.md: Step 0 authoring posture + Step 12 policy gate + validation-report.json in _contributes
- generate.md: validation-report.json in _produces/_check_file_exists + policy_status _assert
- generate-assemble.md: policy gate is a required (not optional) completion check
- 2 Heroku-shaped fixtures (POLICY_OK + POLICY_FAIL) + 2 tests; fixture count 23->25
- migrate README scope-note corrected
- policy script unchanged (source-agnostic); EB false-positive check ruled out exemptions

Both plugin trees updated byte-identically.
leon1418 pushed a commit that referenced this pull request Sep 24, 2026
…rges

Doc-only. Reflects capability changes merged into aws-startup-advisor since
mid-August that the top-level docs never picked up, plus two pre-existing
inaccuracies found in the same pass:

- gcp-to-aws: OpenAI sources now land on the same GPT model on Bedrock when
  one exists (#210, #250, #237), not just a Claude/Nova cross-family swap.
  Discover can pull real spend from the OpenAI Admin API, consent-gated
  (#236). GCP Document AI / Vision / Speech-to-Text are detected and routed
  to AWS traditional-AI services (#240, #241).
- Both migration skills: commitment-discount guidance (shared RI/SP
  eligibility matrix, three-state model, dedicated report section — #266,
  #270, #272, #277) was undocumented anywhere in the top-level docs.
- heroku-to-aws: Generate emits `baseline.tf`, an account security baseline,
  alongside the app Terraform (#261).
- setup.md: agent-advisor was missing Lambda MicroVMs; prompt-library counts
  were wrong (30 prompts / 4 agents named "Migration" -> actually 29 prompts
  / 5 agents, none called "Migration") and README's agent list was
  incomplete. Pre-existing, not tied to a specific recent PR.
- knowledge-base-for-startups: bumped the skill's own "Last updated" date to
  2026-09-09 to match the offers refresh in #281 (the date drives its
  6-month staleness nudge to users).

Verified: markdownlint clean. No skill/schema/tool files touched.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants