Add aws-ecs-operations-review skill

cloudformation/devops-agent-skill-policies.yaml

@@ -20,6 +20,7 @@ Metadata:
           - EnableSupportCases
           - EnableRdsOperationReview
           - EnableEksOperationReview
+          - EnableEcsOperationReview
           - EnableInvestigationCostGuardrail
           - EnableEnrichWithSecurityAgent
           - EnableCrmInvestigationGuidelines
@@ -71,6 +72,12 @@ Parameters:
     AllowedValues: ['true', 'false']
     Default: 'true'
 
+  EnableEcsOperationReview:
+    Type: String
+    Description: ECS Operation Review skill (adds compute-optimizer:GetECSServiceRecommendations for the PERF8 rightsizing check).
+    AllowedValues: ['true', 'false']
+    Default: 'true'
+
   EnableInvestigationCostGuardrail:
     Type: String
     Description: Investigation Cost Guardrail skill.
@@ -130,6 +137,7 @@ Conditions:
   SkillAwsHealthEvents: !Equals [!Ref EnableAwsHealthEvents, 'true']
   SkillSupportCases: !Equals [!Ref EnableSupportCases, 'true']
   SkillRdsOperationReview: !Equals [!Ref EnableRdsOperationReview, 'true']
+  SkillEcsOperationReview: !Equals [!Ref EnableEcsOperationReview, 'true']
   SkillInvestigationCostGuardrail: !Equals [!Ref EnableInvestigationCostGuardrail, 'true']
   SkillMskOperations: !Equals [!Ref EnableMskOperations, 'true']
   SkillServiceQuotaCheck: !Equals [!Ref EnableServiceQuotaCheck, 'true']
@@ -236,6 +244,24 @@ Resources:
               - logs:GetLogEvents
             Resource: '*'
 
+  # ecs-operation-review: adds compute-optimizer:GetECSServiceRecommendations (PERF8 rightsizing);
+  # every other API the skill uses is covered by AIDevOpsAgentAccessPolicy
+  PolicyEcsOperationReview:
+    Type: AWS::IAM::Policy
+    Condition: SkillEcsOperationReview
+    Properties:
+      PolicyName: DevOpsAgentSkill-EcsOperationReview
+      Roles:
+        - !If [CreateNewRole, !Ref DevOpsAgentRole, !Ref ExistingRoleName]
+      PolicyDocument:
+        Version: '2012-10-17'
+        Statement:
+          - Sid: ComputeOptimizerEcsRecommendations
+            Effect: Allow
+            Action:
+              - compute-optimizer:GetECSServiceRecommendations
+            Resource: '*'
+
   # investigation-cost-guardrail: adds pricing:GetProducts
   PolicyInvestigationCostGuardrail:
     Type: AWS::IAM::Policy
@@ -447,6 +473,7 @@ Outputs:
         - aws-health-events: ${EnableAwsHealthEvents} (health:DescribeEventTypes)
         - support-cases: ${EnableSupportCases} (support:DescribeCommunications)
         - rds-operation-review: ${EnableRdsOperationReview} (rds:DownloadDBLogFilePortion, logs:GetLogEvents)
+        - ecs-operation-review: ${EnableEcsOperationReview} (compute-optimizer:GetECSServiceRecommendations)
         - investigation-cost-guardrail: ${EnableInvestigationCostGuardrail} (pricing:GetProducts)
         - msk-operations: ${EnableMskOperations} (kafka:GetBootstrapBrokers)
         - service-quota-check: ${EnableServiceQuotaCheck} (servicequotas:*, cloudwatch:GetMetricData/GetMetricStatistics)

llms.txt

@@ -28,6 +28,7 @@ Tools can be used with these AWS DevOps Agent types:
 - [Enrich with AWS Security Agent Skill](skills/enrich-with-aws-security-agent/SKILL.md): Queries AWS Security Agent CloudWatch logs to retrieve code-level security findings (file, line number, vulnerability type) during incident investigations with potential security root causes
 - [Wiz Security Context Skill](skills/wiz-security-context/SKILL.md): Queries the Wiz MCP server for a resource's security context (vulnerabilities, misconfigurations, secrets, active threats, malware, toxic combinations) to determine whether an operational anomaly is an operational issue or a security incident
 - [Service Quota Check Skill](skills/service-quota-check/SKILL.md): Checks AWS service quota utilization during investigations and before provisioning resources, flags quotas at 85%+ utilization, and requests increases via the Service Quotas API or recommends support cases
+- [ECS Operation Review Skill](skills/ecs-operation-review/SKILL.md): Performs comprehensive Amazon ECS operations reviews across 6 pillars (Resiliency & HA, Observability, Security, Operations, Performance, Additional Analysis) using read-only AWS APIs, with a 7-day CloudWatch metrics baseline, per-pillar PASS/FAIL/N/A scorecards, recommended alarm thresholds for IDR onboarding, and a prioritized remediation-linked report
 - [DMS Operational Review Skill](skills/database-migration-service-expertise/SKILL.md): Conducts AWS Database Migration Service operational reviews with 5-category health scoring, task failure troubleshooting, migration cutover runbooks, version deprecation tracking, and cost optimization
 - [Redshift Support Specialist Skill](skills/redshift-support-specialist/SKILL.md): Amazon Redshift domain expertise for query optimization, operational reviews, and cost optimization on provisioned clusters and Serverless workgroups, via the awslabs.redshift-mcp-server MCP server
 - [S3 Resiliency Review Skill](skills/storage-s3-resiliency-expertise/SKILL.md): Reviews one or many S3 buckets across nine resiliency, security, and data-protection dimensions using read-only control-plane calls, producing a rated report with prioritized findings and remediation guidance

skills/ecs-operation-review/.skilleval.yaml

@@ -0,0 +1,3 @@
+audit:
+  ignore:
+    - STR-016    # README alongside SKILL.md is intentional

skills/ecs-operation-review/CHANGELOG.md

@@ -0,0 +1,114 @@
+# Changelog
+
+## [2.5.0] - 2026-09-24
+
+Review feedback from PR #42 (70 baseline checks, up from 69):
+
+- **Renamed `aws-ecs-operations-review` → `ecs-operation-review`** to match
+  the skill family convention (`eks-operation-review`,
+  `rds-operation-review`): no `aws-` prefix, singular "operation". Directory,
+  frontmatter `name`, and the `llms.txt` entry all updated.
+- **Compute Optimizer permission gated in CloudFormation** — the live
+  `AIDevOpsAgentAccessPolicy` grants no `compute-optimizer` actions, so
+  PERF8's `computeoptimizer.getECSServiceRecommendations` call would return
+  AccessDenied under the agent's own role. Added
+  `EnableEcsOperationReview` parameter, condition, inline policy
+  (`compute-optimizer:GetECSServiceRecommendations`), and a
+  `SkillPolicySummary` line to
+  `cloudformation/devops-agent-skill-policies.yaml`; reframed the README
+  permissions section around `AIDevOpsAgentAccessPolicy` coverage instead
+  of `ReadOnlyAccess`.
+- **Strict docs build fixed** — removed the two relative `.md` links in
+  `README.md` (`evals/TESTING.md`, `references/report-format.md`) that
+  failed `mkdocs build --strict`.
+- **Frontmatter version aligned with this changelog** (was `1.0.0`), and
+  frontmatter `author` set to the GitHub username.
+- **OPS9 (new): ECS Exec audit logging** — when `enableExecuteCommand` is
+  true, grade the cluster's `executeCommandConfiguration.logging`
+  (CloudWatch Logs / S3, not `NONE`) via `ecs.describeClusters` with
+  `include=["CONFIGURATIONS"]`. OPS3 keeps grading whether Exec is enabled
+  in production; OPS9 grades whether enabled Exec sessions leave an audit
+  trail. N/A when Exec is disabled.
+- **SEC19 applicability fixed** — GuardDuty Runtime Monitoring does not
+  support ECS Managed Instances; "Applies To" narrowed from All to
+  Fargate/EC2 and the MI N/A rule added to the compute-platform note in
+  `references/checks.md`.
+- **SEC5 caveat** — `readonlyRootFilesystem` is incompatible with ECS Exec;
+  the recommendation now says to note the tradeoff where Exec is in use
+  instead of recommending the setting unconditionally.
+- **Header fixes** — `pillars/operations.md` said OPS1-OPS7 (now OPS1-OPS9)
+  and `pillars/security.md` said SEC1-SEC19 (now SEC1-SEC20); both headers
+  now match their tables and the checks index.
+
+## [2.4.0] - 2026-08-09
+
+Capacity provider depth + compute platform awareness (69 baseline checks, up
+from 64):
+
+- **Compute platform decision step** (new workflow step 4, rules in
+  `references/checks.md`): the agent now classifies the service as Fargate
+  (± Spot), EC2 ASG capacity provider, **ECS Managed Instances**
+  (`managedInstancesProvider`), launchType-only, or ECS Anywhere — including
+  mixed strategies — and this decision drives "Applies To" applicability in
+  every pillar. New applicability values: `EC2-ASG-CP`, `MI`, `CP-strategy`.
+- **PERF7 deepened** (Low → Medium): flags launchType-only services (ignored
+  by managed scaling) and missing cluster `defaultCapacityProviderStrategy`.
+- **PERF9 (new)**: managed scaling enabled with `targetCapacity` headroom
+  (80-100, <100 for spiky workloads) and `instanceWarmupPeriod` sanity.
+- **PERF10 (new)**: metrics-driven capacity analysis — 7-day
+  `CapacityProviderReservation` (AWS/ECS/ManagedScaling) baseline compared
+  against configured `targetCapacity` to detect capacity-constrained
+  scale-outs vs idle over-provisioning.
+- **PERF11 (new)**: capacity provider strategy base/weight design — on-demand
+  base for production, Spot burst by weight, task-size-fits-instance check.
+- **REL14 (new)**: capacity provider infrastructure multi-AZ (ASG subnets or
+  Managed Instances `networkConfiguration.subnets` span 2+ AZs).
+- **ADD7 (new)**: ECS Managed Instances migration evaluation for self-managed
+  EC2 services, keyed off OPS6/OPS7 (agent/AMI currency) signals.
+- **OPS6/OPS7/OPS2** now explicitly N/A for Managed Instances (AWS manages
+  agent/AMI lifecycle).
+- **alarm-thresholds.md**: new Capacity Provider Alarms section
+  (`CapacityProviderReservation` saturation alarm) and baseline-metrics row;
+  report header now records the resolved compute platform.
+
+## [2.3.2] - 2026-08-09
+
+Fix skill upload rejection (`400 ValidationException` from the AWS DevOps Agent
+Asset API):
+
+- Reduced `SKILL.md` frontmatter to **only `name` and `description`**, the
+  fields the DevOps Agent uploader supports for zip skills. Removed the
+  `license`, `compatibility`, and nested `metadata` blocks added in 2.3.1 — the
+  DevOps Agent parser reads only `name`/`description` from frontmatter and
+  rejects the extra keys. `agent_types` and other asset metadata are supplied
+  in the Asset API request (or the Operator Web App) at upload time, not in
+  frontmatter. Description (with its trigger phrases) is unchanged and within
+  the 1024-char limit.
+
+## [2.3.1] - 2026-08-09
+
+Compliance with the AgentSkills.io open standard (aligns this skill with the
+`aws-eks-operations-review` skill):
+
+- Renamed directory to `ecs-operation-review` (registry
+  `aws-<service>-<capability>` naming convention).
+- Rewrote SKILL.md frontmatter to the spec: only `name`, `description`,
+  `license`, `compatibility`, and `metadata` at the top level. Moved `version`
+  and `tags` inside `metadata:`; added `license`, `compatibility`, and the
+  `aws-devops-agent-skills.*` + `devops-agent-tools.*` registry metadata.
+  Front-loaded the `description` with trigger phrases for discovery.
+- Fixed the `name` field to match the directory (`ecs-operation-review`).
+- Renamed `reference/` → `references/` (spec convention) and updated all
+  SKILL.md links.
+- Added `README.md` (packaging / prerequisites / upload / usage) and an
+  `evals/` harness (routing + knowledge evals) mirroring the EKS skill. No
+  change to the assessment workflow, pillars, checks, or report format.
+
+## [2.3.0] - 2026-08-09
+
+- Baseline: comprehensive ECS operations review across the 6 review pillars
+  (Resiliency & HA, Observability, Security, Operations, Performance,
+  Additional Analysis) with a 7-day CloudWatch metrics baseline, recommended
+  alarm thresholds for IDR onboarding, per-pillar ✓/✗/N/A scorecards, a
+  coverage gate, and the `review-common` baseline crosswalk. Read-only AWS API
+  data collection; Markdown report by default, DOCX on request.

skills/ecs-operation-review/README.md

@@ -0,0 +1,117 @@
+# ECS Operations Review — AWS DevOps Agent Skill
+
+An end-to-end Amazon ECS operational review skill for [AWS DevOps Agent](https://docs.aws.amazon.com/devopsagent/latest/userguide/about-aws-devops-agent.html). It assesses an ECS service against the ECS Best Practices Guide across six review pillars and produces a prioritized, remediation-linked report artifact per service, plus recommended CloudWatch alarms for IDR onboarding.
+
+It is **strictly read-only**: it uses only `describe*` / `list*` / `get*` AWS API operations (ECS, CloudWatch, IAM, Application Auto Scaling, ELB, ECR, EC2, GuardDuty, Compute Optimizer). It never runs a mutating call; remediations are drafted for human approval, not applied.
+
+> ⚠️ **Non-production disclaimer.** This skill is sample code, not intended for
+> production use without additional review and testing. Users should validate in
+> a non-production environment first.
+
+## What It Does
+
+1. **Parse the service ARN** (`arn:aws*:ecs:*:*:service/*/*`) and validate region.
+2. **Collect configuration data** across the API tier dependency chain — Tier 1 (`ecs.describeServices`) must succeed first; non-Tier-1 access errors mark dependent checks N/A and continue.
+3. **Resolve the compute platform** — Fargate (± Spot), EC2 ASG capacity provider, ECS Managed Instances, launchType-only, or ECS Anywhere — from the capacity provider strategy and `ecs.describeCapacityProviders`; this decision drives check applicability in every pillar.
+4. **Run pillar checks** — read `references/checks.md` (the index) first, then each `references/pillars/<pillar>.md` one at a time, grading ✓ / ✗ / N/A with evidence, severity, and a recommendation.
+5. **Report** — write a per-service artifact following `references/report-format.md`: workload details, per-pillar scorecards for all 6 pillars, prioritized action plan, detailed findings, a Recommended CloudWatch Alarms table, access limitations, and a review summary.
+
+Pillars graded: **Resiliency & HA (REL), Observability (OBS), Security (SEC), Operations (OPS), Performance (PERF), and Additional Analysis (ADD)**.
+
+## Data Sources
+
+| Source | Used for | Required? |
+|--------|----------|-----------|
+| Read-only AWS APIs (AWS CLI / SDK / AWS API MCP) | All configuration data across the six pillars | Yes |
+| CloudWatch `getMetricStatistics` | 7-day baseline (CPU, memory, task count) + `describeAlarms` | Yes (limitation noted if <7 days) |
+| AWS Knowledge MCP | Doc-link lookups for findings and alarm recommendations | Yes |
+
+## Agent Types
+
+Intended for these agent types (selected in the Operator Web App at upload time):
+
+- **On-demand** — conversational invocation in Chat ("run an ECS operations review on service X", "ECS security review").
+- **Evaluation** — proactive operational improvement recommendations.
+
+Select **Generic** to make the skill available to all agent types.
+
+## Prerequisites
+
+### 1. An AWS DevOps Agent Space with the target AWS account
+
+An existing [Agent Space](https://docs.aws.amazon.com/devopsagent/latest/userguide/getting-started-with-aws-devops-agent-creating-an-agent-space.html) with the target AWS account configured as a cloud source.
+
+### 2. Read-only permissions
+
+The Agent Space IAM role needs read-only (`describe*` / `list*` / `get*`) access to: ECS, CloudWatch, CloudWatch Logs, IAM, Application Auto Scaling, Elastic Load Balancing v2, ECR, EC2/VPC, GuardDuty, and Compute Optimizer. The DevOps Agent managed policy **`AIDevOpsAgentAccessPolicy`** covers all of these except one: it grants no `compute-optimizer` actions, so the PERF8 rightsizing check needs `compute-optimizer:GetECSServiceRecommendations` added. Deploy `cloudformation/devops-agent-skill-policies.yaml` with `EnableEcsOperationReview=true` to attach it as a gated inline policy. Without it the skill still runs — PERF8 is marked N/A under the access-limitation protocol. No cluster-level access entry or kubectl connectivity is required — ECS is assessed entirely through AWS control-plane APIs.
+
+### 3. AWS Knowledge MCP
+
+Used for documentation-link lookups on findings and alarm recommendations. This is built into AWS DevOps Agent.
+
+## Packaging the skill
+
+From the directory **containing** `ecs-operation-review/`:
+
+```bash
+zip -r ecs-operation-review.zip ecs-operation-review/ \
+  -i '*.md' '*.txt' '*.json' '*.yaml' '*.yml' \
+  -x '*/.git/*' '*/evals/*' '*/CHANGELOG.md' '*/README.md' '*.DS_Store'
+```
+
+The uploaded zip contains:
+
+```
+ecs-operation-review/
+├── SKILL.md                    # frontmatter + skill instructions (required)
+└── references/
+    ├── checks.md               # checks index (read first)
+    ├── alarm-thresholds.md      # recommended CloudWatch alarm thresholds
+    ├── common-checks-coverage.md
+    ├── report-format.md
+    └── pillars/                # one file per pillar (REL/OBS/SEC/OPS/PERF/ADD)
+```
+
+Upload-time constraints: `SKILL.md` required with `name` + `description` frontmatter; **no `scripts/` directory**. `evals/`, `README.md`, and `CHANGELOG.md` are dev-only and excluded above.
+
+## Uploading to AWS DevOps Agent
+
+> Reference: [Uploading a skill](https://docs.aws.amazon.com/devopsagent/latest/userguide/about-aws-devops-agent-devops-agent-skills.html#uploading-a-skill)
+
+1. Open the **Skills** page in your Agent Space Operator Web App.
+2. **Add skill** → **Upload skill**.
+3. Drag and drop `ecs-operation-review.zip`.
+4. Select agent types: **On-demand** and **Evaluation** (or **Generic**).
+5. Review validation results → **Upload**.
+
+## Usage
+
+In Chat, use natural language:
+
+- *"Run an ECS operations review on service `arn:aws:ecs:us-east-1:111122223333:service/prod/web`."*
+- *"ECS security review for the `web` service in cluster `prod`."*
+- *"Assess reliability and cost for my ECS services."*
+
+The agent validates the service ARN, collects read-only AWS data, grades the six pillars, and writes a per-service review artifact.
+
+## Evaluation
+
+The `evals/` directory holds an evaluation harness:
+
+- `eval_queries.json` — routing checks (does the right query trigger the skill?).
+- `evals.json` — skill-knowledge evals (six pillars, read-only contract, ARN validation, coverage gate, alarm deliverable), run against `evals/files/service-context.json`.
+
+Run them with your skill-eval runner. Record results in `evals/TESTING.md` (model × eval-suite pass-rate matrix) and re-run after any change to the frontmatter or workflow steps. Results are recorded from real runs, never fabricated.
+
+## Severity
+
+Internally the skill grades on `Critical / High / Medium / Low / Info` tiers; the report writer maps these to customer-facing descriptive labels (see `references/report-format.md`). It never emits internal severity numbers in customer-facing output.
+
+## Source attribution
+
+- [About AWS DevOps Agent](https://docs.aws.amazon.com/devopsagent/latest/userguide/about-aws-devops-agent.html)
+- [Amazon ECS Best Practices Guide](https://docs.aws.amazon.com/AmazonECS/latest/bestpracticesguide/intro.html)
+
+## License
+
+Internal use.