Add analytics-dataops-expertise skill for DataOps maturity assessment

cloudformation/devops-agent-skill-policies.yaml

@@ -31,6 +31,7 @@ Metadata:
           - EnableAwsBackupCoverageReview
           - EnableAgentCoreObservabilitySetup
           - EnableAgentCoreOpsReview
+          - EnableAnalyticsDataopsExpertise
       - Label:
           default: Optional Resource Scoping
         Parameters:
@@ -141,6 +142,16 @@ Parameters:
     AllowedValues: ['true', 'false']
     Default: 'true'
 
+  EnableAnalyticsDataopsExpertise:
+    Type: String
+    Description: >
+      Analytics DataOps Expertise skill (adds the five read-only actions the
+      maturity scorecard uses that AIDevOpsAgentAccessPolicy does not grant:
+      cost-optimization-hub:ListRecommendations, quicksight:ListDashboards,
+      glue:GetJobs, iam:ListPolicies, lakeformation:GetDataLakeSettings).
+    AllowedValues: ['true', 'false']
+    Default: 'true'
+
 Conditions:
   CreateNewRole: !Equals [!Ref ExistingRoleName, '']
   SkillAwsHealthEvents: !Equals [!Ref EnableAwsHealthEvents, 'true']
@@ -154,6 +165,7 @@ Conditions:
   SkillAwsBackupCoverageReview: !Equals [!Ref EnableAwsBackupCoverageReview, 'true']
   SkillAgentCoreObservabilitySetup: !Equals [!Ref EnableAgentCoreObservabilitySetup, 'true']
   SkillAgentCoreOpsReview: !Equals [!Ref EnableAgentCoreOpsReview, 'true']
+  SkillAnalyticsDataopsExpertise: !Equals [!Ref EnableAnalyticsDataopsExpertise, 'true']
   HasRegionRestriction: !Not [!Equals [!Join ['', !Ref AllowedRegions], '']]
 
 Resources:
@@ -473,6 +485,37 @@ Resources:
               - ec2:DescribeSubnets
             Resource: '*'
 
+  # analytics-dataops-expertise: adds the five read-only actions the maturity
+  # scorecard calls that AIDevOpsAgentAccessPolicy does not grant. Verified against
+  # the managed policy: 53 of the 58 IAM actions the skill uses are already allowed;
+  # these five are the delta.
+  #   - cost-optimization-hub:ListRecommendations — no cost-optimization-hub actions in the managed policy
+  #   - quicksight:ListDashboards               — no quicksight actions in the managed policy
+  #   - glue:GetJobs                            — managed policy grants glue:GetJob (singular) + glue:List*, not the plural Get
+  #   - iam:ListPolicies                        — managed policy grants iam:ListRoles/ListUsers/ListEntitiesForPolicy, not ListPolicies
+  #   - lakeformation:GetDataLakeSettings       — managed policy grants lakeformation Describe*/GetLFTag/GetResourceLFTags/List*, not GetDataLakeSettings
+  # Strictly read-only; no write/mutating action is granted. The skill degrades
+  # gracefully (scores the affected question as a floor) if any of these is absent.
+  PolicyAnalyticsDataopsExpertise:
+    Type: AWS::IAM::Policy
+    Condition: SkillAnalyticsDataopsExpertise
+    Properties:
+      PolicyName: DevOpsAgentSkill-AnalyticsDataopsExpertise
+      Roles:
+        - !If [CreateNewRole, !Ref DevOpsAgentRole, !Ref ExistingRoleName]
+      PolicyDocument:
+        Version: '2012-10-17'
+        Statement:
+          - Sid: DataopsExpertiseReadDelta
+            Effect: Allow
+            Action:
+              - cost-optimization-hub:ListRecommendations
+              - quicksight:ListDashboards
+              - glue:GetJobs
+              - iam:ListPolicies
+              - lakeformation:GetDataLakeSettings
+            Resource: '*'
+
   # Optional: restrict agent to specific regions
   PolicyRegionalRestriction:
     Type: AWS::IAM::Policy
@@ -524,6 +567,7 @@ Outputs:
         - aws-backup-coverage-review: ${EnableAwsBackupCoverageReview} (backup:GetSupportedResourceTypes, config:SelectResourceConfig, dsql:ListClusters, storagegateway:List*)
         - agentcore-observability-setup: ${EnableAgentCoreObservabilitySetup} (bedrock-agentcore:Get/ListAgentRuntime, xray:GetTraceSegmentDestination, logs:DescribeDeliveries/DeliverySources/DeliveryDestinations/ResourcePolicies, lambda:GetFunctionConfiguration, ecs:DescribeTaskDefinition/DescribeServices/ListTasks, eks:DescribeCluster)
         - agentcore-ops-review: ${EnableAgentCoreOpsReview} (bedrock-agentcore read-only List/Get for runtimes/memories/gateways/browsers/code-interpreters/workload-identities, ec2:DescribeSubnets)
+        - analytics-dataops-expertise: ${EnableAnalyticsDataopsExpertise} (cost-optimization-hub:ListRecommendations, quicksight:ListDashboards, glue:GetJobs, iam:ListPolicies, lakeformation:GetDataLakeSettings)
       Skills covered by AIDevOpsAgentAccessPolicy (no extra policy needed):
         - aws-eks-operations-review, eks-upgrade-readiness, enrich-with-aws-security-agent, crm-production-investigation-guidelines
       No IAM required:

llms.txt

@@ -36,6 +36,7 @@ Tools can be used with these AWS DevOps Agent types:
 - [AWS Routing Skill](skills/aws-routing/SKILL.md): Read-only analysis and troubleshooting of AWS routing and BGP path selection across Cloud WAN, Direct Connect, Transit Gateway, VPC, and VPN — traces the end-to-end path, applies each construct's route-evaluation order, engineers traffic with local-preference communities and AS-path, flags non-deterministic selection, and produces describe/get/list validation commands grounded in public AWS documentation
 - [Bedrock Adoption Readiness Skill](skills/bedrock-adoption-readiness/SKILL.md): Assesses an AWS account's readiness to run Amazon Bedrock at production scale across IAM governance, data retention (ZDR), quota and capacity headroom, and operational observability, covering both the standard Bedrock and bedrock-mantle (OpenAI-compatible) surfaces with multi-region discovery
 - [Analytics OpenSearch Expertise Skill](skills/analytics-opensearch-expertise/SKILL.md): Performs read-only health assessments of Amazon OpenSearch Service domains through 24 deterministic checks across cluster health, storage and shards, performance, security, and cost optimization, producing a structured findings report with prioritized remediation guidance
+- [Analytics DataOps Expertise Skill](skills/analytics-dataops-expertise/SKILL.md): Performs read-only DataOps maturity assessments of an AWS data platform through 26 questions scored 1-5 across five dimensions (architecture, security and governance, incident management and observability, automation and testing, and cost), producing a structured scorecard with per-dimension roll-ups and prioritized remediation guidance
 - [FSx for Windows SLA Optimizer Skill](skills/storage-fsx-windows-sla-optimizer/SKILL.md): Reviews one or many Amazon FSx for Windows File Server file systems for SLA readiness across seven availability dimensions (deployment type, Active Directory health, throughput and storage sizing, backups, maintenance window, and alarms) using read-only control-plane calls, with usage-pattern trend analysis (peak-aware throughput sizing, weekday/weekend profile, and storage growth projection) that produces a rated report and flags over-provisioned or idle capacity as cost-optimization opportunities
 - [AI/ML Access Diagnostics Skill](skills/aiml-access-diagnostics/SKILL.md): Diagnoses IAM and access failures for Amazon Bedrock and SageMaker calls by tracing the authorization chain from caller identity through iam:PassRole, role trust policy, role permissions, resource policies, and SCPs to identify which hop denied the call
 - [Bedrock Operation Review Skill](skills/bedrock-operation-review/SKILL.md): Performs comprehensive Amazon Bedrock operational reviews aligned with the AWS Well-Architected Framework and Bedrock best practices across five pillars — security, performance, service quotas, cost optimization, and resilience — using control-plane and CloudWatch APIs only (no model invocations or prompt/response content read)

skills/analytics-dataops-expertise/.skilleval.yaml

@@ -0,0 +1,3 @@
+audit:
+  ignore:
+    - STR-016    # README alongside SKILL.md is intentional

skills/analytics-dataops-expertise/CHANGELOG.md

@@ -0,0 +1,101 @@
+# Changelog
+
+All notable changes to the `analytics-dataops-expertise` skill are documented here.
+
+## [1.1.2] - 2026-09-30
+### Fixed
+- Reframed the IAM section in `SKILL.md` to match `README.md`: it now names
+  `AIDevOpsAgentAccessPolicy`, lists the five supplemental actions with the reason
+  each is not covered, and points at the `EnableAnalyticsDataopsExpertise` gate in
+  `cloudformation/devops-agent-skill-policies.yaml` (previously only `README.md`
+  carried this framing; `SKILL.md` still had the old ViewOnly wording).
+- Removed a duplicate `glue:GetJob` (singular) from the `SKILL.md` permission list
+  so it agrees with `README.md`.
+- Corrected the action count in the CloudFormation template comment: the canonical
+  list has 58 actions, 53 covered by the managed policy and 5 in the gate (was
+  "52 of the 57").
+
+## [1.1.1] - 2026-09-29
+### Added
+- Gated IAM policy `DevOpsAgentSkill-AnalyticsDataopsExpertise` in
+  `cloudformation/devops-agent-skill-policies.yaml` (parameter
+  `EnableAnalyticsDataopsExpertise`) granting the five read-only actions the skill
+  uses that `AIDevOpsAgentAccessPolicy` does not: `cost-optimization-hub:ListRecommendations`,
+  `quicksight:ListDashboards`, `glue:GetJobs`, `iam:ListPolicies`,
+  `lakeformation:GetDataLakeSettings`.
+### Fixed
+- Corrected four IAM action names in the permissions list (docs-only — capability was
+  already granted under the correct name): `mwaa:ListEnvironments` → `airflow:ListEnvironments`,
+  `resourcegroupstaggingapi:GetResources` → `tag:GetResources`,
+  `kinesisanalyticsv2:ListApplications` → `kinesisanalytics:ListApplications`,
+  `s3:GetBucketLifecycleConfiguration` → `s3:GetLifecycleConfiguration`.
+- Reframed the README IAM section against the agent's own `AIDevOpsAgentAccessPolicy`
+  and pointed at the CloudFormation gate for the five supplemental actions.
+
+## [1.1.0] - 2026-08-28
+### Added
+- Optional downloadable **HTML report**: a self-contained, styled template at
+  `assets/templates/dataops-maturity-report.html` (executive score-card tiles, per-dimension
+  cards with RAG summary, per-question expandable detail blocks with confidence badges,
+  observed-metric tiles, and ⚠️/💬 callouts, the 26-row score matrix, and a self-download
+  button) that the agent fills with live data and saves as a chat artifact. Follows the
+  static-template pattern used by the redshift-support-specialist skill — no scripts, no
+  renderer; the agent substitutes values into the template.
+- Companion Markdown template `assets/templates/dataops-maturity-report.md` (the always-on
+  in-chat output) mirroring the HTML structure section-for-section.
+- SKILL.md rules for the HTML path: always produce the Markdown; generate HTML only on
+  request; HTML-escape all substituted values; template is structure-only, never a data
+  source; identical data across both outputs.
+
+## [1.0.3] - 2026-08-28
+### Added
+- Richer report format mirroring the reference pre-assessment layout: an Executive
+  Summary with an overall + per-dimension score-card row and a maturity tier; a
+  scoring-caveat banner listing any unavailable signal APIs; per-dimension RAG summary
+  bands (Strengths ≥3.0 / Watch 2.0–2.9 / Gaps <2.0); and a per-question detail block
+  for all 26 questions with a HIGH/MEDIUM confidence badge, a one-line rationale,
+  real observed metrics, an optional ⚠️ warning callout, and 💬 discussion-ask prompts
+  to drive the customer conversation. Added a Confidence column to the score matrix.
+### Changed
+- Output remains 100% markdown (no scripts, no HTML renderer) so the skill stays
+  within the DevOps Agent no-executable-content constraint; added a guard prohibiting
+  raw structure/dict/JSON dumps in observed-metric values (summarize in words instead).
+
+## [1.0.2] - 2026-08-28
+### Fixed
+- Dimension fidelity: pinned the scorecard to EXACTLY five dimensions (Architecture;
+  Security & Governance; Incident Management & Observability; Automation & Testing; Cost)
+  with fixed per-question membership, and aligned the frontmatter description to those five.
+  Fixes an observed run where the agent reported "7 dimensions" by promoting Architecture
+  questions (Real-time Processing Q4, Resilience Q7/Q8) into standalone top-level dimensions.
+
+## [1.0.1] - 2026-08-28
+### Fixed
+- Report rendering: added a LITERAL-CONTENT RULE and an incremental RENDERING METHOD
+  to the Output Format so the agent writes every section and all 26 score-matrix rows
+  with real computed values. Fixes an observed failure in DevOps Agent where a saved
+  artifact contained placeholder labels ("full 3-tier content…", `{"q":"Q3"}` rows with
+  empty columns) instead of the actual report. Reinforced the same requirement in
+  Execution Flow step 6.
+
+## [1.0.0] - 2026-08-27
+### Added
+- Initial release: 26 read-only control-plane maturity questions across five dimensions
+  (Architecture; Security & Governance; Incident Management & Observability;
+  Automation & Testing; Cost), each scored on a 1-5 maturity scale.
+- Per-question checks that name the AWS API(s), the field(s) to read, and the signal-to-rating
+  mapping — 100% control-plane / API-driven with no data-plane access and no AWS-internal
+  data sources.
+- Auto-score ceilings: questions that cannot be fully confirmed from control-plane signals
+  alone (Q6, Q8, Q11, Q13, Q14, Q19, Q20, Q23, Q24, Q26, Q30, Q37, Q38, Q39) are capped at 3;
+  the report states when a higher score requires conversational confirmation.
+- Remediation Reference: a 26-entry verbatim dictionary (why-it-matters, resolution steps,
+  and verified official AWS documentation links) in `references/remediation-reference.md`,
+  loaded on demand via `read_skill_resource`. Every question scored ≤ 3 pulls its entry
+  verbatim; documentation links come only from the dictionary, eliminating URL hallucination.
+- Structured scorecard output: per-section and overall averages, tiered findings, prioritized
+  recommendations, a raw-data reference, and a mandatory 26-row score matrix with a self-count
+  check.
+- Error handling: missing services and region-unavailable APIs are treated as score-1 signals,
+  not blockers; a question is only SKIPPED when every API it needs is denied by IAM. Cost
+  Explorer must be enabled and is called in `us-east-1`.