Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion docs/coverage/aws/cloudwatch.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,4 +22,6 @@ AWS's `monitoring` service · portable interface `driver.Monitoring` · [AWS ind

## Not in scope

_Not documented yet. See the [emulator boundary](../../../README.md) for cloudemu-wide non-goals._
- CloudWatch `GetMetricWidgetImage` is not implemented. It renders a metric graph as a PNG image, which is presentation, not control-plane state, and the emulator has no graph renderer. A call gets the unknown-operation error.
- CloudWatch metric math evaluates arithmetic, comparison and logical operators, `IF`, `FILL`, `SEARCH` and a whole-expression `ANOMALY_DETECTION_BAND`. Other functions (such as `SUM`, `METRICS`, `RATE`) are stored and return no data. The label of each `SEARCH` result (dimension values, then the metric name) and the labels of the two band rows are the emulator's choice, because the AWS docs do not state them. `FILL` with `REPEAT` or `LINEAR` leaves a gap before the first value (and, for `LINEAR`, after the last one) empty.
- The CloudWatch anomaly band is mean -/+ k standard deviations of the past two weeks, not AWS's trained model. `AnomalyDetectorId` and the `AnomalyDetectorIds` filter are not returned or accepted.
4 changes: 3 additions & 1 deletion docs/coverage/azure/monitor.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,4 +35,6 @@ ActivityLogRecorder is an OPTIONAL capability, discovered by type assertion

## Not in scope

_Not documented yet. See the [emulator boundary](../../../README.md) for cloudemu-wide non-goals._
- CloudWatch `GetMetricWidgetImage` is not implemented. It renders a metric graph as a PNG image, which is presentation, not control-plane state, and the emulator has no graph renderer. A call gets the unknown-operation error.
- CloudWatch metric math evaluates arithmetic, comparison and logical operators, `IF`, `FILL`, `SEARCH` and a whole-expression `ANOMALY_DETECTION_BAND`. Other functions (such as `SUM`, `METRICS`, `RATE`) are stored and return no data. The label of each `SEARCH` result (dimension values, then the metric name) and the labels of the two band rows are the emulator's choice, because the AWS docs do not state them. `FILL` with `REPEAT` or `LINEAR` leaves a gap before the first value (and, for `LINEAR`, after the last one) empty.
- The CloudWatch anomaly band is mean -/+ k standard deviations of the past two weeks, not AWS's trained model. `AnomalyDetectorId` and the `AnomalyDetectorIds` filter are not returned or accepted.
4 changes: 3 additions & 1 deletion docs/coverage/gcp/cloudmonitoring.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,4 +22,6 @@ GCP's `monitoring` service · portable interface `driver.Monitoring` · [GCP ind

## Not in scope

_Not documented yet. See the [emulator boundary](../../../README.md) for cloudemu-wide non-goals._
- CloudWatch `GetMetricWidgetImage` is not implemented. It renders a metric graph as a PNG image, which is presentation, not control-plane state, and the emulator has no graph renderer. A call gets the unknown-operation error.
- CloudWatch metric math evaluates arithmetic, comparison and logical operators, `IF`, `FILL`, `SEARCH` and a whole-expression `ANOMALY_DETECTION_BAND`. Other functions (such as `SUM`, `METRICS`, `RATE`) are stored and return no data. The label of each `SEARCH` result (dimension values, then the metric name) and the labels of the two band rows are the emulator's choice, because the AWS docs do not state them. `FILL` with `REPEAT` or `LINEAR` leaves a gap before the first value (and, for `LINEAR`, after the last one) empty.
- The CloudWatch anomaly band is mean -/+ k standard deviations of the past two weeks, not AWS's trained model. `AnomalyDetectorId` and the `AnomalyDetectorIds` filter are not returned or accepted.
3 changes: 3 additions & 0 deletions docs/coverage/nongoals/monitoring.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
- CloudWatch `GetMetricWidgetImage` is not implemented. It renders a metric graph as a PNG image, which is presentation, not control-plane state, and the emulator has no graph renderer. A call gets the unknown-operation error.
- CloudWatch metric math evaluates arithmetic, comparison and logical operators, `IF`, `FILL`, `SEARCH` and a whole-expression `ANOMALY_DETECTION_BAND`. Other functions (such as `SUM`, `METRICS`, `RATE`) are stored and return no data. The label of each `SEARCH` result (dimension values, then the metric name) and the labels of the two band rows are the emulator's choice, because the AWS docs do not state them. `FILL` with `REPEAT` or `LINEAR` leaves a gap before the first value (and, for `LINEAR`, after the last one) empty.
- The CloudWatch anomaly band is mean -/+ k standard deviations of the past two weeks, not AWS's trained model. `AnomalyDetectorId` and the `AnomalyDetectorIds` filter are not returned or accepted.
4 changes: 3 additions & 1 deletion docs/coverage/oci/monitoring.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,4 +22,6 @@ OCI's `monitoring` service · portable interface `driver.Monitoring` · [OCI ind

## Not in scope

_Not documented yet. See the [emulator boundary](../../../README.md) for cloudemu-wide non-goals._
- CloudWatch `GetMetricWidgetImage` is not implemented. It renders a metric graph as a PNG image, which is presentation, not control-plane state, and the emulator has no graph renderer. A call gets the unknown-operation error.
- CloudWatch metric math evaluates arithmetic, comparison and logical operators, `IF`, `FILL`, `SEARCH` and a whole-expression `ANOMALY_DETECTION_BAND`. Other functions (such as `SUM`, `METRICS`, `RATE`) are stored and return no data. The label of each `SEARCH` result (dimension values, then the metric name) and the labels of the two band rows are the emulator's choice, because the AWS docs do not state them. `FILL` with `REPEAT` or `LINEAR` leaves a gap before the first value (and, for `LINEAR`, after the last one) empty.
- The CloudWatch anomaly band is mean -/+ k standard deviations of the past two weeks, not AWS's trained model. `AnomalyDetectorId` and the `AnomalyDetectorIds` filter are not returned or accepted.
2 changes: 1 addition & 1 deletion providers/aws/cloudwatch/anomaly_detectors.go
Original file line number Diff line number Diff line change
Expand Up @@ -243,7 +243,7 @@ func (m *Mock) detectorPoints(d *driver.AnomalyDetector, now time.Time) int {
return 0
}

s, err := metricmath.New(d.Metrics, m.rangeFetcher(start, end)).Resolve(watched[0].ID)
s, err := metricmath.New(d.Metrics, m.rangeFetcher(start, end)).WithRange(start, end).Resolve(watched[0].ID)
if err != nil {
return 0
}
Expand Down
2 changes: 1 addition & 1 deletion providers/aws/cloudwatch/math_alarms.go
Original file line number Diff line number Diff line change
Expand Up @@ -187,7 +187,7 @@ func (m *Mock) mathDatums(a *alarmData, p *alarmeval.Params, at time.Time) []dri
// Reads end before EndTime. Shifting the window by 1ns keeps a datum put
// at the evaluation instant, as a plain alarm does.
start, end := p.WindowStart(at).Add(time.Nanosecond), at.Add(time.Nanosecond)
ev := metricmath.New(a.Metrics, m.rangeFetcher(start, end))
ev := metricmath.New(a.Metrics, m.rangeFetcher(start, end)).WithRange(start, end)

bandID, _, isBand := bandThreshold(a)
if isBand {
Expand Down
6 changes: 3 additions & 3 deletions providers/aws/cloudwatch/math_alarms_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -89,11 +89,11 @@ func TestMathAlarmUnsupportedExpression(t *testing.T) {

putMath(t, m, fc, "Errors", 30)

cfg := rateAlarm("fill")
cfg.Metrics[2].Expression = "FILL(err, 0)"
cfg := rateAlarm("rate-fn")
cfg.Metrics[2].Expression = "RATE(err)"

requireNoError(t, m.CreateAlarm(ctx, cfg))
assertEqual(t, stateInsufficientData, stateOf(t, m, "fill"))
assertEqual(t, stateInsufficientData, stateOf(t, m, "rate-fn"))
}

// The state change event lists the math alarm's queries.
Expand Down
38 changes: 4 additions & 34 deletions providers/aws/cloudwatch/period.go
Original file line number Diff line number Diff line change
@@ -1,48 +1,18 @@
package cloudwatch

import (
"math"
"slices"

"github.com/stackshy/cloudemu/v2/errors"
"github.com/stackshy/cloudemu/v2/services/monitoring/metricmath"
)

// Period rules from the MetricStat, MetricDataQuery and GetMetricStatistics
// API references. Period is an Integer of at least 1. A regular resolution
// period is a multiple of 60, and a high-resolution metric also allows 1, 5,
// 10, 20 and 30.
const (
defaultPeriodSeconds = 60
maxPeriodSeconds = math.MaxInt32
periodSecondsInMinute = 60
)

// validPeriod reports whether p is 1, 5, 10, 20, 30 or a multiple of 60 that
// fits the API's Integer type.
func validPeriod(p int) bool {
switch {
case p < 1 || p > maxPeriodSeconds:
return false
case p%periodSecondsInMinute == 0:
return true
default:
periods := highResolutionPeriods()

return slices.Contains(periods[:], p)
}
}

// highResolutionPeriods are the periods under a minute that a high-resolution
// metric allows.
func highResolutionPeriods() [5]int {
return [5]int{1, 5, 10, 20, 30}
}
// defaultPeriodSeconds is the period of a metric read that sets none.
const defaultPeriodSeconds = 60

// ValidatePeriod returns an InvalidArgument error when period is not a valid
// metric period. param names the request field in the message, such as
// "MetricDataQueries.member.1.MetricStat.Period".
func (*Mock) ValidatePeriod(param string, period int) error {
if validPeriod(period) {
if metricmath.ValidPeriod(period) {
return nil
}

Expand Down
61 changes: 55 additions & 6 deletions server/aws/cloudwatch/core_metric_data.go
Original file line number Diff line number Diff line change
Expand Up @@ -95,8 +95,13 @@ type periodValidator interface {
}

// validateQueryPeriods checks the MetricStat Period and the optional Period
// of every query, including ones that return no data.
// of every query, including ones that return no data, and the period
// argument of every SEARCH.
func (h *Handler) validateQueryPeriods(queries []metricDataQueryCBR) error {
if err := validateSearchPeriods(queries); err != nil {
return err
}

v, ok := h.monitoring.(periodValidator)
if !ok {
return nil
Expand All @@ -121,6 +126,19 @@ func (h *Handler) validateQueryPeriods(queries []metricDataQueryCBR) error {
return nil
}

// validateSearchPeriods rejects an expression whose SEARCH period argument
// is not a valid metric period, the same rule a MetricStat Period follows.
func validateSearchPeriods(queries []metricDataQueryCBR) error {
for i := range queries {
if metricmath.InvalidSearchPeriod(queries[i].Expression) {
return newWireError(errValidation, "Error in expression '"+queries[i].ID+
"': Invalid SEARCH period. Valid periods are 1, 5, 10, 20, 30, 60 and any multiple of 60.")
}
}

return nil
}

// invalidParameterError maps a driver InvalidArgument error to the
// InvalidParameterValue wire error and passes any other error through.
func invalidParameterError(err error) error {
Expand All @@ -138,7 +156,9 @@ type bandExclusionSource interface {
}

// metricDataEvaluator builds the evaluator for one GetMetricData call. A band
// trains on the two weeks before start as well.
// trains on the two weeks before start as well. FILL fills the periods of
// [start, end), and SEARCH finds the metrics ListMetrics lists: those with
// data in the past two weeks.
func (h *Handler) metricDataEvaluator(
ctx context.Context, queries []mondriver.MetricDataQuery, start, end time.Time,
) *metricmath.Evaluator {
Expand All @@ -148,11 +168,18 @@ func (h *Handler) metricDataEvaluator(
cfg.Excluded = src.BandExclusions
}

return metricmath.New(queries, h.metricFetcher(ctx, start, end)).WithBand(cfg)
search := func() ([]mondriver.MetricIdentifier, error) { return h.allMetricRows(ctx, listMetricsSpan) }

return metricmath.New(queries, h.metricFetcher(ctx, start, end)).
WithBand(cfg).
WithRange(start, end).
WithSearch(search)
}

// metricDataRows returns the result rows of one query. A band returns two
// rows with the query's Id, the lower edge and then the upper edge.
// rows with the query's Id, the lower edge and then the upper edge, and an
// expression that returns an array of series, such as a SEARCH, returns one
// row per series, all with the query's Id.
func metricDataRows(eval *metricmath.Evaluator, q *metricDataQueryCBR, descending bool) ([]metricDataRow, error) {
band, isBand, err := eval.Band(q.ID)
if err != nil {
Expand All @@ -166,12 +193,34 @@ func metricDataRows(eval *metricmath.Evaluator, q *metricDataQueryCBR, descendin
return []metricDataRow{buildMetricDataRow(q, lower, descending), buildMetricDataRow(q, upper, descending)}, nil
}

series, err := eval.Resolve(q.ID)
all, err := eval.ResolveAll(q.ID)
if err != nil {
return nil, err
}

return []metricDataRow{buildMetricDataRow(q, series, descending)}, nil
rows := make([]metricDataRow, 0, len(all))

for _, s := range all {
row := buildMetricDataRow(q, s.Series, descending)
if s.Label != "" {
row.Label = seriesLabel(q.Label, s.Label)
}

rows = append(rows, row)
}

return rows, nil
}

// seriesLabel is the label of one series of an array result: the query's
// Label, if any, then the series' own label, as the CloudWatch console
// captions such a series ("Expression-Label Metric-Label").
func seriesLabel(queryLabel, ownLabel string) string {
if queryLabel == "" {
return ownLabel
}

return queryLabel + " " + ownLabel
}

// metricFetcher reads one metric from the monitoring driver over [start, end).
Expand Down
4 changes: 2 additions & 2 deletions server/aws/cloudwatch/dashboards.go
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ func (h *Handler) getDashboard(w http.ResponseWriter, r *http.Request, body []by

d, err := store.GetDashboard(r.Context(), in.DashboardName)
if err != nil {
writeDriverErr(w, err)
writeDriverErr(w, dashboardErr(err))
return
}

Expand Down Expand Up @@ -196,7 +196,7 @@ func (h *Handler) deleteDashboards(w http.ResponseWriter, r *http.Request, body
}

if err := store.DeleteDashboards(r.Context(), in.DashboardNames); err != nil {
writeDriverErr(w, err)
writeDriverErr(w, dashboardErr(err))
return
}

Expand Down
49 changes: 49 additions & 0 deletions server/aws/cloudwatch/dashboards_test.go
Original file line number Diff line number Diff line change
@@ -1,10 +1,14 @@
package cloudwatch_test

import (
"errors"
"net/http"
"testing"

"github.com/aws/aws-sdk-go-v2/aws"
awshttp "github.com/aws/aws-sdk-go-v2/aws/transport/http"
awscw "github.com/aws/aws-sdk-go-v2/service/cloudwatch"
cwtypes "github.com/aws/aws-sdk-go-v2/service/cloudwatch/types"
)

// TestSDKDashboardLifecycle drives the aws-sdk-go-v2 client through the
Expand Down Expand Up @@ -98,3 +102,48 @@ func TestSDKGetDashboardNotFound(t *testing.T) {
t.Fatal("GetDashboard for unknown name: expected error, got nil")
}
}

// assertDashboardNotFound checks err is the model's DashboardNotFoundError:
// the SDK maps it to types.DashboardNotFoundError, its code is the awsQuery
// code ResourceNotFound and the status is 404, as in the CloudWatch API
// reference and the service model (error code ResourceNotFound, 404).
func assertDashboardNotFound(t *testing.T, op string, err error) {
t.Helper()

var nf *cwtypes.DashboardNotFoundError
if !errors.As(err, &nf) {
t.Fatalf("%s: error %T (%v) is not *types.DashboardNotFoundError", op, err, err)
}

if code := nf.ErrorCode(); code != "ResourceNotFound" {
t.Fatalf("%s: ErrorCode = %q, want ResourceNotFound", op, code)
}

var re *awshttp.ResponseError
if !errors.As(err, &re) || re.HTTPStatusCode() != http.StatusNotFound {
t.Fatalf("%s: want HTTP 404, got %v", op, err)
}
}

// TestSDKDashboardNotFoundShape covers GetDashboard and DeleteDashboards on an
// unknown dashboard. DeleteDashboards is all-or-nothing, so the existing
// dashboard named with the missing one must survive.
func TestSDKDashboardNotFoundShape(t *testing.T) {
client, ctx := newCWClient(t)

_, err := client.GetDashboard(ctx, &awscw.GetDashboardInput{DashboardName: aws.String("missing")})
assertDashboardNotFound(t, "GetDashboard", err)

if _, err := client.PutDashboard(ctx, &awscw.PutDashboardInput{
DashboardName: aws.String("keep"), DashboardBody: aws.String(`{}`),
}); err != nil {
t.Fatalf("PutDashboard: %v", err)
}

_, err = client.DeleteDashboards(ctx, &awscw.DeleteDashboardsInput{DashboardNames: []string{"keep", "missing"}})
assertDashboardNotFound(t, "DeleteDashboards", err)

if _, err := client.GetDashboard(ctx, &awscw.GetDashboardInput{DashboardName: aws.String("keep")}); err != nil {
t.Fatalf("GetDashboard keep after failed delete: %v", err)
}
}
33 changes: 33 additions & 0 deletions server/aws/cloudwatch/handler.go
Original file line number Diff line number Diff line change
Expand Up @@ -255,6 +255,33 @@ func writeCBORError(w http.ResponseWriter, status int, errType, msg string) {
return
}

writeCBORErrorBody(w, status, errType, msg)
}

// writeShapedError writes an error whose model shape name differs from its
// awsQuery code, such as DashboardNotFoundError (code ResourceNotFound). The
// shape goes in __type and the code in X-Amzn-Query-Error, which is how an
// awsQueryCompatible service answers in both JSON and rpc-v2-cbor.
func writeShapedError(w http.ResponseWriter, status int, shape, code, msg string) {
if jw, ok := w.(*jsonWriter); ok {
jw.writeShapedError(status, shape, code, msg)
return
}

w.Header().Set(queryErrorHeader, code+";"+faultOf(status))

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This header and the shape mapping now only happen for the dashboard error. The generic writeCBORError above still puts the query code (InvalidParameterValue, MissingParameter, ...) in __type with no X-Amzn-Query-Error, so over rpc-v2-cbor errors.As(err, *types.InvalidParameterValueException) never matches (the SDK switches on the ...Exception shape names). Reusing jsonErrorShapes and this header in writeCBORError would fix that whole class. Fine as a follow-up.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed on the root cause. I tried pointing writeCBORError at jsonErrorShapes plus the header. It changes the raw CBOR __type that the existing parity tests assert for about a dozen unrelated operations (PutMetricData, PutMetricAlarm, metric streams, dashboards). It also needs care for ValidationError, which has no shape in the model (the table maps it to ValidationException, so the SDK would still fall back to a generic error). I left it for a dedicated follow-up PR that covers every operation, rather than widening this one.

writeCBORErrorBody(w, status, shape, msg)
}

// faultOf is the awsQuery fault of an error status.
func faultOf(status int) string {
if status >= http.StatusInternalServerError {
return "Receiver"
}

return "Sender"
}

func writeCBORErrorBody(w http.ResponseWriter, status int, errType, msg string) {
payload := map[string]any{
"__type": errType,
"message": msg,
Expand Down Expand Up @@ -305,7 +332,13 @@ func writeCBORResponse(w http.ResponseWriter, payload any) {
// writeDriverErr maps CloudEmu errors to CloudWatch error responses.
func writeDriverErr(w http.ResponseWriter, err error) {
if we, ok := asWireError(err); ok {
if we.shape != "" {
writeShapedError(w, we.status, we.shape, we.code, we.msg)
return
}

writeCBORError(w, we.status, we.code, we.msg)

return
}

Expand Down
10 changes: 5 additions & 5 deletions server/aws/cloudwatch/json_protocol.go
Original file line number Diff line number Diff line change
Expand Up @@ -158,16 +158,16 @@ func (j *jsonWriter) writeError(status int, code, msg string) {
shape = s
}

fault := "Sender"
if status >= http.StatusInternalServerError {
fault = "Receiver"
}
j.writeShapedError(status, shape, code, msg)
}

// writeShapedError writes an awsJson1_0 error with an explicit shape name.
func (j *jsonWriter) writeShapedError(status int, shape, code, msg string) {
body, _ := json.Marshal(map[string]string{"__type": shape, "message": msg})

hdr := j.w.Header()
hdr.Set(errTypeHeader, shape)
hdr.Set(queryErrorHeader, code+";"+fault)
hdr.Set(queryErrorHeader, code+";"+faultOf(status))
j.WriteHeader(status)
_, _ = j.Write(body)
}
Expand Down
2 changes: 1 addition & 1 deletion server/aws/cloudwatch/json_protocol_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -357,7 +357,7 @@ func TestJSONProtocolErrors(t *testing.T) {
}{
{
name: "dashboard not found", op: "GetDashboard", body: `{"DashboardName":"nope"}`,
status: http.StatusNotFound, errType: "ResourceNotFound", queryError: "ResourceNotFound;Sender",
status: http.StatusNotFound, errType: "DashboardNotFoundError", queryError: "ResourceNotFound;Sender",
},
{
name: "invalid parameter value", op: "SetAlarmState", body: `{"AlarmName":"x","StateValue":"BOGUS","StateReason":"r"}`,
Expand Down
Loading
Loading