[Bug]: "Retry Attempts" (max_retries) is ignored by the service checker, one failed probe marks a service down

[Flegma]

Environment

CheckCle self-hosted (Docker), reproduced on current develop (e2b20fe). Affects every version since the Go service-operation engine was introduced (June 2025).

Bug Description

The per-service Retry Attempts setting (stored as max_retries in the services collection) is never used by the checking engine. performCheck runs exactly one attempt and marks the service down on the first failure, so a single slow or dropped probe immediately produces a DOWN notification, followed by UP on the next cycle.

Root cause, with code references on develop:

• The checker performs a single operation per cycle and derives status from that one result, with a hardcoded per-attempt timeout:

  checkcle/server/service-operation/monitoring/checker.go

  Line 28 in e2b20fe

  timeout := 10 * time.Second // Default timeout

• The Go Service struct already deserializes the field, but no code under server/ reads it:

  checkcle/server/service-operation/pocketbase/types.go

  Line 150 in e2b20fe

  MaxRetries int `json:"max_retries"`

• The frontend stores it on create/update:

  checkcle/application/src/services/serviceService.ts

  Line 70 in e2b20fe

  max_retries: params.retries,

• The legacy frontend checker honored it (service.max_retries || 3), so this is a regression from the migration to the Go microservice:

  checkcle/application/src/services/monitoring/httpChecker.ts

  Line 40 in e2b20fe

  const maxRetries = service.max_retries || 3;

Steps to Reproduce

1. Create an HTTP service with Check Interval 5 minutes and Retry Attempts 5, notifications on.
2. Make the target respond slower than 10s once (or drop one request).
3. A DOWN notification fires immediately after that single failed probe; UP follows on the next 5-minute check.

Real-world evidence

On our production instance (interval 300s, Retry Attempts 5) the origin nginx access log shows exactly one probe request per incident: the probe arrives at :29s, nginx logs HTTP 499 at :39s (client gave up after the 10s timeout), CheckCle records DOWN about 20-30s later, and no further requests arrive until the next 5-minute cycle. With retries actually configured at 5, none of these alerts should have fired. The result is a recurring false "4.5 minute downtime" alert whenever one probe lands on a brief load spike (nightly backups in our case).

Expected Behavior

The service should only be marked down (and notifications sent) after max_retries consecutive failed attempts within the check cycle, as the field description in the UI says: "Number of retry attempts before marking as down".

Related

• [Bug]: Notifications Trigger Too Early Despite Retry Configuration #180 reported the same symptom and was closed as a duplicate of [Feature]: Add configurable notification delay for downtime alerts[Feature]: #176.
• [Feature]: Add configurable notification delay for downtime alerts[Feature]: #176 asks for a time-based notification delay; honoring max_retries is complementary and fixes the existing, already-configurable field.

I have a PR ready that implements this in performCheck (retry loop with the existing per-attempt timeout, defaults preserved, plus the module's first unit tests). Will link it here.

Checklist

• I have searched existing issues to avoid creating duplicates.
• I have provided sufficient information to reproduce the issue.