Notifiers only return a single error rather than all error.

[SoloJacobs]

What did you do?

Create a configuration with multiple errors in the same receiver.

What did you expect to see?

All errors.

What did you see instead? Under which circumstances?

❯ ./amtool check-config config/testdata/conf.multi-error.yml
Checking 'config/testdata/conf.multi-error.yml' FAILED: one of url or url_file must be configured

System information

No response

Alertmanager version

Alertmanager configuration file

receivers:
  - name: 'webhook-critical'
    webhook_configs:
      - url: 'http://example.com/api'
        # ERROR 1: send_resolved is a boolean, but a string is provided.
        # The parser hits this and returns an error immediately.
        send_resolved: "yes_please" 

      - url: '' 
        # ERROR 2: The URL is empty (required field). 
        # This error is currently hidden because the process stopped above.
        http_config:
          # ERROR 3: invalid basic_auth (password/password_file are missing).
          basic_auth:
            username: 'admin'

route:
  group_by: ['alertname']
  receiver: 'webhook-critical'

Prometheus version

Prometheus configuration file

Logs

[Bisman-Singh]

Per the claim-first process, I'd like to take this together with #4991 — they read as one investigation into the config-validation error path (accumulate all errors instead of stopping at the first: within a receiver here, across receivers in #4991). Plan: reproduce both with amtool check-config fixtures, then errors.Join-style accumulation with table-driven unit tests. I'll keep the fixes as small separate PRs unless the code path turns out to be one and the same.

[SoloJacobs]

We don't have a claim-first process.

[Bisman-Singh]

Understood, thanks for the correction. I took that wording from prometheus/prometheus's CONTRIBUTING and wrongly assumed it carried over to this repo. My mistake.

The substance is unaffected: #5504 is open and closes both this and #4991. It moves the Validate() calls out of the notifier UnmarshalYAML methods (split out for exactly this purpose in #4992) into Config.UnmarshalYAML and accumulates with errors.Join, so config loading and amtool check-config report every receiver-level and notifier-level error in one pass instead of aborting at the first.

Happy to split it into two PRs if you would rather review the within-receiver and across-receiver cases separately, though they turned out to be the same code path.