Repository navigation
bdns: add metric for doh_transport_errors. - #9103
Merged
Merged
Conversation
This will let us add clear alerting for when Unbound is having problems, as opposed to our subscribers' authoritative servers having problems. The new counter, `doh_transport_errors`, counts things like connection refused, connection timeout, TLS errors, and so on. It does not count cases where we successfully received a DNS response but the RCODE was NXDOMAIN, REFUSED, SERVFAIL, etc. It also does not count cases where the HTTP _response_ timed out, because those are typically "authoritative server was slow" cases, and not our fault. Replace http.Client.Timeout (a whole-request timeout) with these three: - Dial timeout, hardcoded at 2s - TLS handshake timeout, hardcoded at 2s - ResponseHeaderTimeout, set to the configured dnsTimeout. Add sentinel errors in `ExchangeContext` to indicate when an HTTP request timed out or was canceled. These replace the typecasting net.OpError / url.Error cases, and make them more precise: we only want to tell the user "query timed out" when it was their fault. Change the retry logic significantly: previously we would retry only on net.Error where Timeout() was true. Now we retry on any error so long as the context is not done. Reasoning: the old code retried both on user-caused errors and _some_ internal errors. Rearranging error returns made it necessary to tweak this and I decided to go with "retry everything up to maxTries." Practical impact: a DNS request may now take at most 2s + 2s + maxTries * requestTimeout. Previously it would take at most maxTries * requestTimeout. In practice, dials are usually fast and we should not see much impact.
ezekiel
approved these changes
Oct 19, 2026
ezekiel
requested review from
a team and
beautifulentropy
and removed request for
a team and
beautifulentropy
October 19, 2026 16:30
aarongable
approved these changes
Oct 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This will let us add clear alerting for when Unbound is having problems, as opposed to our subscribers' authoritative servers having problems.
The new counter,
doh_transport_errors, counts things like connection refused, connection timeout, TLS errors, and so on. It does not count cases where we successfully received a DNS response but the RCODE was NXDOMAIN, REFUSED, SERVFAIL, etc. It also does not count cases where the HTTP response timed out, because those are typically "authoritative server was slow" cases, and not our fault.Replace http.Client.Timeout (a whole-request timeout) with these three:
Add sentinel errors in
ExchangeContextto indicate when an HTTP request timed out or was canceled. These replace the typecasting net.OpError / url.Error cases, and make them more precise: we only want to tell the user "query timed out" when it was their fault.Change the retry logic significantly: previously we would retry only on net.Error where Timeout() was true. Now we retry on any error so long as the context is not done. Reasoning: the old code retried both on user-caused errors and some internal errors. Rearranging error returns made it necessary to tweak this and I decided to go with "retry everything up to maxTries."
Practical impact: a DNS request may now take at most
2s + 2s + maxTries * requestTimeout. Previously it would take at mostmaxTries * requestTimeout. In practice, dials are usually fast and we should not see much impact.An LLM wrote the tests, which I reviewed, tweaked, and rewrote the comments for.
Fixes #9018