Skip to content

fix(resolved): the fallback resolver is the bug, not the safety net - #17

Merged
ralyodio merged 1 commit into
mainfrom
fix/resolved-no-fallback
Aug 3, 2026
Merged

fix(resolved): the fallback resolver is the bug, not the safety net#17
ralyodio merged 1 commit into
mainfrom
fix/resolved-no-fallback

Conversation

@ralyodio

@ralyodio ralyodio commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

00-moshpit.conf reasoned carefully about drop-in ordering and reached the wrong conclusion. Its comment says:

Named 00- so it is read before DigitalOcean.conf: systemd-resolved appends DNS= across drop-ins in filename order, and it uses the first server for every query, rotating only on failure. So the bridge answers, and DO's resolvers stay listed as a fallback for if it ever stops.

The half that is missing: it rotates away from a failing server and does not rotate back.

What that costs

One restart of moshpit-dns moves every query to the fallback, permanently. The fallback answers NXDOMAIN for every Moshpit name, so:

dig @127.0.0.1 -p 5354 seo.rank A +short   →  67.205.189.229     works
curl seo.rank                              →  Could not resolve host
resolvectl status                          →  DNS Servers: 127.0.0.1:5354 67.207.67.3 67.207.67.2
                                              Current DNS Server: 67.207.67.3

DNS looks healthy. google.com resolves. Only the namespace this box exists to serve is dead, and nothing in systemctl status or resolvectl reads as an error. That is the worst shape a failure can take, and it happened three times in one afternoon before the cause was understood — each time diagnosed as something else.

The fallback buys nothing

The bridge already forwards what it cannot answer to the upstreams itself (mode=clearnet — the ordinary internet owns any name it can answer, the registry is only a backfill). So a second entry adds no capability, only that failure mode.

And if the bridge does stop, the answer is to restart it — which moshpit-dns.service already does in two seconds (Restart=always, RestartSec=2). Two seconds of no DNS beats an afternoon of DNS that lies.

The change

Comment only — the [Resolve] block is unchanged. It now states the contract this file depends on but never said out loud: it must be the only drop-in setting DNS=, with the commands to remove the others and the one-line check that confirms it:

rm /etc/systemd/resolved.conf.d/DigitalOcean.conf
systemctl restart systemd-resolved
resolvectl status | grep -A1 'DNS Servers'   # only 127.0.0.1:5354

A file whose correctness depends on nothing else existing should say so, because the failure when something else does exist is invisible.

Worth considering separately

install.sh does not manage resolved.conf.d at all, so this drop-in is placed by hand or by moshcode dns enable — and the latter wrote a third file (moshpit.conf) setting the same key plus Domains=~., which this file deliberately omits. Two files disagreeing about one setting is its own hazard. Making the installer own the directory would close it, but that is a bigger change than this one.

00-moshpit.conf assumed systemd-resolved "uses the first server for every
query, rotating only on failure", so listing DigitalOcean's resolvers
after the bridge was harmless insurance.

It rotates away from a failing server and does not rotate back. One
restart of moshpit-dns moves every query to the fallback permanently, and
the fallback answers NXDOMAIN for every Moshpit name — so DNS looks
healthy while the whole namespace is dead. That is the worst shape a
failure can take: nothing is obviously broken to look at.

Seen three times in one afternoon before it was understood. `dig
@127.0.0.1 -p 5354 seo.rank` answered, `curl seo.rank` did not, and
resolvectl showed the bridge listed and 67.207.67.3 current.

The fallback also buys nothing. The bridge forwards what it cannot answer
upstream itself (mode=clearnet), so a second entry adds no capability —
only that failure. If the bridge stops, restart it: the unit already does,
in two seconds. Two seconds of no DNS beats an afternoon of DNS that lies.

Documents that this must be the only drop-in setting DNS=, and how to
remove the others.
@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown

vu1nz Security Review

0 finding(s) in PR #?

No security issues found.

@ralyodio
ralyodio merged commit 6154c52 into main Aug 3, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant