[scotwells]

Requested by Scot Schuchert-Wells

Addresses datum-cloud/compute#630. Related: datum-cloud/enhancements#921.

Problem

A freshly started Kata guest cannot resolve names for its first few seconds. A boot-time trace inside a test guest showed this:

• Its address and its on-link default route were in place from the first sample.
• Its DNS lookups timed out until about 5 s after its first packet.
• Lookups started working right after the guest's neighbor entry for the host was reconfirmed (DELAY to REACHABLE).

The host had no neighbor entry for the guest. The guest had to resolve its gateway itself. So the first packets in each direction depended on neighbor discovery completing.

Change

The DAN file already fixes the guest's MAC (derived from the tap name), and galactic-tap knows the tap's host MAC. So neither side needs to discover the other:

• Guest side: galactic-tap writes a permanent neighbor entry for each gateway (IPv6 and IPv4) into the DAN file, pointing at the tap's host MAC. The Kata runtime-rs shim installs these in the guest at boot.
• Host side: galactic-bgp reads the guest MAC back from the DAN file. RestoreHostGateway then installs a permanent neighbor entry for the guest's addresses on the host side of the tap, as it already does for a veth.
• Unchanged: taps without a DAN file (guests that pick their own MAC) keep the existing neighbor sweep.

The DAN neighbors field is now typed. It follows Kata's schema: ip_address as CIDR, hardware_addr, state, flags. state is NUD_PERMANENT. The CNI conflist reference describes the behaviour.

Testing

• New unit tests cover:
  • gateway neighbors in the DAN document
  • reading the guest MAC back from a DAN file
  • galactic-bgp deriving the tap guest's MAC
• go build ./..., go vet, golangci-lint (0 issues) and go test pass in a Linux build container for internal/cnitap, internal/plumbing/dan and the new internal/cnibgp test.
• The kernel-backed internal/cnibgp and internal/hostgw tests need VRF support. They fail in that container on main too, with the same 13 failures.
• Not yet run in a lab. The check after rollout is the same boot trace: a fresh guest's first lookup should answer, and the guest's gateway entry should show as PERMANENT from boot.

[ecv]

Approve.

Pinned entries are per-link, so they die with the tap and cannot leak to a later tenant; the guest MAC is host-derived, not guest-supplied.

• Isolation: the host-side permanent entry is installed on the tap link in the host netns via the existing installGatewayNeighbor; the guest-side entries live in the guest's own stack from the DAN file. A reused IP lands on a new tap with its own neighbor table, so nothing stale carries over. DEL removes the DAN file (ops_del.go) and the tap, which drops its entries.
• MAC trust: the guest MAC comes from guestMAC(tapName), a sha256 of the host-generated tap name, written by cnitap and read back by cnibgp. It is not guest-supplied, and it is stable across reboots, so the old comment's "stale permanent entry after a guest reboot" worry does not apply to DAN taps.
• Repair and re-up: activateTrafficPolicy passes cfg.guestHWAddr into RestoreHostGateway, and reenslave.go already preserves NUD_PERMANENT entries, so both paths keep the pin.
• Upgrade: existing DAN files carry "neighbors": [], which still unmarshals into []Neighbor. A repeated ADD on a live tap just re-sets the entry idempotently (NeighSet). Already-running guests keep their old file with no guest-side pin and still resolve through the existing EnsureTapGuestNeighbors sweep.
• No new conflist field. DANDir already existed, and cnibgp now reads it with the same env/conflist/default resolution as cnitap, so the two agree. A DAN read failure only warns and skips the pin.
• Guest-side pin uses the tap's kernel-assigned MAC at ADD. If a tap were ever recreated for a live guest with a new MAC, the guest's permanent gateway entry would go stale and never refresh; nothing here does that today.

Follow-ups:

• Confirm on a real Kata runtime-rs node that the shim honors state: 0x80 for guest neighbors (the PR does not say it was observed), and that the entry shows PERMANENT in the guest.