The troubleshooting page does not list problems related to nf_tables

[ferrybig]

Is this a docs issue?

• My issue is about the documentation content or website

Type of issue

Information is incorrect

Description

Arch linux moves to NFT tables instead of IP tables: https://archlinux.org/news/iptables-now-defaults-to-the-nft-backend/

This creates a compatibility problem not documented on the troubleshooting page under the arch linux section

Location

https://docs.docker.com/engine/security/rootless/troubleshoot/

Suggestion

No response

[dvdksn]

Thanks for the report. We already call out the iptables-nft compatibility requirement on eg the ubuntu install page, but it's worth adding a generic troubleshooting entry for this I reckon.

[ferrybig]

The install page is not where I expect to find this information.

Docker was running fine before, and today it was no longer working. Going to a troubleshooting page seems more logical to me that going through the install page again

EDIT:

Especially not when the rootless command reports everything is fine:

$ dockerd-rootless-setuptool.sh check
[INFO] Requirements are satisfied
$ dockerd-rootless-setuptool.sh install
[WARNING] File already exists, skipping: /home/fernando/.config/systemd/user/docker.service
[INFO] starting systemd service docker.service
+ systemctl --user start docker.service
Job for docker.service failed because the control process exited with error code.
See "systemctl --user status docker.service" and "journalctl --user -xeu docker.service" for details.
+ set +x
[ERROR] Failed to start docker.service. Run `journalctl -n 20 --no-pager --user --unit docker.service` to show the error log.
[ERROR] Before retrying installation, you might need to uninstall the current setup: `/home/fernando/bin/dockerd-rootless-setuptool.sh uninstall -f ; /home/fernando/bin/rootlesskit rm -rf /home/fernando/.local/share/docker`

[dvdksn]

@thaJeztah do we check this in any of the installation scripts?

[thaJeztah]

Curious; could this be related to rootlesskit v3? What version of rootlesskit is installed (i.e., is it possible Arch already started shipping v3 ?);

• Update RootlessKit (3.0.0) moby/moby#52319

cc @AkihiroSuda @robmry

[AkihiroSuda]

Curious; could this be related to rootlesskit v3?

No, it is still not released

[AkihiroSuda]

What error happens if you run dockerd-rootless.sh manually?

Did you try rebooting?

[robmry]

Docker was running fine before, and today it was no longer working. Going to a troubleshooting page seems more logical to me that going through the install page again

What changed - was it an Arch Linux upgrade?

Did the log ("journalctl --user -xeu docker.service") say something about iptables?

[ferrybig]

I just solved the issue. (and the following is the writeup, sine I am not going to do repeat of https://xkcd.com/979/)

Adding some more information: (with rough functional time stamps)

At 9:00 I started my computer form cold boot

At 10:00 I did a full system update, because I wanted to install a package. My OS replaced iptables with nftables because of https://archlinux.org/news/iptables-now-defaults-to-the-nft-backend/ (I then forgot about the full update, since in my head I just wanted to install a single package)

At 11:00 I tried to run a docker container with forwarded ports, it gave the error:

$ docker run -p 8090:80 test
docker: Error response from daemon: driver failed programming external connectivity on endpoint blissful_bassi (0cd65a36651f81900b48893ad1d155ccf06106fcb83f5a4db0ff41716d78591e):  (iptables failed: iptables --wait -t nat -A DOCKER -p tcp -d 0/0 --dport 8090 -j DNAT --to-destination 172.17.0.2:80 ! -i docker0: Warning: Extension tcp revision 0 not supported, missing kernel module?
Warning: Extension DNAT revision 0 not supported, missing kernel module?
iptables v1.8.11 (nf_tables): Could not fetch rule set generation id: Invalid argument
 (exit status 4)).
ERRO[0000] error waiting for container: context canceled 

Because docker ps reported other containers with working forwards, I initially did not suspect a problem

At 11:30 i tried to find my error in the throubleshooting page of docker rootless, I could not find any

At 12:00 I gave up and asked AI. It suggested that nftables could be to blame and gave me an instruction to add {"firewall-backend": "nftables"} to .config/docker/daemon.json. This did not work

At 12:10 the answer of the AI became to add { "experimental": true, "firewall-backend": "nftables"} to .config/docker/daemon.json

At 12.20 I checked the package versions. According to my distro, I had docker version 1:29.3.1-1, which should support the above flag (1)

At 13.00 I suspected my rootless docker setup broke, I tried to reinstall using the steps. The first command listed on the page is dockerd-rootless-setuptool.sh install (1), so I ran it. It had no problems with installing, but then gave an error telling me to look in the journalctl as i failed to start. I ran the check output, it said no issues. The errors were still confusing.

At 14.00 I tried to run docker version, it reported multiple version numbers, like a "version 25, api version ... " (1). I found it weird that no numbers of docker version matched the version of docker I had installed.

At 15.00 I ran type -a docker, it told me I had 3 packages of docker installed, one in /home/fernando/bin/docker, one in /bin/docker and one in /usr/bin/docker. Here i realised I had some clone of docker in the bin directory of my profile. Which were hopelessly outdated. This is not what expected when I followed the install steps of docker rootless back in 01 Feb 2024!. I threw out the files and tried to follow the install steps

Trying to run the installer was not without issues. The documentation of docker says docker 29.xx can run in experimental mode with nftables. The installer still checks for iptables, saying you need to use --skip-iptables to bypass it.

Even with iptables bypassed, docker still had problems installing, changing the daemon config changed the error, but neither mode worked

At 15.30 I used some more AI chat services. One told me that the error I got likely were because I did not restart after installing nftables. So I restarted

At 15.40 The computer restarted, and I was able to use the rootless install script with the --skip-iptables flag.

At 15.50 I fully tested existing containers running on my computer, to see if they worked with nftables. No problems found

At 17.00 Writeup finished

---

Relevant log output: (for people using search engines to find this)

The initial error, after I tried to restart the docker service when I was unable to make port forwardings:

$ journalctl -n 20 --no-pager --user --unit docker.service
apr 07 15:32:59 Fernando-LP03 dockerd-rootless.sh[478047]: time="2026-04-07T15:32:59.941243397+02:00" level=info msg="NRI interface is disabled by configuration."
apr 07 15:32:59 Fernando-LP03 dockerd-rootless.sh[478047]: time="2026-04-07T15:32:59.941415251+02:00" level=info msg=serving... address=/run/user/1000/docker/containerd/containerd-debug.sock
apr 07 15:32:59 Fernando-LP03 dockerd-rootless.sh[478047]: time="2026-04-07T15:32:59.941484437+02:00" level=info msg=serving... address=/run/user/1000/docker/containerd/containerd.sock.ttrpc
apr 07 15:32:59 Fernando-LP03 dockerd-rootless.sh[478047]: time="2026-04-07T15:32:59.941544175+02:00" level=info msg=serving... address=/run/user/1000/docker/containerd/containerd.sock
apr 07 15:32:59 Fernando-LP03 dockerd-rootless.sh[478047]: time="2026-04-07T15:32:59.941565605+02:00" level=info msg="containerd successfully booted in 0.021212s"
apr 07 15:33:00 Fernando-LP03 dockerd-rootless.sh[478018]: time="2026-04-07T15:33:00.974290202+02:00" level=info msg="[graphdriver] using prior storage driver: overlay2"
apr 07 15:33:01 Fernando-LP03 dockerd-rootless.sh[478018]: time="2026-04-07T15:33:01.195169360+02:00" level=info msg="Loading containers: start."
apr 07 15:33:01 Fernando-LP03 dockerd-rootless.sh[478018]: time="2026-04-07T15:33:01.217909800+02:00" level=warning msg="Running modprobe bridge br_netfilter failed with message: modprobe: WARNING: Module bridge not found in directory /lib/modules/6.19.10-arch1-1\nmodprobe: WARNING: Module br_netfilter not found in directory /lib/modules/6.19.10-arch1-1\n, error: exit status 1"
apr 07 15:33:01 Fernando-LP03 dockerd-rootless.sh[478018]: time="2026-04-07T15:33:01.217948320+02:00" level=info msg="skipping firewalld management for rootless mode"
apr 07 15:33:01 Fernando-LP03 dockerd-rootless.sh[478018]: time="2026-04-07T15:33:01.220854966+02:00" level=info msg="unable to detect if iptables supports xlock: 'iptables --wait -L -n': `iptables v1.8.11 (nf_tables): Could not fetch rule set generation id: Invalid argument`" error="exit status 4"
apr 07 15:33:01 Fernando-LP03 dockerd-rootless.sh[478018]: time="2026-04-07T15:33:01.291481298+02:00" level=info msg="stopping event stream following graceful shutdown" error="<nil>" module=libcontainerd namespace=moby
apr 07 15:33:01 Fernando-LP03 dockerd-rootless.sh[478018]: time="2026-04-07T15:33:01.291880646+02:00" level=info msg="stopping healthcheck following graceful shutdown" module=libcontainerd
apr 07 15:33:01 Fernando-LP03 dockerd-rootless.sh[478018]: time="2026-04-07T15:33:01.291900241+02:00" level=info msg="stopping event stream following graceful shutdown" error="context canceled" module=libcontainerd namespace=plugins.moby
apr 07 15:33:02 Fernando-LP03 dockerd-rootless.sh[478018]: failed to start daemon: Error initializing network controller: error obtaining controller instance: failed to register "bridge" driver: failed to create NAT chain DOCKER: iptables failed: iptables -t nat -N DOCKER: iptables v1.8.11 (nf_tables): Could not fetch rule set generation id: Invalid argument
apr 07 15:33:02 Fernando-LP03 dockerd-rootless.sh[478018]:  (exit status 4)
apr 07 15:33:02 Fernando-LP03 dockerd-rootless.sh[477987]: [rootlesskit:child ] error: command [/home/fernando/bin/dockerd-rootless.sh] exited: exit status 1
apr 07 15:33:02 Fernando-LP03 dockerd-rootless.sh[477980]: [rootlesskit:parent] error: child exited: exit status 1
apr 07 15:33:02 Fernando-LP03 systemd[2732]: docker.service: Main process exited, code=exited, status=1/FAILURE
apr 07 15:33:02 Fernando-LP03 systemd[2732]: docker.service: Failed with result 'exit-code'.
apr 07 15:33:02 Fernando-LP03 systemd[2732]: Failed to start Docker Application Container Engine (Rootless).

After I updated docker, but my system did not start nftables yet:

$ journalctl -n 20 --no-pager --user --unit docker.service
apr 07 16:09:05 Fernando-LP03 dockerd-rootless.sh[519079]: time="2026-04-07T16:09:05.674835447+02:00" level=error msg="nftables: failed to update nftables:\n1:\t\n2:\ttable ip docker-bridges {\n3:\t\tmap filter-forward-in-jumps {\n4:\t\t\ttype ifname : verdict\n5:\t\t\t\n6:\t\t}\n7:\t\tmap filter-forward-out-jumps {\n8:\t\t\ttype ifname : verdict\n9:\t\t\t\n10:\t\t}\n11:\t\tmap nat-postrouting-in-jumps {\n12:\t\t\ttype ifname : verdict\n13:\t\t\t\n14:\t\t}\n15:\t\tmap nat-postrouting-out-jumps {\n16:\t\t\ttype ifname : verdict\n17:\t\t\t\n18:\t\t}\n19:\t\t\n20:\t\t\n21:\t\tchain filter-FORWARD {\n22:\t\t\ttype filter hook forward priority 0; policy accept\n23:\t\t} ; chain nat-OUTPUT {\n24:\t\t\ttype nat hook output priority -100; policy accept\n25:\t\t} ; chain nat-POSTROUTING {\n26:\t\t\ttype nat hook postrouting priority 100; policy accept\n27:\t\t} ; chain nat-PREROUTING {\n28:\t\t\ttype nat hook prerouting priority -100; policy accept\n29:\t\t} ; chain nat-prerouting-and-output {\n30:\t\t\t\n31:\t\t} ; chain raw-PREROUTING {\n32:\t\t\ttype filter hook prerouting priority -300; policy accept\n33:\t\t} ; \n34:\t}\n35:\tflush table ip docker-bridges\n36:\tflush map ip docker-bridges filter-forward-in-jumps\n37:\tflush map ip docker-bridges filter-forward-out-jumps\n38:\tflush map ip docker-bridges nat-postrouting-in-jumps\n39:\tflush map ip docker-bridges nat-postrouting-out-jumps\n40:\t\n41:\t\n42:\tflush chain ip docker-bridges filter-FORWARD\n43:\tflush chain ip docker-bridges nat-OUTPUT\n44:\tflush chain ip docker-bridges nat-POSTROUTING\n45:\tflush chain ip docker-bridges nat-PREROUTING\n46:\tflush chain ip docker-bridges nat-prerouting-and-output\n47:\tflush chain ip docker-bridges raw-PREROUTING\n48:\t\n49:\t\n50:\t\n51:\t\n52:\ttable ip docker-bridges {\n53:\t\tchain filter-FORWARD {\n54:\t\t\ttype filter hook forward priority 0; policy accept\n55:\t\t\toifname vmap @ filter-forward-in-jumps\n56:\t\t\tiifname vmap @ filter-forward-out-jumps\n57:\t\t\t\n58:\t\t}\n59:\t\tchain nat-OUTPUT {\n60:\t\t\ttype nat hook output priority -100; policy accept\n61:\t\t\tip daddr != 127.0.0.1/8  fib daddr type local counter jump nat-prerouting-and-output\n62:\t\t\t\n63:\t\t}\n64:\t\tchain nat-POSTROUTING {\n65:\t\t\ttype nat hook postrouting priority 100; policy accept\n66:\t\t\tiifname vmap @ nat-postrouting-out-jumps\n67:\t\t\toifname vmap @ nat-postrouting-in-jumps\n68:\t\t\t\n69:\t\t}\n70:\t\tchain nat-PREROUTING {\n71:\t\t\ttype nat hook prerouting priority -100; policy accept\n72:\t\t\tfib daddr type local counter jump nat-prerouting-and-output\n73:\t\t\t\n74:\t\t}\n75:\t\tchain nat-prerouting-and-output {\n76:\t\t\t\n77:\t\t\t\n78:\t\t}\n79:\t\tchain raw-PREROUTING {\n80:\t\t\ttype filter hook prerouting priority -300; policy accept\n81:\t\t\t\n82:\t\t}\n83:\t\t\n84:\t}\n85:\t\n86:\t\n87:\t\nrunning nft: netlink: Error: cache initialization failed: Invalid argument\n exit status 1"
apr 07 16:09:05 Fernando-LP03 dockerd-rootless.sh[519079]: time="2026-04-07T16:09:05.674899627+02:00" level=warning msg="nftables: reloading table" family=ip table=docker-bridges
apr 07 16:09:05 Fernando-LP03 dockerd-rootless.sh[519079]: time="2026-04-07T16:09:05.681086389+02:00" level=error msg="nftables: failed to reload nftable:\n1:\t\n2:\ttable ip docker-bridges {}\n3:\tdelete table ip docker-bridges\n4:\ttable ip docker-bridges {\n5:\t\tmap filter-forward-in-jumps {\n6:\t\t\ttype ifname : verdict\n7:\t\t\t\n8:\t        \n9:\t\t}\n10:\t\tmap filter-forward-out-jumps {\n11:\t\t\ttype ifname : verdict\n12:\t\t\t\n13:\t        \n14:\t\t}\n15:\t\tmap nat-postrouting-in-jumps {\n16:\t\t\ttype ifname : verdict\n17:\t\t\t\n18:\t        \n19:\t\t}\n20:\t\tmap nat-postrouting-out-jumps {\n21:\t\t\ttype ifname : verdict\n22:\t\t\t\n23:\t        \n24:\t\t}\n25:\t\t\n26:\t\t\n27:\t\tchain filter-FORWARD {\n28:\t\t\ttype filter hook forward priority 0; policy accept\n29:\t\t\toifname vmap @ filter-forward-in-jumps\n30:\t\t\tiifname vmap @ filter-forward-out-jumps\n31:\t\t\t\n32:\t\t}\n33:\t\tchain nat-OUTPUT {\n34:\t\t\ttype nat hook output priority -100; policy accept\n35:\t\t\tip daddr != 127.0.0.1/8  fib daddr type local counter jump nat-prerouting-and-output\n36:\t\t\t\n37:\t\t}\n38:\t\tchain nat-POSTROUTING {\n39:\t\t\ttype nat hook postrouting priority 100; policy accept\n40:\t\t\tiifname vmap @ nat-postrouting-out-jumps\n41:\t\t\toifname vmap @ nat-postrouting-in-jumps\n42:\t\t\t\n43:\t\t}\n44:\t\tchain nat-PREROUTING {\n45:\t\t\ttype nat hook prerouting priority -100; policy accept\n46:\t\t\tfib daddr type local counter jump nat-prerouting-and-output\n47:\t\t\t\n48:\t\t}\n49:\t\tchain nat-prerouting-and-output {\n50:\t\t\t\n51:\t\t\t\n52:\t\t}\n53:\t\tchain raw-PREROUTING {\n54:\t\t\ttype filter hook prerouting priority -300; policy accept\n55:\t\t\t\n56:\t\t}\n57:\t\t\n58:\t}\n59:\t\nrunning nft: netlink: Error: cache initialization failed: Invalid argument\n exit status 1" family=ip table=docker-bridges
apr 07 16:09:05 Fernando-LP03 dockerd-rootless.sh[519079]: time="2026-04-07T16:09:05.683323439+02:00" level=info msg="stopping event stream following graceful shutdown" error="<nil>" module=libcontainerd namespace=moby
apr 07 16:09:05 Fernando-LP03 dockerd-rootless.sh[519079]: time="2026-04-07T16:09:05.683559329+02:00" level=info msg="stopping healthcheck following graceful shutdown" module=libcontainerd
apr 07 16:09:05 Fernando-LP03 dockerd-rootless.sh[519079]: time="2026-04-07T16:09:05.683569167+02:00" level=info msg="stopping event stream following graceful shutdown" error="context canceled" module=libcontainerd namespace=plugins.moby
apr 07 16:09:06 Fernando-LP03 dockerd-rootless.sh[519079]: time="2026-04-07T16:09:06.684429451+02:00" level=info msg="Daemon shutdown complete" error="failed to start daemon: Error initializing network controller: error obtaining controller instance: failed to register \"bridge\" driver: running nft: netlink: Error: cache initialization failed: Invalid argument\n exit status 1"
apr 07 16:09:06 Fernando-LP03 dockerd-rootless.sh[519079]: failed to start daemon: Error initializing network controller: error obtaining controller instance: failed to register "bridge" driver: running nft: netlink: Error: cache initialization failed: Invalid argument
apr 07 16:09:06 Fernando-LP03 dockerd-rootless.sh[519079]:  exit status 1
apr 07 16:09:06 Fernando-LP03 dockerd-rootless.sh[519048]: [rootlesskit:child ] error: command [/home/fernando/bin/dockerd-rootless.sh] exited: exit status 1

---

What I expect of the documentation:

• A section that talks that if you run docker rootless, it makes its own installation, it doesn't convert the system version into a rootless docker version. It should warn that the user needs to keep it updated.
• A section for port forwarding not working, asking the user to run $ iptables --version. if it return something ending in (nf_tables), the user should either enable experimental support for nftables, or go back to legacy iptables

Other remarks:

• The fact that docker rootless comes with dockerd-rootless-setuptool.sh is a major trap, one thing to improve it for future installs is including an echo of the version of this script as the first line output in future installs, with a prompt to an end user to check the official website for a newer version. (distro specific configuration should remove this prompt towards the docker website with a command for their distro's package manager)
• The errors reported by docker are pretty vague and hard to parse, this makes it difficulty to find information on them

[robmry]

At 10:00 I did a full system update, because I wanted to install a package. My OS replaced iptables with nftables because of https://archlinux.org/news/iptables-now-defaults-to-the-nft-backend/ (I then forgot about the full update, since in my head I just wanted to install a single package)

Thanks @ferrybig ... I'm glad to hear switching to nftables sorted it.

I'm not sure I understand how it broke though - that link says Arch Linux changed the package names. So package "iptables" now installs "iptables-nft" instead of "iptables-legacy". But that should work fine ... maybe the "iptables" package got uninstalled, or needed to be explicitly installed afterwards?