[ifnotcode]

Problem

Container runtimes give a container that runs in a user namespace access to host directories through
idmapped mounts (mount_setattr(MOUNT_ATTR_IDMAP)). Kubernetes does this for every volume of a pod
with hostUsers: false
(user namespaces are GA since Kubernetes 1.36).
Linux 6.12 added idmapped mount support for FUSE, but the kernel only allows it when the FUSE
server sets FUSE_ALLOW_IDMAP in its FUSE_INIT reply
(fs/fuse: allow idmapped mounts,
LWN).
dfuse does not set it, so such a pod cannot use a dfuse mount; it fails to start with

failed to set MOUNT_ATTR_IDMAP on /mnt/dfuse: invalid argument
(maybe the filesystem used doesn't support idmap mounts on this kernel?)

Change

• New option. --allow-idmap makes dfuse request FUSE_CAP_ALLOW_IDMAP in
  dfuse_fuse_init(). The kernel only accepts it together with default_permissions, which dfuse
  always uses. When it is allowed, dfuse logs that at INFO level, like the other capabilities.
  When the kernel or the fuse library does not support it, dfuse logs an error, so that it is
  seen with the default log mask of clients (ERR): the option was asked for and has no effect.
  dfuse keeps running, as without the option.
• Off by default. Enabling it changes what the kernel tells dfuse about callers. For requests
  that do not create an inode, the uid and gid in the request header become FUSE_INVALID_UIDGID.
  This applies to every mount of the filesystem, not only to idmapped ones (fuse_get_req() in
  fs/fuse/dev.c).
• ioctl handle sharing. This is the only place where dfuse relies on the caller's ids outside
  inode creation. The other uses are in create, mknod and symlink, where the kernel still sends
  them, mapped through the idmapping of the mount.
  • dfuse shares its pool and container handles only with libioil/libpil4dfs processes running as
    its own uid and gid.
  • When the header does not carry the ids, caller_matches() reads the fsuid and fsgid of the
    calling thread from /proc/<pid>/task/<pid>/status. libfuse's fuse_req_getgroups() already
    does this for supplementary groups, which dfuse uses in _dfuse_mode_update().
  • With the kernel-provided ids, the code path is unchanged.
  • Why a caller is refused is logged at DEBUG level: a /dev/fd/N mount, pid 0 (outside the pid
    namespace), a /proc of another pid namespace, an unreadable /proc entry, or another uid or
    gid.
• Build compatibility. FUSE_CAP_ALLOW_IDMAP exists only in fuse libraries that support it.
  With another library, --allow-idmap has no effect and dfuse logs an error, as it does when the
  kernel lacks support. The PR therefore does not depend on the fuse update, and its test
  also passes with the current fused v1.0.0, as a plain interception test (see Dependencies).
• Tests and docs.
  • docs/user/filesystem.md documents the option, when it is needed, what it changes and what it
    does not: neither who gets the handles nor who may access files. It also says that dfuse should
    not run as the overflow user (nobody) with the option.
  • The new NLT test test_il_allow_idmap runs libioil and libpil4dfs against dfuse started with
    --allow-idmap. It is non-trivial only on Linux 6.12 or later with a fuse library that has
    FUSE_CAP_ALLOW_IDMAP (DAOS-19898 build: Update fused to v1.1.0 with FUSE_CAP_ALLOW_IDMAP #19405): then the kernel does not send the ids, and the test fails if
    dfuse does not share its handles: dfuse logs why it refused the caller (the test fails on
    such a line in its log), and libpil4dfs, which only intercepts I/O on a dfuse mount with
    dfuse's handles, reports no intercepted calls ("op_sum is zero").
  • Otherwise dfuse's log lacks "Idmapped mounts allowed", the test prints a notice and checks
    plain interception. This is the case on kernels before 6.12 and in CI until DAOS-19898 build: Update fused to v1.1.0 with FUSE_CAP_ALLOW_IDMAP #19405 is merged
    (fused v1.0.0). The NLT log check (log_test(), ignore_no_idmap) then ignores dfuse's
    error that the kernel or the fuse library does not support idmapped mounts, only for a
    dfuse started with --allow-idmap, so it does not become a finding.
  • utils/cq/words.dict gets "idmap" and "idmapped", which the pylint spelling check does not
    know otherwise.

Security

The ioctl handler replies with handles that carry dfuse's credentials, so the caller check must not
become weaker:

• Trust model. The option is not an access-control mechanism and does not change dfuse's trust
  model: who gets the handles and who may access files is unchanged. The ids that the kernel
  used to send are read from /proc instead, and the kernel keeps checking file permissions
  (default_permissions).
• Thread id reuse. The kernel sends the thread id of the caller, which is blocked in ioctl()
  until dfuse replies, so the id cannot be reused by another task meanwhile. If the caller is killed,
  the reply is discarded by the kernel.
• Pid namespaces. The pid is relative to the pid namespace of the FUSE connection, that is of
  the process that mounted the filesystem. Normally this is dfuse itself, or fusermount3 started by
  it.
  • A caller outside that namespace or its descendants gets pid 0 and is refused.
  • If dfuse sees a /proc that belongs to another pid namespace (checked with
    readlink("/proc/self") == getpid()), the pid cannot be looked up and every caller is refused.
  • If dfuse is given a mounted /dev/fuse as /dev/fd/N, the process that mounted it may be in
    another pid namespace, e.g. a privileged helper that passes the descriptor to dfuse in a
    container. dfuse cannot check that, so it refuses every caller. Otherwise a pid of that
    namespace could be looked up in dfuse's /proc and match an unrelated process.
• Other users. Callers running as another user are still refused, including processes in user
  namespaces, which run as other uids from dfuse's point of view.
• Failure mode. Every failure to identify the caller means EPERM, which the interception
  libraries already handle: libpil4dfs leaves the I/O to dfuse and libioil opens its own handles
  with the caller's credentials.

Limitations

• Pid namespace for interception libraries. With --allow-idmap, they only get dfuse's handles
  if they run in dfuse's pid namespace or one nested in it. For example, when dfuse runs in a
  container of its own, processes in other containers don't get them. Running dfuse in the pid
  namespace of the host avoids this. With a /dev/fd/N mount point they never get them. This is
  documented.
• Overflow user. A caller whose ids are not mapped in dfuse's user namespace appears in
  /proc as the overflow user (65534, nobody). dfuse should therefore not run as that user with
  --allow-idmap. This is documented.
• Supplementary groups. _dfuse_mode_update() compares the caller's supplementary groups, read
  from /proc by fuse_req_getgroups(), with the parent directory's gid.
  • These groups are not translated through the mount's idmapping. For a process on an idmapped
    mount, the check may therefore miss a group the process is a member of inside its user
    namespace.
  • This only affects that existing heuristic, and only on idmapped mounts.
  • Using the kernel's idmapping-aware FUSE_CREATE_SUPP_GROUP instead would be a separate change.

Dependencies

A fuse library with FUSE_CAP_ALLOW_IDMAP: libfuse commit
1f4c0608,
which is not in a release yet.

The fused version DAOS uses today (v1.0.0, a libfuse snapshot before 3.17) does not have it and
cannot: the flag is bit 32 of the capabilities, which libfuse only has from 3.17 on.

• DAOS-19898 build: Update fused to v1.1.0 with FUSE_CAP_ALLOW_IDMAP #19405, sent together with this PR, updates fused to v1.1.0 (libfuse 3.17.4). It carries
  the backport in deps/patches/fused. With both PRs the option works.
• Upstream libfuse. If you prefer that route (DAOS-18222, DAOS-18222 build: Update to latest tag of fuse #17784), the same backport applies
  to libfuse 3.18.2 and 3.18.3 as a deps patch. It could also go into fused itself.

This PR does not depend on #19405. Without it the option has no effect and dfuse logs an error, so
this PR can be merged first.

Testing

DAOS master 58d0485 with this PR, built with fused v1.1.0 plus the backport as in #19405 (EL9
RPMs, scons install), 31.10.2026. The client ran against a DAOS 2.8.0 server and agent.

• NLT, Linux 7.2 (EL9 with kernel-ml):
  • test_il_allow_idmap passes, and dfuse's log shows that idmapped mounts were allowed.
  • On an earlier base, with a dfuse built from the same tree but with the old check in
    ioctl.c, it failed with "op_sum is zero".
  • The whole suite, 98 tests with 116 dfuse instances, passes both without the option and with
    --allow-idmap forced on every dfuse. The server log has the same two warnings both times,
    from code this PR does not touch: a container open refused with DER_NO_PERM
    (srv_container.c) and an object I/O context without a handle (srv_obj.c).
• DAOS functional tests (ftest) that CI runs on VMs for dfuse changes (tags dfuse and vm;
  without daos_build_vm and ioctl_pool_handles). One server and one client VM, with the client
  on Linux 7.2. Each test ran twice: as it is, and with the harness starting every dfuse with
  --allow-idmap. Every test gave the same result both times:
  • 38 pass. They include the four mu_perms tests (dfuse of a regular user in multi-user mode;
    owner, group and other users with and without the interception libraries) and the 18
    variants of DaosCoreTestDfuse (libioil and libpil4dfs methods, 5 cache modes with and
    without interception, writeback only without).
  • The others fail the same way on master without this PR (checked on 0dbc4dd), or need the CI
    network:
    • test_bashfd_pil4dfs: bash 5.1 under libpil4dfs fails with
      xrealloc: cannot allocate 4294967464 bytes;
    • the libaio fio step of test_bashcmd_pil4dfs* hits an assertion in pil4dfs/aio.c:78;
    • the other test_bashcmd* tests stop at a curl of a host on the CI network, after all their
      file system steps have passed.
• dfuse_test (the program behind DaosCoreTestDfuse) was also run on its own, on AlmaLinux
  9.7 with Linux 5.14 and 7.2. It covered 4 cache modes without an interception library, and the
  3 modes other than writeback (as in ftest) with libioil, with libpil4dfs taking dfuse's handles,
  and with libpil4dfs as ftest runs it. dfuse ran as a user on both kernels and as root on 7.2, with
  and without the option. All tests pass.
• Functional checks, Linux 7.2. dfuse is started with and without --allow-idmap, and an
  idmapped mount of it is made with mount_setattr(), mapping 0-65535 to 100000-165535.
  • Without the option, the idmapped mount is refused with EINVAL.
  • With it, the mount is created. On-disk uid 0 appears as 100000. Files created by uid 101000 are
    stored as 1000:1000, and uid 101000 gets EACCES in a 0:0 0755 directory. chown, mkdir and
    rename work.
  • libioil and libpil4dfs of the dfuse user get dfuse's handles; libioil also from a nested pid
    namespace (libpil4dfs from a nested pid namespace: the Kubernetes pod below). Another user
    never gets them.
  • With dfuse in a pid namespace of its own and the host's /proc, callers outside or inside that
    namespace are refused.
  • With dfuse given a mounted /dev/fuse as /dev/fd/N by a helper that mounted it, idmapped
    mounts work, and libioil and libpil4dfs of the dfuse user are refused the handles. Without
    --allow-idmap they get them, as before. Without the /dev/fd/N check they got them with
    --allow-idmap too.
• Functional checks, Linux 5.14 (EL9). --allow-idmap starts and logs "Kernel does not support
  idmapped mounts". Everything behaves as without the option.
• Plain client host (AlmaLinux 9.7, SELinux enforcing): the client is installed from RPMs,
  daos_agent and dfuse run as systemd services.
  • The following behave the same with and without the option, on 5.14 and 7.2:
    • dfuse started by a user in daemon mode (fusermount3), with that user's interception
      libraries;
    • daos filesystem query and evict, daos container query --path;
    • libioil called from a thread other than the main one, which dfuse identifies through
      /proc/<pid>/task/<tid>.
  • Stock kernel 5.14.0-611: the option is harmless and the functional checks pass as without it.
    The kernel does not support idmapped FUSE mounts.
  • Kernel 7.2: podman containers with --userns=auto use the dfuse mount as an :idmap
    volume. Files of container root and uid 1000 are stored as 0:0 and 1000:1000. Without
    --allow-idmap, crun fails with mount_setattr ... Invalid argument. Four containers at
    once, each with its own uid range, see each other's files with these owners, and permissions
    hold between them.
• Kubernetes 1.37, on Linux 7.2 (kubeadm, containerd 2.4, runc 1.5) and on Talos 1.14
  (Linux 6.18). dfuse runs in mount pods with hostPID: true (a CSI driver):
  • Pods with hostUsers: false, as root and as uid 1000, use a volume of dfuse --allow-idmap.
    Their files are stored with the uid and gid they have in the pod.
    Permissions are enforced (uid 1000 cannot write root's 0644 file), and chown works.
  • libpil4dfs in a root pod without a user namespace (a pid namespace nested in dfuse's)
    intercepts I/O with dfuse's handles. In a user-namespaced pod it leaves the I/O to dfuse.
    libioil there opens its own handles.
  • Control: a pod with hostUsers: false on a volume of dfuse without --allow-idmap fails to
    start with the MOUNT_ATTR_IDMAP error.
  • 10 pods on one volume, 8 of them in user namespaces, each with its own uid range. Checked:
    • data, owners and permissions between the pods;
    • owners outside the uid range (seen as 65534);
    • subPath and readOnly mounts;
    • a pod replaced during I/O;
    • a second dfuse of the same container;
    • interception libraries in parallel, 8 libpil4dfs and 4 libioil processes in a pod with and
      one without a user namespace: only the latter's get dfuse's handles.
• Built with fused v1.0.0 (this PR alone on 58d0485, as CI builds it), Linux 7.2:
  • dfuse starts with --allow-idmap and logs "Fuse library does not support idmapped mounts";
    idmapped mounts are refused as without the option, and the kernel keeps sending the caller
    ids, so the interception libraries get dfuse's handles as before.
  • The whole NLT suite passes (98 tests), with the same two server log warnings;
    test_il_allow_idmap prints that idmapped mounts are not allowed.
• Lint. The checks of .github/workflows/linting.yml are clean on the changed lines:
  clang-format (repository style), flake8, pylint (utils/cq/daos_pylint.py with pyenchant and
  hunspell en_US, so its spelling check ran: no new warning), codespell and the copyright check.
  The copyright check requires the HPE line added to ioctl.c: check_update_copyright.sh adds it
  for authors outside HPE, Intel, Google and Enakta. Tell me if it should read differently.

Features: dfuse

Steps for the author:

• Commit message follows the guidelines.
• Appropriate Features or Test-tag pragmas were used.
• Appropriate Functional Test Stages were run.
• At least two positive code reviews including at least one code owner from each category referenced in the PR.
• Testing is complete. If necessary, forced-landing label added and a reason added in a comment.

After all prior steps are complete:

• Gatekeeper requested (daos-gatekeeper added as a reviewer).

[github-actions]

Ticket title is 'dfuse: support idmapped mounts for containers in user namespaces'
Status is 'Open'
https://daosio.atlassian.net/browse/DAOS-19898