Skip to content

rm doesn't really traverse directories like GNU rm at all, will fail on deeply-nested paths #2949

Description

@untitaker

GNU rm avoids constructing path strings of unbounded length to allow recursively deleting directories whose absolute paths exceed the max path length a Linux syscall can take. For more information please refer to this blogpost I wrote a while ago: https://unterwaditzer.net/2021/linux-paths.html

The rm in this repo appears to construct Path objects and therefore almost certainly will fail to remove the nested directories constructed in that article.

IIRC BSD rm works similarly to GNU rm here, and I would not be surprised if POSIX mandates a solution that works on really deeply nested filepaths.

There's probably more utilities in this repo which use Path objects, which immediately makes them fail in those edgecase situations where the equivalent GNU version might not.

Activity

  1. tertsdiepraam commented on Jan 29, 2022

    @tertsdiepraam
    Collaborator

    Interesting! So, if I understand correctly GNU does a sort of depth-first-traversal of the directories, removing each one it steps out of? We're currently traversing the directories with WalkDir. They have an open issue for this too, with some interesting discussion.

  2. untitaker commented on Jan 29, 2022

    @untitaker
    Author

    that's exactly the issue. i attempted to implement directory traversal with openat here: https://github.com/untitaker/fdwalk

    it will depend on usecase whether that's feasible. for ripgrep i believe it doesn't make any sense, as it needs to print filepaths, keep track of globs, ignorefiles, etc. for something "simple" and "essential" like coreutils i think it's necessary (it would suck if the OS gave me no tools to delete certain directories, so a "coreutils" command should always choose reliability over speed imo). it's also not too hard to do it as long as you don't use verbose option.

  3. tavianator commented on Apr 4, 2022

    @tavianator

    I would not be surprised if POSIX mandates a solution that works on really deeply nested filepaths.

    Indeed, https://pubs.opengroup.org/onlinepubs/9699919799/utilities/rm.html says:

    The rm utility shall be able to descend to arbitrary depths in a file hierarchy, and shall not fail due to path length limitations (unless an operand specified by the user exceeds system limitations).

  4. stale commented on Apr 4, 2025

    @stale

    This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

  5. untitaker commented on Apr 4, 2025

    @untitaker
    Author
  6. valpackett commented on Sep 17, 2025

    @valpackett

    Please consider cap-std for working with nested paths.

  7. untitaker commented on Sep 17, 2025

    @untitaker
    Author

    this issue might actually be fixed by #8517

    Please consider cap-std for working with nested paths.

    if you want to recursively delete a directory, you will eventually want to run openat(X, "../") to exit the recursion, and then delete that directory. from what i understand cap-std is designed to prevent this. keeping the parent file descriptor open will cause another pathological case where you run out of file descriptors in deeply-nested directories

  8. the8472 commented on Sep 17, 2025

    @the8472

    openat(X, "../")

    Note that you have to be extremely careful around this, otherwise you expose yourself confused deputy attacks.
    See the discussion in rust-lang/rust#93160 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions