Repository navigation
Conversation
Copying a source or also_copy directory that contains a dangling symlink crashed mutmut run: shutil.copytree and shutil.copy2 follow symlinks by default. Recreate dangling symlinks as symlinks instead, and keep following valid ones so their behaviour is unchanged. A link left in mutants/ by a previous run is replaced, so repeated runs work.
A dangling symlink kept in mutants/ by an earlier run was copied through when its target appeared later: copy2 writes through the destination link, which left a symlink instead of a copy, and raised SameFileError for an absolute target. Remove the stale link before copying. Also test a valid link that becomes dangling, and add a changelog entry.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #582.
Problem
mutmut runcrashes if a directory it copies intomutants/contains a dangling symlink:Under an
also_copydirectory:copy_also_copy_files()→shutil.copytree→shutil.ErrorUnder a source path:
copy_src_dir()→shutil.copy2→FileNotFoundErrorBoth follow symlinks by default, so a link whose target is missing can't be read.
Fix
A small helper,
_copy_keeping_dangling_symlinks, replacescopy2incopy_src_dirand is passed ascopy_functionto thecopytreecalls:A dangling symlink is recreated as a symlink in
mutants/. Anything an earlier run left at that path is replaced, so repeated runs don't fail withFileExistsError.Everything else goes through
shutil.copy2as before. Valid symlinks, including relative ones that point outside the copied directory, are still copied as the files they point to.If a link kept by an earlier run now has an existing target, it is removed before copying. Otherwise
copy2writes through it, which leaves a symlink where a fresh run makes a regular file, and raisesSameFileErrorfor an absolute target.Why not the standard-library options (checked on Python 3.10 and 3.13)?
copytree(..., symlinks=True)copies valid relative links as links, so links that point outside the copied directory break insidemutants/.copytree(..., ignore_dangling_symlinks=True)drops dangling links instead of keeping them. It checks a relative link's target against the current directory, so it can also drop valid relative links.Tests
New tests in
tests/utils/test_file_utils.py:A dangling link and a valid relative link, in an
also_copydirectory and in a source path.Repeated runs with dangling links left in
mutants/.A dangling link whose target appears before the next run (relative and absolute target).
A valid link that becomes dangling before the next run.
All 6 test cases fail on
mainand pass with this change, on Python 3.10 and 3.13. Full suite on Python 3.10: 459 passed, 4 skipped.pre-commit run --all-filespasses (ruff, ruff-format, mypy).I also ran a small project with a dangling link under
tests/and one undersrc/:mutmut runexits with an error onmain, and completes and kills the mutant with this change.Performance
You asked about the performance cost, so I measured it. Setup: a 2-vCPU Linux VM, ext4 disk, plus tmpfs as a no-disk worst case. Each benchmark interleaves
mainand this PR, copies into a fresh destination each run, and reports the median.Where the cost comes from. For each regular file, the helper adds only metadata calls (
Copy path | Syscalls per file on main | With this PR -- | -- | -- copytree (also_copy directories) | 21.1 | 25.1 copy_src_dir file loop | 24.7 | 26.7lstat/stat, no data I/O). Counted withstracewhile copying 1,000 files, identical on Python 3.10 and 3.13:For the Python standard-library first run, the interquartile ranges were 214 ms on
mainand 337 ms with this PR, indicating substantial run-to-run variation.Worst case, in memory. On tmpfs there's no disk latency, so the relative overhead is largest. Copying 5,000 small files with
copytree(median of 21 runs) took 236 → 282 ms on Python 3.10 and 217 → 267 ms on 3.13: about 10 µs per file, or 20% more time. On ext4 (Python 3.13, 15 runs), synthetic workloads without dangling links differed by less than run-to-run noise. Workloads with dangling links can't be compared becausemaincrashes on them.The cost is roughly 10 µs per copied file in the worst case. On the real trees above, the difference was under 20 ms per
mutmut runand within run-to-run noise.copy_src_dirskips files that already exist inmutants/, andalso_copydirectories are recopied on each run.I don't think this needs an option, but I'm happy to add one if you see it differently. I can share the benchmark scripts if they'd help.