Skip to content

Fix crash on dangling symlinks when copying into mutants/ (#582) - #595

Open
Ishaan400 wants to merge 2 commits into
boxed:mainfrom
Ishaan400:fix-582-dangling-symlinks
Open

Ishaan400 wants to merge 2 commits into
boxed:mainfrom
Ishaan400:fix-582-dangling-symlinks

Conversation

@Ishaan400

Copy link
Copy Markdown

Fixes #582.

Problem

mutmut run crashes if a directory it copies into mutants/ contains a dangling symlink:

  • Under an also_copy directory: copy_also_copy_files() → shutil.copytree → shutil.Error

  • Under a source path: copy_src_dir() → shutil.copy2 → FileNotFoundError

Both follow symlinks by default, so a link whose target is missing can't be read.

Fix

A small helper, _copy_keeping_dangling_symlinks, replaces copy2 in copy_src_dir and is passed as copy_function to the copytree calls:

  • A dangling symlink is recreated as a symlink in mutants/. Anything an earlier run left at that path is replaced, so repeated runs don't fail with FileExistsError.

  • Everything else goes through shutil.copy2 as before. Valid symlinks, including relative ones that point outside the copied directory, are still copied as the files they point to.

  • If a link kept by an earlier run now has an existing target, it is removed before copying. Otherwise copy2 writes through it, which leaves a symlink where a fresh run makes a regular file, and raises SameFileError for an absolute target.

Why not the standard-library options (checked on Python 3.10 and 3.13)?

  • copytree(..., symlinks=True) copies valid relative links as links, so links that point outside the copied directory break inside mutants/.

  • copytree(..., ignore_dangling_symlinks=True) drops dangling links instead of keeping them. It checks a relative link's target against the current directory, so it can also drop valid relative links.

Tests

New tests in tests/utils/test_file_utils.py:

  • A dangling link and a valid relative link, in an also_copy directory and in a source path.

  • Repeated runs with dangling links left in mutants/.

  • A dangling link whose target appears before the next run (relative and absolute target).

  • A valid link that becomes dangling before the next run.

All 6 test cases fail on main and pass with this change, on Python 3.10 and 3.13. Full suite on Python 3.10: 459 passed, 4 skipped. pre-commit run --all-files passes (ruff, ruff-format, mypy).

I also ran a small project with a dangling link under tests/ and one under src/: mutmut run exits with an error on main, and completes and kills the mutant with this change.

Performance

You asked about the performance cost, so I measured it. Setup: a 2-vCPU Linux VM, ext4 disk, plus tmpfs as a no-disk worst case. Each benchmark interleaves main and this PR, copies into a fresh destination each run, and reports the median.

Where the cost comes from. For each regular file, the helper adds only metadata calls (lstat/stat, no data I/O). Counted with strace while copying 1,000 files, identical on Python 3.10 and 3.13:

Copy path | Syscalls per file on main | With this PR -- | -- | -- copytree (also_copy directories) | 21.1 | 25.1 copy_src_dir file loop | 24.7 | 26.7

For the Python standard-library first run, the interquartile ranges were 214 ms on main and 337 ms with this PR, indicating substantial run-to-run variation.

Worst case, in memory. On tmpfs there's no disk latency, so the relative overhead is largest. Copying 5,000 small files with copytree (median of 21 runs) took 236 → 282 ms on Python 3.10 and 217 → 267 ms on 3.13: about 10 µs per file, or 20% more time. On ext4 (Python 3.13, 15 runs), synthetic workloads without dangling links differed by less than run-to-run noise. Workloads with dangling links can't be compared because main crashes on them.

The cost is roughly 10 µs per copied file in the worst case. On the real trees above, the difference was under 20 ms per mutmut run and within run-to-run noise. copy_src_dir skips files that already exist in mutants/, and also_copy directories are recopied on each run.

I don't think this needs an option, but I'm happy to add one if you see it differently. I can share the benchmark scripts if they'd help.

Copying a source or also_copy directory that contains a dangling symlink
crashed mutmut run: shutil.copytree and shutil.copy2 follow symlinks by
default. Recreate dangling symlinks as symlinks instead, and keep
following valid ones so their behaviour is unchanged. A link left in
mutants/ by a previous run is replaced, so repeated runs work.
A dangling symlink kept in mutants/ by an earlier run was copied through
when its target appeared later: copy2 writes through the destination
link, which left a symlink instead of a copy, and raised SameFileError
for an absolute target. Remove the stale link before copying. Also test
a valid link that becomes dangling, and add a changelog entry.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

mutmut breaks on tests with fixture directories with broken symlinks

1 participant