Skip to content

Skip .git, .svn, .hg and .bzr when dedup scans a tree [patch] - #334

Open
matt-edmondson wants to merge 1 commit into
mainfrom
fix/222-dedup-skip-vcs-metadata
Open

matt-edmondson wants to merge 1 commit into
mainfrom
fix/222-dedup-skip-vcs-metadata

Conversation

@matt-edmondson

Copy link
Copy Markdown
Contributor

Fixes #222

What changes

Inside .git, every branch ref that points at the same commit has identical content, and so do the reflogs of branches created together. dedup walked into .git like any other folder, so dedup dedupe kept one of those files and deleted the rest. That deleted branches, remote-tracking refs and reflogs in every repository under the root.

FileDedupeService.EnumerateRegularFiles now walks with a FileSystemEnumerable. It prunes .git, .svn, .hg and .bzr directories (name match, case-insensitive) instead of filtering paths afterwards, and it leaves out a .git pointer file, as the issue asks: deleting a worktree's or submodule's pointer would detach that checkout. The reparse-point and link-target rules from #219 are unchanged. scan, dry-run, dedupe and stats all go through this helper, so all four are covered.

This is the same rule as ktsu-dev/FileDeduplicator#194 (fix for FileDeduplicator#192). The one difference is the .git pointer file, which this issue asked to skip and that one didn't.

The optional --exclude <name> idea from the issue isn't included.

Tests

  • DedupeLeavesVersionControlMetadataAlone, one row each for .git/.svn/.hg/.bzr. Three identical refs plus a real duplicate pair: only the copy outside the metadata is planned for removal and deleted, and every ref survives.
  • EnumerateRegularFilesSkipsAGitPointerFileButNotLookAlikes: a worktree .git file is left out, while .github/, mirror.git/ and .gitignore are listed.
  • With the service change reverted, all 5 new cases fail. Full KtsuTools.Test run: 390 passed, 0 failed.

CI note

main currently fails restore with NU1903 on SixLabors.ImageSharp 3.1.12 (GHSA-v76p-62qx-wwq2). Every open KtsuTools PR's test jobs are red for that reason, e.g. #331. That failure isn't from this change, and I verified locally with -p:WarningsNotAsErrors=NU1903. The ImageSharp bump in #270 is the candidate fix, but IconHelper#155 reports that 3.2.0 fails CI without a Six Labors license, so it needs a maintainer decision.


Generated by Claude Code

Inside .git, every branch ref pointing at the same commit has identical
content, as do the reflogs of branches created together. dedup walked into
.git like any other folder, so dedupe kept one of those files and deleted the
rest, deleting branches, remote-tracking refs and reflogs in every
repository under the root.

EnumerateRegularFiles now prunes version-control metadata directories by
name and leaves out a .git pointer file, so scan, dry-run, dedupe and stats
never see them. Directories such as .github and mirror.git are still
scanned.

Fixes #222

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017vfJW34mg5yez39UfTV7yf

Copy link
Copy Markdown
Contributor Author

ci / .NET / Test on ubuntu-latest fails at restore, before any test runs. The same failure is expected on macOS and Windows:

error NU1903: Warning As Error: Package 'SixLabors.ImageSharp' 3.1.12 has a known high severity vulnerability, https://github.com/advisories/GHSA-v76p-62qx-wwq2

This failure isn't caused by this PR, which only touches KtsuTools.FileDedupe and its tests. main fails the same way, and so does every other open PR, for example #331. I didn't re-run the job because the failure is deterministic and a re-run would fail the same way.

I haven't found a fix to port into this PR. The candidate is #270, which bumps ImageSharp to 3.2.0, but IconHelper#155 reports that 3.2.0 fails CI without a Six Labors license. Choosing between that bump, a license or another option needs a maintainer decision.

Locally, with -p:WarningsNotAsErrors=NU1903, the full KtsuTools.Test suite passes: 390 of 390.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

dedup dedupe descends into .git and deletes branch refs and reflogs that happen to hold the same commit SHA

2 participants