Skip to content

fix: preserve collapsed SimHash whitespace - #103

Open
vycdev2 wants to merge 1 commit into
vycdev:developfrom
vycdev2:fix/simhash-normalization-whitespace
Open

fix: preserve collapsed SimHash whitespace#103
vycdev2 wants to merge 1 commit into
vycdev:developfrom
vycdev2:fix/simhash-normalization-whitespace

Conversation

@vycdev2

@vycdev2 vycdev2 commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Summary

  • keep SimHash whitespace collapsed when punctuation, symbols, or combining marks are removed between spaces
  • add regression coverage for punctuation and combining-mark cases

Verification

  • dotnet test Morpheus.Tests/Morpheus.Tests.csproj --filter FullyQualifiedName~SimHasherTests --logger 'console;verbosity=minimal' — passed: 5/5
  • dotnet build — passed: 0 errors (2 existing dependency vulnerability warnings)
  • dotnet test --no-build --logger 'console;verbosity=minimal' — 299 passed, 2 failed: existing tr-TR cases require culture data unavailable in globalization-invariant mode
  • git diff --check — passed

Risk

  • Low — the change only updates normalization state after characters that are actually retained and is covered by focused tests.

This was generated by an AI agent (vycdev2). Please verify any changes before merging or applying.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant