Skip to content

Apply one-to-one selection when all match scores are equal - #107

Merged
kPsarakis merged 1 commit into
fix/105-metric-name-collisionsfrom
fix/one-to-one-tied-scores
Oct 30, 2026
Merged

kPsarakis merged 1 commit into
fix/105-metric-name-collisionsfrom
fix/one-to-one-tied-scores

Conversation

@kPsarakis

@kPsarakis kPsarakis commented Oct 30, 2026 •

Copy link
Copy Markdown
Member

Stack: 2 of 3. #106 → this PR → #109 (metric input validation and docs). Based on #106, so the diff shows only this change. Merge #106 first.

Problem

one_to_one_hungarian() and one_to_one_greedy() return early when the results have fewer than two distinct scores, because the median threshold can't be computed then. The early return also skips the 1:1 selection itself, so:

  • Tied results stay many-to-many. JaccardDistanceMatcher on two tables whose columns hold identical values scores all 4 pairs 1.0; "1:1" keeps all 4, and Precision() reports 0.5 where a 1:1 selection gives 0.0 or 1.0.
  • An explicit threshold is ignored. one_to_one_hungarian(threshold=0.9) on a single 0.2-scored pair still returns that pair.

Since these are the default filters behind Precision, Recall, F1Score and PrecisionTopNPercent, those metrics are affected.

Change

  • The default threshold moves into MatcherResults._default_threshold(). When all scores are equal it returns that score, so nothing is cut; otherwise it uses the same formula as before.
  • Both selectors always run the 1:1 selection and always apply an explicit threshold.
  • one_to_one_greedy() returns early only for empty results.
  • Results with two or more distinct scores are unaffected.

Tests

New TestOneToOneTiedScores in tests/test_coverage_gaps.py, replacing TestGreedyEarlyReturn:

  • tied scores stay one-to-one (greedy and Hungarian)
  • an explicit threshold is applied with one distinct score
  • Precision uses the 1:1 selection
  • end-to-end check with JaccardDistanceMatcher

6 of the new tests fail on #106 without this change. Full suite: 318 passed locally.

🤖 Generated with Claude Code

@kPsarakis
kPsarakis force-pushed the fix/one-to-one-tied-scores branch from 9f4f64d to a6106b4 Compare October 30, 2026 11:22
@kPsarakis
kPsarakis added this pull request to stack #108 October 30, 2026 11:29
one_to_one_hungarian() and one_to_one_greedy() returned early when the
results had fewer than two distinct scores, because the median
threshold can't be computed then. The early return also skipped the 1:1
selection itself and ignored an explicit threshold, so tied results
(e.g. JaccardDistanceMatcher on columns with identical values) stayed
many-to-many and the 1:1 Precision/Recall/F1Score were computed on them.

Move the default threshold into _default_threshold(), which returns the
single score when all scores are equal, and always run the selection.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Oct 30, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.27%. Comparing base (957414b) to head (99f2d20).

Additional details and impacted files

Impacted file tree graph

@@                        Coverage Diff                         @@
##           fix/105-metric-name-collisions     #107      +/-   ##
==================================================================
+ Coverage                           95.24%   95.27%   +0.02%     
==================================================================
  Files                                  53       53              
  Lines                                2839     2833       -6     
  Branches                              446      443       -3     
==================================================================
- Hits                                 2704     2699       -5     
  Misses                                 83       83              
+ Partials                               52       51       -1     
Flag Coverage Δ
unit 95.27% <100.00%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
valentine/algorithms/matcher_results.py 98.70% <100.00%> (+0.57%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@kPsarakis
kPsarakis merged commit 5df5bfd into master Oct 30, 2026
57 of 59 checks passed
@kPsarakis
kPsarakis deleted the fix/one-to-one-tied-scores branch October 30, 2026 11:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant