Skip to content

A literal value reaches the adjudicator and the review card instead of a question mark - #1220

Merged
WaylandYang merged 2 commits into
devfrom
fix/a-value-reaches-the-adjudicator
Oct 14, 2026
Merged

WaylandYang merged 2 commits into
devfrom
fix/a-value-reaches-the-adjudicator

Conversation

@WaylandYang

Copy link
Copy Markdown
Contributor

Why

A fact whose object is a value (a title, an employee number, an amount) has no entity on the other end, so entity_fact_lines_many rendered it as phrase → ?. The adjudicator's request and the review card both read these lines. On the identity bench about 52% of live facts are values, so half of what the adjudicator was shown about each record was a question mark. The scripted namesake cases from #1202 named this as a known failure (adjudicator_literal_clues_reach_request).

What changes

  • The line shows the value, cut to 120 characters (a value can be a whole sentence).
  • The two adjudicator_literal_clues_reach_request rows leave KNOWN_FAILURES; the resolver row stays.

Measured

Identity bench, both builds in one session, eight bases a side (deepseek-v4-flash through OpenRouter, reasoning off, bge-m3):

dev 42cfb94 this branch
F1, mean (range) 0.687 (0.59–0.80) 0.629 (0.47–0.79)
false merges 0 0
bases where the two 张伟 end as one entity 0 of 8 0 of 8
false splits, total 55 58

The F1 gap comes from two bases and sits inside the spread. No namesake mixing happened that day on either side, so the bench could not show the benefit either.

Controlled re-adjudication: four bases built once with batch adjudication held back (71 queued pairs), restored four times and adjudicated by each build twice. On the 54 pairs every run decided (one baseline batch failed three times on a truncated model reply):

pairs whose verdict differs
dev vs dev (second run) 8 of 54
branch vs branch 9 of 54
dev vs branch (four pairings) 7, 8, 9, 10 of 54

The change moves verdicts no more than rerunning the same build does. The adjudicator itself flips about 15% of these verdicts between runs. The two distinct 张伟 were never merged in any run.

Tests

namesake_scripted_cases passes with the shorter list; cargo test -p utopia-store and the adjudication and governance tests pass; clippy is clean.

🤖 Generated with Claude Code

WaylandYang and others added 2 commits October 14, 2026 16:14
…f a question mark

A fact whose object is a value (a title, an employee number, an amount)
has no entity on the other end, so its line read "phrase → ?". The
adjudicator and the review card both lost it; on the identity bench about
half of all live facts are values. The line now shows the value, cut to
120 characters. The two adjudicator rows of the scripted namesake cases
(#1202) that measured this leave the known-failure list.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Wayland Yang <wayland0916@gmail.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Wayland Yang <wayland0916@gmail.com>
@WaylandYang
WaylandYang merged commit b365630 into dev Oct 14, 2026
6 checks passed
@WaylandYang
WaylandYang deleted the fix/a-value-reaches-the-adjudicator branch October 14, 2026 08:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant