fix(extraction): keep validation-emptied batches out of the noise bank - #959
Conversation
…, adapted to master's ok-result shape)
3a9648d to
5fe4075
Compare
|
Recomposed against current master (post-#988) as a single commit carrying only this family's delta; full suite green locally. Ready for review. |
rwmjhb
left a comment
There was a problem hiding this comment.
Reviewed head 5fe4075. The new rawCandidateCount gate correctly distinguishes genuinely empty model output from non-empty batches removed by validation, preventing those conversations from training the noise bank. The focused noise-gating suite passes 7/7, the full suite and required GitHub checks are green, npm run build succeeds, and the generated tree remains clean.
Approved. Two non-blocking cleanup items: the new demotion log always says “constructed siblings present” even for failure-closed cells with no constructed sibling, and the nearby return-path comment still describes the superseded quality-drop learning behavior.
What Problem This Solves
Two side effects of the extraction validation layer:
extractAndPersisttrains the embedding noise bank whenever extraction returns ok with zero candidates. A batch can reach zero candidates through validation drops (constructed grounding, fiction register, the batch contradiction demotion) rather than genuine emptiness. In that shape the model DID emit candidates; feeding the conversation to the noise bank as a noise exemplar teaches "content like this is noise" and risks pre-filtering similar real content away from future extractions. Observed live: a mixed-register conversation containing a real durable fact was fed to the noise bank after the contradiction check demoted the whole batch.The batch contradiction check logs its demotions at debug only, so a fully demoted batch is indistinguishable from "model found nothing" at the standard log level. That ambiguity cost a live investigation.
Why This Change Was Made
extractCandidatesnow reportsrawCandidateCount(items the model emitted before validation) on the ok result. Noise learning fires only when that raw count is zero; validation-emptied batches log a standard-level skip line instead.The demotion policy itself is unchanged; this only fixes its side effects.
User Impact
Deployments stop committing wrong "this is noise" beliefs to the learning store when validation empties a batch, and operators can see demotions without enabling debug logging.
Evidence
Note: this branch is cut from the grounding PR's head (#927) because the demotion machinery lives there; it stays a draft until #927 settles and merges.