Skip to content

Warn about a missing history component only when one could be missing - #1209

Merged
mdekstrand merged 1 commit into
lenskit:mainfrom
appleweiping:fix/804-history-warning-false-positives
Sep 4, 2026
Merged

mdekstrand merged 1 commit into
lenskit:mainfrom
appleweiping:fix/804-history-warning-false-positives

Conversation

@appleweiping

Copy link
Copy Markdown
Contributor

Fixes #804.

ItemKNNScorer emits no query items, did you omit a history component? whenever query.query_items is None. That covers two very different situations and is only correct for one of them.

RecQuery.create returns a query unchanged if it is already a RecQuery, and otherwise wraps a bare user ID in a query with no history at all. So an absent history indicates a pipeline gap only when the caller passed something that was not already a query. When a query arrives, a history component has already run, and an empty history is its normal result for a user who is not in the training data — the false positive the issue describes.

The warning is now emitted only in the first case. The second still logs at debug level via the existing user has no history, returning message. This follows the direction suggested in the issue (keying off whether the query is a RecQuery).

Reproduction

Against 2026.3.0, with a history component present and doing its job:

lookup = UserTrainingHistoryLookup(); lookup.train(ds)
scorer = ItemKNNScorer(k=5); scorer.train(ds)
scorer(lookup(999), ItemList([6, 7, 8]))   # user 999 is not in ds
[warning] no query items, did you omit a history component? _batch=lenskit.knn.item.NO_HISTORY user_id=999

After the change that call is silent, while scorer(999, ItemList([6, 7, 8])) — a bare ID, no history component in front — still warns.

UserKNNScorer is deliberately unchanged

The issue mentions other components. UserKNNScorer already handles this correctly: it looks the user up in its training vocabulary before warning, and reports has no ratings and none provided, which is accurate and does not blame a component. ItemKNNScorer cannot do the same — it retains only items, item_means and sim_matrix, so it has no user vocabulary to consult, which is why the query-shape test is used instead.

Testing

Adds a regression test asserting both directions via structlog.testing.capture_logs. It fails on the first assertion against current main and passes with this change. tests/models/test_knn_item_item.py is 41 passed / 9 skipped and test_knn_user_user.py is 40 passed / 8 skipped with the change applied.

One note on my environment: the Rust toolchain on this machine could not build the extension, so I verified against the released 2026.3.0 wheel with this patch applied to the installed package, running the repo's own test file against it. The change itself is pure Python and touches no compiled code.

🤖 Generated with Claude Code

`ItemKNNScorer` warns "no query items, did you omit a history component?"
whenever `query.query_items` is None. That fires for two very different
situations, and is only correct for one of them.

`RecQuery.create` returns a query unchanged if it is already a `RecQuery`, and
otherwise wraps a bare user ID in a query with no history at all. So an absent
history means a pipeline gap only when the caller passed something that was not
already a query. When a query arrives, a history component has already run, and
an empty history is its normal result for a user who is not in the training
data — a false positive, as reported in lenskit#804.

The warning is now emitted only for the first case. The second still logs at
debug level through the existing "user has no history, returning" message.

`UserKNNScorer` is deliberately left alone. It checks the user against its
training vocabulary before warning and reports "has no ratings and none
provided", which is accurate for the case it covers and does not blame a
component. `ItemKNNScorer` cannot do the same: it retains only items, item
means and the similarity matrix, so it has no user vocabulary to consult.

Adds a regression test covering both directions: a query for an unknown user
produces no warning, while a bare ID for the same user still does. The test
fails against the current behaviour on the first assertion.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@codecov

codecov Bot commented Sep 4, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.40%. Comparing base (34c81c5) to head (a1ae703).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1209   +/-   ##
=======================================
  Coverage   90.40%   90.40%           
=======================================
  Files         263      263           
  Lines       17516    17517    +1     
=======================================
+ Hits        15836    15837    +1     
  Misses       1680     1680           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@mdekstrand
mdekstrand merged commit 1a5c7e1 into lenskit:main Sep 4, 2026
43 checks passed
@mdekstrand mdekstrand added this to the 2026.4 milestone Sep 4, 2026
@mdekstrand mdekstrand added the components LensKit recommendation components label Sep 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

components LensKit recommendation components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Revise missing history warning for item-item (and other components)

2 participants