Skip to content

feat: Strengthen the same-file saturation decay - #306

Closed
flyinprogrammer wants to merge 1 commit into
MinishLab:mainfrom
flyinprogrammer:feat/stronger-file-saturation
Closed

flyinprogrammer wants to merge 1 commit into
MinishLab:mainfrom
flyinprogrammer:feat/stronger-file-saturation

Conversation

@flyinprogrammer

Copy link
Copy Markdown

Closes #305

Summary

Each extra chunk from an already-listed file is scaled by _FILE_SATURATION_DECAY. At 0.5, a strong second chunk of the top file still pushes other files out of the top 10; 0.25 lets another file through:

 # semble/ranking/penalties.py
-_FILE_SATURATION_DECAY = 0.5
+_FILE_SATURATION_DECAY = 0.25
rerank_topk({parser.py#1: 1.0, parser.py#2: 0.9, loader.py: 0.4})
  0.5 : parser.py#1, parser.py#2 (0.45), loader.py (0.40)
  0.25: parser.py#1, loader.py (0.40), parser.py#2 (0.225)

Evidence

The new test_rerank_topk_prefers_another_file_over_a_second_chunk asserts the 0.25 order above.

  • Before (main's penalties.py): 1 failed.
  • After: 1 passed.
  • Full suite: make test 392 passed; make lint and make typecheck clean.

NDCG@10, compared with main (95% CIs resample repos):

Query set Δ 95% CI Better / worse queries
Benchmark annotations +0.09 [+0.04, +0.15] —
Commit-message queries (#301) +0.08 [+0.04, +0.13] 41 / 0

run_benchmark: 0.852 → 0.853. Decay 0.1 and 0.0 change nothing further; 0.75 and 1.0 are worse.

Merge Danger

Door: two-way

A one-constant ranking change with no index or cache change; reverting restores the old ranking exactly.

Blast Radius: ranking

Only result order changes, and only when a file has several chunks among the candidates; the result count is unchanged.

Each extra chunk from an already-listed file was scaled by 0.5, so a strong second
chunk of the top file still pushed other files out of the top 10. That hurts
queries whose answer spans several files. Lower _FILE_SATURATION_DECAY to 0.25.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Same-file saturation decay of 0.5 lets a second chunk of the top file crowd out other files

2 participants