Skip to content

fix: cap k at 16, the longest k-mer the index can store, instead of panicking - #38

Open
Roman-Young wants to merge 1 commit into
IEDB:masterfrom
Roman-Young:fix/k-max-16
Open

Roman-Young wants to merge 1 commit into
IEDB:masterfrom
Roman-Young:fix/k-max-16

Conversation

@Roman-Young

Copy link
Copy Markdown
Contributor

Any search whose k came out above 16 panicked in the preprocessor:

pyo3_runtime.PanicException: range start index 18446744073709551615 out of range for slice of length 16

The chunked preprocessor (baab435) packs each k-mer into a u128, one byte per residue, so 16 is the longest k-mer an index can hold, and to_be_bytes()[16 - k..] underflows at k=17. Since auto-k replaced the fixed default of 5 (31cbba3), exact search picks k = the shortest query's length, so default settings reach that limit for any batch whose shortest query is 17 residues or longer. Reproduced on master:

Search Query length that panics
exact (and counts_only) 17+
1 mismatch 34+
1 / 2 indels 34+ / 51+
best_match (starts at the shortest query's length) 17+
explicit k=17, or Preprocessor.preprocess(17) any

This PR:

  • Adds MAX_INDEX_K = 16 in preprocessor.py, next to the index it describes.
  • Caps auto-k at 16: _auto_k (exact, mismatch, counts), the indel group k, and best_match's starting k. A smaller k only adds seeds, so complete recall is unchanged: with k ≤ L // (edits + 1), every query still splits into edits + 1 disjoint seeds.
  • Rejects an explicit k > 16 in Matcher and Preprocessor.preprocess with a ValueError instead of a panic.
  • README: documents the 2 to 16 range and corrects the stale "default: 5" for pepmatch-match -k.

Nothing changes for k ≤ 16. For best_match, the first pass on long queries now runs at k=16 with min_len // 16 - 1 mismatches instead of at k=L with 0. The per-query filter still keeps the fewest-mismatch hit, so each query's best match is the same.

Tests: new test_k_limit.py (with a CI matrix row):

  • each path above finds its hit, and the search runs at k=16
  • explicit k=17 and preprocess(17) raise ValueError
  • 16-mers still run at k=16
  • long queries (17 to 59 residues) match a brute-force Hamming scan (0 to 2 mismatches) and the indel2_brute_force oracle (1 to 2 indels)

27 of the 31 new tests fail on master. Full suite green: 267 passed, 2 skipped.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant