kv-cache : index (seq,pos) cells to make ngram prev-token lookups O(log n) for qwen4exp decode speedup - #27992
Closed
tvanderka wants to merge 2 commits into
Closed
kv-cache : index (seq,pos) cells to make ngram prev-token lookups O(log n) for qwen4exp decode speedup#27992tvanderka wants to merge 2 commits into
tvanderka wants to merge 2 commits into
background
wait
wait-all
cancel
parallel
Loading