perf: enhance tokens block memory usage

[cheb0]

Description

Memory reduction (prod fraction):

message:*foobar*
25335032 => 20584912 bytes

message:*87466437*
71844952 => 58236912

Performance improvement: 5-10% for wildcards like message:*5647545* (cold requests). For wildcards with letters there is no benefit - we block on disk read time more since we skip blocks.

---

• I have read and followed all requirements in CONTRIBUTING.md;
• I used LLM/AI assistance to make this pull request;

[codecov-commenter]

Codecov Report

❌ Patch coverage is 61.22449% with 19 lines in your changes missing coverage. Please review.
✅ Project coverage is 71.04%. Comparing base (6372ee9) to head (c42261c).
⚠️ Report is 5 commits behind head on main.

Files with missing lines	Patch %	Lines
frac/sealed/token/block_loader.go	55.81%	15 Missing and 4 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #486      +/-   ##
==========================================
- Coverage   71.12%   71.04%   -0.08%     
==========================================
  Files         232      232              
  Lines       18404    18533     +129     
==========================================
+ Hits        13089    13167      +78     
- Misses       4337     4383      +46     
- Partials      978      983       +5     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[dkharms]

@seqbenchbot start search-regular main 'k8s_pod:-eu-' --duration 5m

[seqbenchbot]

Nice, @dkharms <(-^,^-)=b!

Started benchmark 58d2faa3.

Show details

• scenario search-regular;
• against main;
• running for 5m0s;

I'll post the statistics here once it finishes.
To end it early, run @seqbenchbot stop 58d2faa3.

Here is a list of helpful links:

• Take a look at Grafana dashboard;
• Live-tailing logs are also available;

Have a great time!

[seqbenchbot]

Nice, @dkharms <(-^,^-)=b!

Benchmark 58d2faa3 (scenario search-regular) has finished.
I've prepared a summary for you. Click on Show summary button to see it:

Show summary

Query	Type	mean (ms)	stddev (ms)	p(50) (ms)	p(95) (ms)	p(99) (ms)	iterations	total
k8s_pod:*-eu-*	hot	11.80	0.74	12.17	17.54	18.20	2992	1437046

Compare it against another run with @seqbenchbot compare 58d2faa3 <other-id>.

[dkharms]

@seqbenchbot start search-regular 0-enhance-token-block-memory 'k8s_pod:-eu-' --duration 5m

[seqbenchbot]

Nice, @dkharms <(-^,^-)=b!

Started benchmark 43c61a79.

Show details

• scenario search-regular;
• against 0-enhance-token-block-memory;
• running for 5m0s;

I'll post the statistics here once it finishes.
To end it early, run @seqbenchbot stop 43c61a79.

Here is a list of helpful links:

• Take a look at Grafana dashboard;
• Live-tailing logs are also available;

Have a great time!

[seqbenchbot]

Nice, @dkharms <(-^,^-)=b!

Benchmark 43c61a79 (scenario search-regular) has finished.
I've prepared a summary for you. Click on Show summary button to see it:

Show summary

Query	Type	mean (ms)	stddev (ms)	p(50) (ms)	p(95) (ms)	p(99) (ms)	iterations	total
k8s_pod:*-eu-*	hot	12.15	2.71	10.95	16.47	17.00	2992	1437020

Compare it against another run with @seqbenchbot compare 43c61a79 <other-id>.