Add Full-text Conversation Search with Match Previews

[fy17ohhh]

Previous conversation search only supports title search, this added feature would enhance the console conversation browser for users to search across saved conversation titles & message content.

• Add full-text search for visible user and assistant messages, lazily load and cache conversation histories during each browser session.
• Display the first matching title or message in the Preview panel and highlight the matched text
• Remove deleted conversations from both filtered and unfiltered lists.
• Added testing for this new feature tests/console/test_conversation_search.py

[daltonnyx]

Could you help to separate the config changes and only keep the full text search in this PR?

[fy17ohhh]

Could you help to separate the config changes and only keep the full text search in this PR?

updated:)

[quytruongsts]

Requesting changes: full-text message search freezes the UI at scale

Thanks for the feature — the design (lazy loading, session caching, casefold-aware matching, and the delete-from-both-lists fix) is solid. However, I benchmarked this against a simulated 1000-conversation store using the same I/O path as the real app (ContextPersistenceService.get_conversation_history → open() + json.loads(), no caching) and found a blocking performance issue with message search.

For calibration: real history files in persistents/conversations average ~60 KB (max ~880 KB).

Measured results (1000 conversations)

Query	~10 KB avg files	~60 KB avg files (realistic)
Title-only search	~6 ms	~6 ms
Message search — 1st keystroke (cold)	~1.1 s	~6.3 s
Message search — later keystrokes (warm, cached)	~1.0 s	~6.0 s
Cached fragment memory	~7 MB	~42 MB

Root cause

File I/O is actually the minor cost (~65 ms light / ~350 ms heavy for 1000 reads + parses). The dominant cost is in _find_casefold_span(): on every keystroke it re-casefolds and rebuilds offset maps for every cached fragment in a Python per-char loop.

Profiling a single warm query shows 99% of time inside _find_casefold_span (7.7M str.casefold + 15.5M list.append/extend calls), while the actual str.find is 0.005 s. The lazy-load cache fixes disk reads but does nothing for this — which is why warm queries are nearly as slow as cold ones.

Impact: with 1000 realistic (~60 KB) conversations, any message-content query that doesn't match titles produces a ~6 s UI freeze per keystroke in the browser render loop. Title-only search is unaffected because title matches short-circuit before message scanning.

Requested change

Pre-normalize at index time: casefold each fragment's text once when the history is loaded, and cache the normalized text + original-offset map alongside the fragment (or store normalized text with precomputed offsets in SearchFragment).

# At index time (in _get_message_fragments):
#   normalized = text.casefold(), offsets = <precomputed char offset map>
# At query time (in filter):
#   pos = fragment.normalized.find(normalized_query)   # C-speed, no per-char loop

Warm queries then become plain str.find over pre-folded text (~100× faster → tens of ms). A small keystroke debounce (150–250 ms) in the input handler would be a cheap additional safeguard.

Happy to help verify with a follow-up benchmark once the normalization is moved to index time.