Let document tools continue reading past their response budget

[Floating-Y]

Why

Closes #1209.

A ready document can contain the answer past the 24,000-character response budget, but every get_document call previously returned the same prefix. Callers could not reach the stored tail.

What changes

• Add an optional, non-negative offset to the shared Chat/MCP tool. Omitting it still starts at zero. Offsets count Unicode characters across trimmed section bodies in order, excluding headers and separators.
• Keep the 24,000-character budget per call and include the next offset and document ID in the truncation notice. Reads at or past the end report no remaining text.
• Reject invalid offsets and preserve existing document access and database error handling.
• Update the shared schema, Chat guidance, and MCP manual. Add regressions for oversized table rows, Unicode, continuation inside/across chunks, defaults, invalid offsets, exhausted reads with empty chunks, and unavailable documents.

How it was checked

Windows x64, Rust 1.98.1, commit b3cb080, based on dev@ff87e80:

• cargo fmt --all --check
• cargo clippy --locked --workspace --all-targets -- -D warnings
• cargo test --locked --workspace: 1,531 passed, 0 failed, 6 existing opt-in/live tests ignored. Database tests used disposable PostgreSQL 16/pgvector with UTOPIA_TEST_REQUIRE_DB=1 and four test threads.
• Targeted document regressions: 3 passed, exercising the Chat parameter guard/shared dispatcher and the authenticated MCP handler, including two-call tail retrieval.
• pnpm --dir web build
• pnpm --dir web test: 36 files, 268 tests passed.

Before review

• Every commit is signed off.
• Format, workspace Clippy, and workspace tests pass.
• Web build and tests pass.

[WaylandYang]

Thank you, this is exactly the fix: an optional offset over the body text, the next offset and document id in the notice, and guidance that a partial read is not proof of absence. I ran the document tests on a fresh database; the two-call tail read passes.

Merging. Thanks again.

[WaylandYang]

It is still marked as a draft, so it cannot be merged yet. Mark it ready for review when you are and we will merge it. Thanks!

[Floating-Y]

Thanks for reviewing and running the tests! The PR is now ready for review.

[WaylandYang]

Thank you for marking it ready. This closes #1209 as scoped there: one optional offset over the body text, the next offset and document id in the notice, and tests that read the tail in two calls over both Chat and MCP. Merging.