Support old and new Piebald database schemas

[mike1858]

Why

Piebald's typed-message-parts migration moves model, token, and generation configuration fields out of messages and removes message_part_tool_call. Splitrail still queries those legacy locations, so updated databases fail with no such column: m.model and their usage cannot be parsed.

Support the new layout without requiring users on older Piebald versions to migrate.

What changed

• Detect finalized typed-parts storage using message_generations and removal of message_part_tool_call. Keep legacy reads during the committed additive migration state, before tool backfill finishes.
• Read normalized generation metadata and service-tier configuration while preserving message timestamps, deduplication identities, and chat-model fallback.
• Count typed tool calls through tool_execution_context and exclude context-container messages.
• Keep schema detection and all queries in one read-only transaction.
• Add five on-disk fixture tests covering legacy, additive, and finalized layouts, equivalent normalized output, and malformed-schema errors.

Validation

• cargo build --quiet — passed.
• cargo test --quiet — 451 passed.
• cargo clippy --quiet -- -D warnings — passed.
• cargo doc --quiet — passed.
• cargo fmt --all --quiet — passed.
• git diff --check — passed.

CLI before/after

Ran /usr/local/bin/splitrail stats --include-messages (released 3.9.0) and the PR debug binary with stats --include-messages against identical synthetic SQLite histories. Each layout used a temporary HOME and isolated XDG_DATA_HOME, XDG_CONFIG_HOME, and XDG_CACHE_HOME; no real usage database or cloud upload was involved. The harness command was python3 /tmp/splitrail-8833-cli-proof.py.

legacy / release-3.9.0: messages=1 input_tokens=900 tool_calls=1
legacy / PR: messages=1 input_tokens=900 tool_calls=1
additive / release-3.9.0: messages=1 input_tokens=900 tool_calls=1
additive / PR: messages=1 input_tokens=900 tool_calls=1
typed / release-3.9.0: messages=0 (Piebald source could not be parsed)
typed / PR: messages=1 input_tokens=900 tool_calls=1

Summary by CodeRabbit

• Bug Fixes
  • Added support for reading both legacy and typed-message-parts Piebald database formats.
  • Improved message and tool-call usage parsing, including generation metadata, service tiers, identities, and zero-usage records.
  • Ensured database reads use a consistent snapshot for reliable results.
  • Added clearer errors for invalid typed database schemas.
  • Improved reliability during in-progress schema migrations by preserving accurate legacy usage and tool-call data.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Team

Run ID: 84f8a004-53c9-473b-af2a-2e506b7c3d69

📥 Commits

Reviewing files that changed from the base of the PR and between c96bbff and d9e823d.

📒 Files selected for processing (2)

• src/analyzers/piebald.rs
• src/analyzers/piebald_schema_tests.rs

Included review availability: Your plan provides up to 2 included reviews per hour; 0 remain after this review.

---

📝 Walkthrough

Walkthrough

The Piebald parser now supports legacy and typed-message-parts schemas. It selects schema-specific queries, reads all data from one SQLite transaction snapshot, and adds fixture-based tests for usage, tools, normalization, migration behavior, and typed-schema errors.

Changes

Piebald schema support

Layer / File(s)	Summary
Schema-specific message and tool queries src/analyzers/piebald.rs	The parser detects supported Piebald schemas and selects schema-specific queries for messages, generation metadata, service tiers, and tool calls.
Consistent transactional parsing src/analyzers/piebald.rs	The parser runs schema detection and chat, message, and tool-call reads within one deferred SQLite transaction snapshot.
Legacy and typed schema validation src/analyzers/piebald_schema_tests.rs, src/analyzers/piebald.rs	Fixtures and assertions validate both schemas, normalized history, usage, provider-tier precedence, identity, tool calls, zero-usage rows, migration behavior, and typed-schema errors.

Estimated code review effort: 4 (Complex) | ~45 minutes

Merge Risk: ⚪ Minimal · up to d9e82

This adds schema-aware Piebald parsing while preserving legacy reads during incomplete migrations, with fixtures covering both layouts, migration behavior, normalized output, and malformed typed schemas. No merge-blocking risk is identified.

Sequence Diagram(s)

sequenceDiagram
  participant PiebaldParser
  participant SQLite
  participant PiebaldSchema
  PiebaldParser->>SQLite: begin deferred read transaction
  PiebaldParser->>PiebaldSchema: detect database schema
  PiebaldSchema->>SQLite: run schema-specific queries
  SQLite-->>PiebaldParser: return chats, messages, and tool calls
  PiebaldParser->>SQLite: commit transaction

Loading

Poem

I’m a rabbit with queries aligned,
Through legacy and typed paths I wind.
Tokens hop, tool calls play,
One snapshot keeps them in stay,
And tests guard each schema we find.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 78.57% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 14 functions across 2 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change: support for both old and new Piebald database schemas.

• Fix all pre-merge checks with AI

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch support-old-and-new-piebald-database-schemas

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}