Skip to content

Add an evidence-backed “Understand the change” report tab - #29

Merged
kentwelcome merged 1 commit into
mainfrom
feat/report-change-explanation
Oct 6, 2026
Merged

kentwelcome merged 1 commit into
mainfrom
feat/report-change-explanation

Conversation

@kentwelcome

Copy link
Copy Markdown
Contributor

Summary

Add Understand the change, an evidence-backed explanation layer between the concise Summary and detailed comparisons.

  • Keep all three Summary sections, including What this means.
  • Place the Understand the change button under What the evidence shows.
  • Open a separate top-level explanation tab; retain separate Behavior diff, Flow diff, Instruction changes, and Trial evidence tabs.
  • Explain the distinction through annotated Before/After steps, practical meanings, unchanged behavior, limitations, full cited distributions including minority branches, and attributed source excerpts.
  • Share explanation content across HTML and Markdown.

Evidence contract

The existing extraction call can supply a bounded, structured explanation. Validate comparison references, unchanged distributions, named source trials, and exact final-answer excerpts. Invalid optional explanations are omitted without discarding valid behavior comparisons. Rendering never calls a model.

Canonical report-data.json moves from schema 8 to 9 and explicitly records explanation availability. Saved run evidence without an explanation still renders with a transparent unavailable state and links to retained evidence; existing reports do not silently acquire new analysis.

All added fixtures are synthetic. No private prototype reports or real trial transcripts are committed.

Verification

  • bash tests/hooks-test.sh — passed.
  • python3 plugin/skills/behavior-diff/scripts/decisions.py --check — passed.
  • bash tests/live-report-contract.sh — passed, including schema/explanation regression coverage.
  • bash tests/release-workflow-test.sh — passed.
  • Docker shfmt v3.14.0 check — passed.
  • uvx ruff@0.16.5 format --check --diff . — passed.
  • git diff --check — passed.
  • Actual renderer/browser smoke: timing and mixed formula/writing cases, Summary button placement, explanation reload, source-excerpt deep links, separate evidence tabs, and 390px layout.
  • Read-only load/render of an older saved case confirmed honest unavailable explanation, without re-extraction or modifying source evidence.

No live model execution was performed; synthetic rendering checks do not establish extractor adherence or improved reader comprehension.

Tracking

Closes https://linear.app/recce/issue/DRC-4810/add-evidence-backed-understand-the-change-report-tab

Independent review

  • Specification review: APPROVE.
  • Standards review: APPROVE after fixing a completeness-guard finding. Both HTML and Markdown now withhold stored conclusions when trial evidence is incomplete, blocked, invalid, or dropped; a failed-before/passed-after regression and browser smoke cover that boundary.

Signed-off-by: Kent Huang <kent@infuseai.io>
@kentwelcome
kentwelcome merged commit 89cc813 into main Oct 6, 2026
2 checks passed
@kentwelcome
kentwelcome deleted the feat/report-change-explanation branch October 6, 2026 07:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant