Problem
The IntelliJ Test Runner view currently mixes two different concepts:
- Specification items are the logical trace results users expect to count.
- Trace links are useful detail nodes beneath those results.
This produces an inflated total and, for clean traces with linked items, a green progress bar that can remain below 100%.
Current tree and counting
A source file is a suite. A specification item becomes expandable when it has link children, and each displayed trace link is currently treated as an additional test.
doc/requirements.md suite — not counted
└─ User can log in specification item — counted
└─ ⊙← Login test trace link — counted
src/LoginTest.java suite — not counted
└─ Login test specification item — counted
└─ ⊙→ User can log in trace link — counted
For this one requirement-to-test relationship, the current total is 4:
2 specification items
+ 2 displayed link children (the same relationship is visible from both ends)
= 4 reported tests
The current implementation calculates each item as 1 + links.size(), then reports that value to the IntelliJ result form.
There is an additional lifecycle mismatch:
linked specification item → suite-started / suite-finished
trace link → test-started / test-finished
The IntelliJ runner does not increment completed-test progress for onSuiteFinished(...). Therefore a clean, expandable specification-item node raises the expected count but may not raise the completed count, leaving the green bar below 100%.
Desired tree and counting
Keep the useful visual hierarchy exactly as it is, but count only specification items.
doc/requirements.md suite — not counted
└─ User can log in specification item — counted
└─ ⊙← Login test trace-link detail — not counted
src/LoginTest.java suite — not counted
└─ Login test specification item — counted
└─ ⊙→ User can log in trace-link detail — not counted
Expected total for the example: 2 tests, and a clean trace completes at 100%.
Proposed approach
Use the IntelliJ SM runner's custom-progress support for the logical result count:
- Set custom total to the number of visible specification items.
- For each specification item, report custom started, failed (when the item itself or one of its visible links is defective), and finished.
- Retain the existing normal SM events for file suites, expandable item nodes, and link child nodes, so the tree, navigation, details, and failure presentation remain unchanged.
While a custom-progress category is active, IntelliJ ignores normal test-counter events. This separates item-only progress/counting from rendering the link detail nodes.
Acceptance criteria
- The tree still shows source-file suites, specification-item entries, and trace-link child entries as today.
- The Test Runner total equals the number of visible specification-item entries only.
- Trace-link children do not increase the displayed total.
- A clean trace with linked items shows 100% completion.
- An item with a defective trace link contributes one failed logical result (the owning specification item).
- Existing navigation and details for suites, items, and links remain available.
Investigation evidence
OftTraceTestTree.OftTraceItemNode#testCount() currently returns 1 + links.size().
OftTraceTestRunnerOutputPresenter#showTrace(...) sends that count via onTestsCountInSuite(...).
- Linked item nodes use suite lifecycle events; link nodes use test lifecycle events.
- IntelliJ's
SMTestRunnerResultsForm#onSuiteFinished(...) does not update the finished-test counter; onTestFinished(...) does.
Problem
The IntelliJ Test Runner view currently mixes two different concepts:
This produces an inflated total and, for clean traces with linked items, a green progress bar that can remain below 100%.
Current tree and counting
A source file is a suite. A specification item becomes expandable when it has link children, and each displayed trace link is currently treated as an additional test.
For this one requirement-to-test relationship, the current total is 4:
The current implementation calculates each item as
1 + links.size(), then reports that value to the IntelliJ result form.There is an additional lifecycle mismatch:
The IntelliJ runner does not increment completed-test progress for
onSuiteFinished(...). Therefore a clean, expandable specification-item node raises the expected count but may not raise the completed count, leaving the green bar below 100%.Desired tree and counting
Keep the useful visual hierarchy exactly as it is, but count only specification items.
Expected total for the example: 2 tests, and a clean trace completes at 100%.
Proposed approach
Use the IntelliJ SM runner's custom-progress support for the logical result count:
While a custom-progress category is active, IntelliJ ignores normal test-counter events. This separates item-only progress/counting from rendering the link detail nodes.
Acceptance criteria
Investigation evidence
OftTraceTestTree.OftTraceItemNode#testCount()currently returns1 + links.size().OftTraceTestRunnerOutputPresenter#showTrace(...)sends that count viaonTestsCountInSuite(...).SMTestRunnerResultsForm#onSuiteFinished(...)does not update the finished-test counter;onTestFinished(...)does.