fix(parser): detect bulleted headings and other block constructs correctly

[tstapler]

Summary

• Fixes a structural bug where an outline bullet's content decorated with a Markdown block construct (fenced code, blockquote, ordered list item, thematic break, or GFM table) fell through to plain bullet/paragraph parsing and rendered as literal Markdown text — the same bug class already found and fixed for ATX headings.
• Extracts the previously inline, top-level-only detection logic in BlockParser.parseBlock() into shared tryConsumeNonHeadingConstruct() / parseFencedCodeBlock() helpers, and calls them both before and after bullet-token consumption (mirroring the existing tryConsumeAtxHeadingMarker() pattern used for headings).
• Threads indentLevel through HeadingBlockNode, and now also through CodeFenceBlockNode, BlockquoteBlockNode, ThematicBreakBlockNode, and TableBlockNode, so every bullet-decorated construct retains its true outline nesting depth end-to-end.
• Follow-up fix: closes a data-loss regression in the construct-classification fix above — bulleted fenced code/blockquote/thematic-break/table constructs were returning immediately and silently dropping any nested outline children or trailing key:: value properties, reparenting them to the grandparent level instead. Independently flagged by three parallel adversarial review agents run against this diff.
• Second follow-up: closes 4 remaining MINOR/out-of-scope items flagged during review of the fixes above — (1) MarkdownParser.convertBlock() no longer hardcodes level = 0 for CodeFenceBlockNode/BlockquoteBlockNode/ThematicBreakBlockNode/TableBlockNode, reading block.indentLevel instead; (2) parseBlockquote's previously-unused _level parameter is now wired through to BlockquoteBlockNode.indentLevel instead of being dead code; (3) raw HTML blocks (CommonMark §4.6) — previously an entirely unimplemented BlockType/AST variant that was never constructed by the parser and fell through to plain paragraph/bullet text — are now detected and parsed, both top-level and bullet-decorated.

Context

A user reported (via screenshot) that a SteleKit page rendered several lines with literal # characters instead of styled headings. Root cause: BlockParser.parseBlock() checked for the ATX heading marker only before bullet-token consumption, never after, so - # Foo fell through to plain-text bullet parsing. That fix was already implemented; this PR extends the audit (project_plans/markdown-rendering-gaps/requirements.md) to the remaining block constructs that shared the same structural gap, and fixes each one found.

ui/components/BlockItem.kt's dispatch and MarkdownParser.convertBlock's AST→BlockType mapping were both audited and confirmed to already have full parity — dispatch is an exhaustive, compiler-enforced when over the sealed BlockType/BlockNode hierarchies, so no gap exists there.

After the initial classification fix landed, three parallel adversarial review agents (correctness, domain-semantics, test-quality) independently converged on the same BLOCKER: tryConsumeNonHeadingConstruct() returned each matched construct node immediately, bypassing parseBlock's shared step 3 (property parsing + parseBlocksAtLevel(level + 1) children collection). Since ParsedBlock.children — not .level — is what actually drives DB parent/child structure (confirmed via MarkdownPageParser.processParsedBlocks), this meant a nested bullet under e.g. - ```kotlin ... ``` was silently reparented to the grandparent, and any :: property line following such a construct was mis-parsed as literal paragraph text. This is fixed by a new parseTrailingPropertiesAndChildren() helper, applied uniformly to all five non-heading constructs (also closing a pre-existing gap where ordered list items never parsed trailing properties).

A subsequent review of that fix flagged 4 remaining gaps as MINOR/out-of-scope at the time, tracked and now closed in this PR:

1. CodeFenceBlockNode, BlockquoteBlockNode, ThematicBreakBlockNode, and TableBlockNode had no field to carry outline nesting depth, so MarkdownParser.convertBlock() hardcoded level = 0 for all four regardless of where they actually sat in the outline.
2. That hardcoding in MarkdownParser.kt is fixed by reading the new indentLevel field instead.
3. parseBlockquote's _level parameter was accepted but never used (dead parameter) — now wired through to BlockquoteBlockNode.indentLevel.
4. RawHtmlBlockNode was fully wired through OutlinerParser, MarkdownParser, and BlockTypeMapper, but BlockParser never constructed one — literal HTML in a page (CommonMark §4.6) fell through entirely to plain paragraph/bullet parsing, the same bug class as the original ATX-heading gap. Root cause fixed by adding detection for block-level HTML tags and comments using the existing L_ANGLE/EXCLAMATION/TEXT token primitives, at both the top level and after bullet-token consumption, plus a contentString case in MarkdownParser so raw HTML round-trips instead of persisting as empty content.

Changes

• parsing/BlockParser.kt: Extracted fenced-code/blockquote/ordered-list/thematic-break/table detection into tryConsumeNonHeadingConstruct() and parseFencedCodeBlock(); invoked from both the pre-bullet (top-level) and post-bullet-consumption call sites. Added parseTrailingPropertiesAndChildren(), called after each of the five constructs, to collect trailing properties and nested outline children (mirroring the existing OrderedListItemBlockNode / heading handling). BlockquoteBlockNode's children field — already used to store the quote's own continuation paragraphs — now has outline children appended after them rather than losing one or the other. tryConsumeNonHeadingConstruct() now takes isBulletDecorated and computes indentLevel = if (isBulletDecorated) level else 0, threading it into CodeFenceBlockNode/BlockquoteBlockNode/ThematicBreakBlockNode/TableBlockNode. parseBlockquote's indentLevel parameter (renamed from the unused _level) is passed straight into BlockquoteBlockNode.indentLevel. Added raw-HTML-block detection (block-level tag list + HTML comment lookahead via peekToken) that constructs RawHtmlBlockNode, reusing parseLine() for verbatim capture and parseTrailingPropertiesAndChildren() for properties/children, consistent with the other five constructs.
• parsing/ast/BlockNodes.kt: Added indentLevel to HeadingBlockNode, CodeFenceBlockNode, BlockquoteBlockNode, ThematicBreakBlockNode, TableBlockNode, and RawHtmlBlockNode.
• parser/MarkdownParser.kt: Uses block.indentLevel when converting HeadingBlockNode/CodeFenceBlockNode/BlockquoteBlockNode/ThematicBreakBlockNode/TableBlockNode/RawHtmlBlockNode to ParsedBlock.level. Added a contentString case for RawHtmlBlockNode (block.rawHtml) so raw HTML content round-trips instead of serializing as an empty string.
• Tests: BlockParserTest.kt (4 heading regression tests); BlockConstructsSpec.kt — 5 classification tests for bulleted constructs, 12 tests covering nested children/trailing properties/sibling resumption/boundary cases for the bulleted forms, 8 new tests covering indentLevel propagation (top-level vs. bulleted) for the four newly-tracked constructs, and 11 new tests covering raw HTML block detection, verbatim capture, comments, closing-tag-only forms, bullet decoration, indentLevel, nested children, trailing properties, a false-positive boundary case, and an end-to-end MarkdownParser.parsePage round-trip.

Impact

• Scope: Block-level Markdown parsing only (BlockParser), read-mode rendering. No changes to inline parsing, the editor, or UI dispatch.
• Breaking Changes: None — all new fields are additive with default values (indentLevel: Int = 0); no fields removed.
• Performance: Neutral — same token-scanning logic, shared between two call sites; property/children collection is bounded by the existing parseBlocksAtLevel recursion, not a new pass over the document. Raw HTML detection only activates on L_ANGLE, a single extra branch check on an already-consumed token stream.
• Dependencies: None new.

Testing

• ./gradlew jvmTest — full suite, BUILD SUCCESSFUL, zero failures (confirmed via grep -c "FAILED" returning 0), includes all new regression tests
• ./gradlew jvmTest --tests "dev.stapler.stelekit.parsing.BlockConstructsSpec" — targeted run, all new and existing tests pass (one pre-existing @Ignored test correctly skipped)
• ./gradlew detekt --no-configuration-cache — BUILD SUCCESSFUL, no findings
• ./gradlew ciCheck — not run as-is in this worktree: androidApp:compileDebugJavaWithJavac requires ANDROID_HOME/local.properties (not configured here), and detekt alone hits an unrelated Gradle configuration-cache task-graph validation issue (generateDemoFileSystem output used without a declared dependency) that reproduces with configuration cache enabled but not with --no-configuration-cache — worked around accordingly. Neither issue is caused by this diff.

Reviewer Notes

• Focus areas: parseTrailingPropertiesAndChildren() in BlockParser.kt — it speculatively consumes a leading INDENT token to try tryParseProperty(), then restores the pre-INDENT lexer state via saveState()/restoreState() if the line isn't a property, so parseBlocksAtLevel sees the correct indent level for non-property continuation content. Also confirm BlockquoteBlockNode.children's dual use (quote continuation paragraphs + appended outline children) reads clearly at the call site. For the raw HTML addition: closing tags (</div>) lex as a single TEXT token "/div" rather than a separate / token, since / is not a special character in Lexer.isSpecial() — tag-name extraction strips the leading / before matching against BLOCK_HTML_TAGS.
• Known limitations: Multi-line bullet-decorated blockquotes/tables with indented continuation lines (as opposed to trailing properties/children) are not specially merged into the construct's own body — this is a pre-existing limitation consistent with existing top-level behavior, not a regression, and out of scope for this PR. Raw HTML block detection is intentionally single-line-scoped (via parseLine()), matching this codebase's existing simplified precedent for ThematicBreakBlockNode/OrderedListItemBlockNode, rather than attempting full CommonMark multi-line blank-line-terminated semantics, which don't map naturally onto Logseq's outliner model.
• Follow-up tasks: None identified.

🤖 Generated with Claude Code

[github-actions]

JVM Load Benchmark (Desktop)

Synthetic in-memory benchmark measuring load performance for the desktop (JVM) app.
Comparing 2d8b5e6f (this PR) vs 7abb2daf (baseline)
Graph config: xlarge — 230 pages

Metric	This PR	Baseline	Delta
Phase 1 TTI ↓	1ms	1ms	0 (0%)
Phase 2 background ↓	0ms	0ms	0 (0%)
Phase 3 index ↓	1ms	1ms	0 (0%)
Total ↓	2ms	2ms	0 (0%)
Write p95 (baseline) ↓	16ms	20ms	-4ms (-20%) ✅
Write p95 (under load) ↓	n/a	n/a
Jank factor ↓	n/a	n/a
↓ lower is better

Flamegraphs (this PR)

**Allocation** — object allocation pressure (JDBC/SQLite churn)

Alloc flamegraph not available

CPU — method-level hotspots by on-CPU time

CPU flamegraph not available

Top allocation hotspots (this PR)

`35.5%` byte[]_[k] `7.4%` java.lang.String_[k] `6.8%` java.util.LinkedHashMap$Entry_[k] `6.4%` int[]_[k] `3.7%` java.lang.Object[]_[k]

Top CPU hotspots (this PR)

`97%` /usr/lib/x86_64-linux-gnu/libc.so.6 `0.9%` clock_nanosleep `0.6%` /tmp/sqlite-3.51.3.0-4c4ce51d-7a9c-4c34-922e-4662832313dd-libsqlitejdbc.so `0.4%` fsync `0.2%` __libc_pwrite

[github-actions]

Android Load Benchmark

Instrumented benchmark on an API 30 x86_64 emulator — 500-page synthetic graph.

Comparing 2d8b5e6f (this PR) vs 7abb2daf (baseline)
Device: API 30 x86_64 emulator — 530 pages loaded

Graph Load

Metric	This PR	Baseline	Delta
Phase 1 TTI ↓	51ms	46ms	+5ms (+11%) ⚠️
Phase 3 index ↓	4708ms	4198ms	+510ms (+12%) ⚠️

Interactive Write Latency (during Phase 3)

Metric	This PR	Baseline	Delta
Write p95 (baseline) ↓	12ms	8ms	+4ms (+50%) ⚠️
Write p95 (during phase 3) ↓	13ms	15ms	-2ms (-13%) ✅
Jank factor ↓	1.08x	1.88x	-0.8x (-43%) ✅
Concurrent writes ↑	24	21	+3ms (+14%) ✅

SAF I/O Overhead (ContentProvider vs direct File read)

Measures Binder IPC cost added by ContentResolver per readFile() call.
Real SAF via ExternalStorageProvider will be higher on device; this is a lower bound.

Metric	This PR	Baseline	Delta
Direct read / file ↓	0.0ms	0.0ms	0 (0%)
Provider read / file ↓	0.2ms	0.6ms	0ms (-68%) ✅
IPC overhead ratio ↓	6x	19x	-13x (-68%) ✅
↓ lower is better · ↑ higher is better