feat: discover NDJSON logs via message extraction

[STRRL]

Problem

The discovery pipeline treats every log file as plain text. Structured
JSON-lines logs get fed to Drain as raw JSON strings, which produces dirty,
low-value templates, and there is no way to keep the structured form for
investigation. ADR 0006 decided NDJSON storage with message extraction.

Solution

Per-file automatic format detection: NDJSON files are mined through an
extracted message line while pattern samples keep the raw structured entries.
Plain-text files flow through the existing path unchanged.

Major Changes

• pkg/ndjson (new)
  • pure functions DetectFormat (a file is NDJSON only when every non-empty
    line is a JSON object) and Extract (envelope {ts, severity, payload}
    extracts from the payload; any other shape is extracted as a whole; message
    from message/msg/log/error, severity prefix from string
    severity/level, compact-JSON fallback)
  • fixture-driven table tests: committed input + expected-output pairs, glob-based, so
    supporting a new shape means adding a fixture pair
• pkg/workspace
  • pipeline routes each file by detected format; TaggedLine carries an
    optional extracted line that Drain and template matching consume, while
    samples and unmatched output keep the raw NDJSON lines
  • mixed-workspace pipeline test (text + NDJSON) with an injected labeler,
    asserting raw-line preservation for both matched and unmatched entries

---

Note

Medium Risk
Changes core discovery input shaping and pattern matching for NDJSON files; behavior is well-tested but misclassification or extraction edge cases could shift templates for structured logs.

Overview
Discovery no longer treats every log file as plain text. Per-file format detection routes pure NDJSON (every non-empty line is a JSON object) through a new extraction path; mixed or plain files stay on the existing multiline text pipeline.

A new pkg/ndjson package implements DetectFormat and Extract per ADR 0006: importer envelope {ts, severity, payload} mines from the payload; other shapes use message/msg/log/error and string severity/level, with compact JSON fallback. Docs rename projection → extraction in CONTEXT and ADR 0006.

TaggedLine gains optional ExtractedLine and DrainLine() so Drain, labeling samples, and template matching use extracted text while samples.log and unmatched output keep raw JSON lines. Builder template matching switches from Content to DrainLine(). Fixture-driven unit tests plus a mixed text/NDJSON Discover integration test cover the behavior.

^{Reviewed by Cursor Bugbot for commit 5bc9bd6. Bugbot is set up for automated code reviews on this repo. Configure here.}