Recover journal prefixes and align Rust/Go journalctl with v262

.agents/skills/project-release/SKILL.md

@@ -65,7 +65,9 @@ actions already authorized. Loading this skill does not start a release.
    The source can precede the workflow's merge commit. Ordinary PRs/pushes do
    not publish, and later master changes do not change the selected source.
 3. Require successful source checks and both full language suites at the
-   source-declared compiler minimums, with CGO disabled for Go.
+   source-declared compiler minimums, with CGO disabled for the Go SDK and
+   a separate CGO-enabled Go CLI test. Command-inclusive suites require PCRE2-8
+   development files and pkg-config; SDK publication does not depend on PCRE2.
 4. Require all eight Rust crates to pass in dependency order: **common →
    registry → core → host → log-writer → index → engine → public SDK**. For
    each new version, CI dry-runs before obtaining temporary publishing

.agents/sow/SOW-status.md

@@ -1,13 +1,31 @@
 # SOW Status
 
-Last updated: 2026-10-11
+Last updated: 2026-10-26
 
 ## Current
 
-- None.
+- SOW-0045 - Go Reader Alignment Optimization: regression reopened and paused.
+  Unchanged HEAD reproduces skipped appends after healthy empty/EOF refresh,
+  including unique-triggered refresh. Separate positioning work remains queued;
+  a pre-existing directory realtime-seek/getter gap and unfiltered time-probe
+  error suppression are also recorded for that analysis. SOW-0108 is complete;
+  SDK-then-CLI execution remains unchanged.
+
+- SOW-0071 - Runtime Purity And Optional Platform Services: regression
+  reopened and paused. The static guard misses standard Go process-launch
+  spellings despite its original acceptance promise. No SDK runtime launch
+  is demonstrated; scanner design and implementation remain unapproved.
+  Tracking adds no prerequisite or change to the 0108 -> 0159 -> 0121 order.
 
 ## Pending
 
+- SOW-0161 - File Indexer Source Posting Membership: open source-analysis
+  lead, not activated. Investigate inherited source-map target membership and
+  captured-tail handling before selecting any behavior/performance change.
+- SOW-0160 - Go XZ Reader Dictionary Allocation: open, not activated. Tracks
+  unchanged cumulative dictionary allocation measured during SOW-0108, separate
+  from required regression prevention. No decoder redesign is authorized and
+  the approved current execution order is unchanged.
 - SOW-0156 - Fixed-Header Consistency Audit: separate follow-up agreed in
   PR #9. Define invariants, audit both SDKs, then implement/test missing rules.
   Not activated by the current array-count repair.
@@ -40,9 +58,10 @@ Last updated: 2026-10-11
   on 2026-06-21. Still valid as Rust quality debt, but live-path work is
   high-risk and not mandatory before integration/release work.
 - SOW-0122 - Journalctl Performance Followups: open. Tracks non-blocking
-  SOW-0121 performance follow-up for indexed `--list-invocations`, portable
+  SOW-0121 broader performance follow-up for large invocation inputs, portable
   `--follow` scalability, and large-fixture performance evidence before any
-  large-archive performance claim.
+  large-archive performance claim. Mandatory no-regression proof for the new
+  SDK/journalctl repairs belongs directly to SOW-0159/SOW-0121.
 - SOW-0123 - Event-Time Lane Fresh-Journal Realtime Clock Seed: open. Records
   the Rust fresh-lazy realtime floor gap and Go parity/documentation gap for
   event-time-ordered journals. Project-manager diagnosis and implementation
@@ -55,6 +74,45 @@ Last updated: 2026-10-11
 
 ## Recently Closed Or Completed
 
+- SOW-0121 - File-Backed Journalctl Full Parity And Ship Decision: completed.
+  Directional cursor integration and PR12 fixes pass three internal reviews,
+  3,514 native comparisons and146 portable controls. Pushed-head native CI is
+  pending; historical qualification follows.
+  The vacuum regression is repaired under approved deterministic history-head
+  ordering. Both languages pass independent 10,000-set models; native CI passes
+  Linux/macOS/Windows/FreeBSD with 130 portable controls. Existing 78 utility
+  oracle observations pass. Final internal review passes the repair; lifecycle
+  impact review and normal final-commit CI complete publication. Upstream
+  Codacy configuration retry and coverage upload fail remotely; native tests,
+  coverage generation and CodeQL pass. These CI limitations stay explicit.
+  Broader scalability remains SOW-0122; PR #12 is draft, not merged/released.
+
+- SOW-0159 - Rust/Go SDK Hardening And Reader Compatibility: completed.
+  SDK directional cursor APIs pass three independent reviews; CLI integration is
+  completed in SOW-0121. Latest SDK validation is in its closure record.
+  Both SDKs provide default intact-prefix recovery/status, complete streams,
+  native filtering and approved position/time/error retry behavior. Go floor
+  amd64/386/race checks pass 1,995/1,996/1,995 events, excluding only the unchanged
+  live-stock stress test with a retained isolated pass and earlier deadline
+  failure. Rust floor workspace/allocative checks pass, including 200 SDK tests.
+  Final tools pass 552 shared observations and 63/34/42 v262 verifier/directory/
+  mixed checks; 35 documentation examples and 16 pages pass. Performance is
+  accepted under representative evidence and careful review, with uncertain
+  intervals and adverse history retained. The fresh whole-SOW review has one
+  positive verdict, five coverage gaps and no verified unresolved blocker.
+  The complete-and-clean closing audit passes; signed commit 25accee keeps
+  implementation and completed lifecycle together. Journalctl SOW-0121 later reopened for CLI review findings; its vacuum repair remains accepted.
+- SOW-0108 - Cross-Language Reader Window Accessor Architecture: completed
+  again with the approved Go indexed aggregation lifetime/header/opener repair.
+  Full/race/386/integrated/floor and cross-platform checks pass. The current
+  118-case timing matrix records 79 improvements, 39 inconclusive results and
+  no regression flag; inconclusive results do not establish equivalence. All
+  1,176 measured allocation pairs avoid increases. External review is
+  adjudicated with no verified blocker; Mimo's P2 is disproven and Qwen/Kimi
+  coverage gaps remain explicit. Independent findings are owned by paused
+  SOW-0045/SOW-0071 and pending SOW-0160. Signed prerequisite commit `4c49205` precedes completed SDK
+  SOW-0159 and reopened CLI SOW-0121.
+
 - SOW-0158 - v0.10.0 Publication: completed. Release run 38123573886 passed
   both language suites and published/verified all eight Rust `0.10.0` crates.
   Annotated `v0.10.0` and `go/v0.10.0` peel to `288efe015c45`; fresh exact
@@ -131,11 +189,6 @@ Last updated: 2026-10-11
   `4619776b74311338b1689b2ab1fb0936b7746e49`; both release tags resolve to that
   commit, all eight Rust crates are published and non-yanked, and clean Go
   `1.26.2` and Rust `1.91.0` registry-consumer verification passes.
-- SOW-0121 - File-Backed Journalctl Full Parity And Ship Decision: completed
-  again after repairing its mixed-directory boot-row oracle. The harness now
-  strips only the stock table header, requires a complete stock oracle, and
-  compares every normalized boot row; the full-feature mixed matrix passed
-  42/42 and tagged-v260.1 directory/query validation passed.
 - SOW-0136 - Rust Directory Harness Explicit Identity: closed without
   implementation. Its evidence is merged into the user-approved regression
   reopening of originating SOW-0115.
@@ -406,14 +459,6 @@ Last updated: 2026-10-11
   O(1) indexed row collection. Local validation, four-peer high-row comparator
   evidence, `git diff --check`, and six production-grade reviewer votes passed.
   Remaining reviewer-discovered edge parity items are tracked by SOW-0109.
-- SOW-0108 - Cross-Language Reader Window Accessor Architecture: completed
-  after regression repair. Rust and Go public file-path verification APIs now
-  use bounded reader-backed byte sources instead of materializing whole journal
-  files; Go verifier source slices return owned buffers to avoid mmap/window
-  aliasing hazards. Focused Rust/Go tiny-window sealed verifier tests,
-  `go test ./...`, affected Rust package tests, the shared verify matrix,
-  wiki docs validation, `git diff --check`, and SOW audit passed. All six
-  reviewers ultimately voted `PRODUCTION GRADE` after the reopened repair.
 - SOW-0104 - Python Explorer And Netdata Parity To Rust: completed. The
   Python SDK now carries the full Rust feature surface: Explorer
   (filters, facets, histogram, FTS, Traversal/Index/Compare strategies,
@@ -647,14 +692,9 @@ Last updated: 2026-10-11
   and repo-local QEMU FreeBSD validation passed for Rust, Go, Python, and
   Node.js; FreeBSD no-stock single-file and directory matrices passed; files
   generated on macOS, Windows, and FreeBSD passed Linux stock
-  `journalctl --verify --file`; SOW-0071 and SOW-0072 blockers are completed.
-- SOW-0071 - Runtime Purity And Optional Platform Services: completed. Core
-  reader/writer paths in Rust, Go, Node.js, and Python no longer host-probe,
-  execute subprocesses, or acquire writer locks implicitly; identity discovery
-  and writer locks are optional helpers; legacy Rust `jf` host identity helpers
-  were removed; runtime-purity scans cover core, facade, Python I/O helper, and
-  legacy `jf` runtime files; Linux/macOS/Windows validation and three
-  whole-SOW reviewer rounds passed.
+  `journalctl --verify --file`; SOW-0071/SOW-0072 runtime dependencies were
+  resolved for that validation. SOW-0071 now separately tracks a paused
+  guard-coverage regression, with no demonstrated SDK runtime launch.
 - SOW-0067 - Go Cross Platform Portability: completed. Go SDK portability
   implementation, whole-SOW reviews, Linux/Windows tests, FreeBSD/macOS compile
   checks, and parent native macOS/Windows generated-file validation passed.
@@ -704,16 +744,6 @@ Last updated: 2026-10-11
   rows/s single-file and 628k rows/s open-files. Go tests, mixed-directory,
   cross-language, live regular/compact matrices, read-only reviewers, and audit
   passed.
-- SOW-0045 - Go Reader Alignment Optimization: completed. Go reader now has
-  mmap-backed Unix access by default, live/snapshot bounds, byte-preserving RAW
-  field APIs, current-entry payload visitor/enumerator APIs, libsystemd facade
-  DATA fast paths, non-overlapping directory fast-path coverage, and shared
-  reader benchmark integration. Compact 100k-row benchmark medians measured Go
-  single-file `sdk-payloads` live/mmap at 1.07M rows/s and `facade-data` at
-  1.09M rows/s versus stock systemd DATA enumeration at 565k rows/s; Go
-  open-files `sdk-payloads` measured 697k rows/s versus stock systemd open-files
-  DATA at 532k rows/s. Rust remains faster; the cursor-seek systemd divergence
-  discovered during review was closed by SOW-0055.
 - SOW-0054 - Node.js Reader And Writer Rust Port: completed. Node.js now carries
   the finalized reader/writer compatibility slice where practical under the
   no-native-runtime policy: byte-preserving RAW field access, active-file

.agents/sow/audit.sh

@@ -145,6 +145,8 @@ scan_sensitive_file() {
     # The canonical public Git SSH remote is not a personal email address.
     my $email_line = $line;
     $email_line =~ s{(?<!\S)git\@github\.com:netdata/systemd-journal-sdk\.git(?=\s|$)}{}g;
+    # This quoted all-zero archive fixture is a filename, not an address.
+    $email_line =~ s{"user\@0{16}-0{16}\.journal~"}{}g;
     if ($email_line =~ /\b[A-Z0-9._%+\-]+@[A-Z0-9.\-]+\.[A-Z]{2,}\b/i) {
       push @hits, "email-address" unless $line =~ /\b(example\.com|example\.org|example\.net|localhost)\b/i;
     }

.agents/sow/done/SOW-0045-20260528-go-reader-alignment-optimization.md → .agents/sow/current/SOW-0045-20260528-go-reader-alignment-optimization.md

@@ -2,13 +2,22 @@
 
 ## Status
 
-Status: completed
+Status: paused
 
 Sub-state: completed on 2026-05-29. Go reader now has Rust-aligned payload
 APIs where practical, mmap-backed Unix access, live/snapshot bounds,
 byte-preserving RAW field access, facade DATA fast paths, Go reader benchmark
 coverage, and recorded benchmark evidence.
 
+Regression reopened 2026-10-15: healthy Go refresh can skip the first newly
+appended entry when the reader was empty or at forward EOF. This SOW owns the
+original refresh implementation and live-reader claim. It is queued/paused
+under the one-active-SOW rule; prerequisite SOW-0108 and SDK SOW-0159 are
+completed, with journalctl SOW-0121 next after the signed SDK commit.
+Do not insert this separate positioning repair into the approved
+SOW-0108 -> SOW-0159 -> SOW-0121 execution chain without a dependency finding
+or a user scheduling decision. Original completion evidence remains below.
+
 ## Requirements
 
 ### Purpose
@@ -426,3 +435,107 @@ implementation.
 ## Regression Log
 
 None yet.
+
+## Regression - 2026-10-15
+
+### What Broke And Evidence
+
+Healthy Go iteration can skip the first append after an explicit Refresh or a
+unique-field query that refreshes the entry list. Two states reproduce it:
+open an empty valid journal, append one row, refresh and Step; or consume a
+one-row journal through EOF, append one row, refresh and Step. Both Step calls
+return false/nil despite the newly loaded row. QueryUnique triggers the same
+transition. A private diagnostic against unchanged HEAD `dafcdac4b971` reproduces
+all four combinations; retain `sow-0045-refresh-position-probe.log` and its
+synthetic test source under ignored task scratch.
+
+At HEAD, `go/journal/reader.go:606-609` leaves the empty constructor index at
+zero, successful refresh restores the old numeric index at `:561-562`, and
+Next increments it before selecting a row. At populated forward EOF the old
+index equals the old row count; refresh then makes that index refer to the
+first append, which the next increment skips. Automatic Step-driven tail
+refresh has a separate adjustment and does not establish that explicit refresh
+works. `go/journal/reader_unique.go` and directory unique-cache refresh call
+the same refresh path. Commit `6a48ccc5` introduced that preservation behavior
+in this SOW; the original execution/validation above claimed live refresh and
+live/snapshot parity.
+
+### Why Earlier Validation Missed It
+
+Existing live tests begin with a populated reader and use automatic tail
+refresh. Explicit-refresh retained-row tests refresh while positioned on a row
+and do not subsequently step from EOF. The new SOW-0108 header test initially
+assumed empty-open positioning; its intended header/array-growth coverage now
+explicitly SeekHead before appending. That adjustment does not repair or claim
+acceptance of this separate positioning defect.
+
+### Repair Plan And Validation
+
+Keep this queued until scheduled. Before code, specify position preservation
+for empty/head/unset, a current row, forward EOF, explicit tail, reverse
+movement, repeated/no-op refresh and unique-triggered refresh. A blanket
+old-length-minus-one adjustment is not an established complete repair: it can
+change reverse movement or expose a current row before a successful Step.
+Cover public Reader, facade and directory callers, regular/compact files,
+mmap/ReadAt, tiny/default windows, retained payloads, failed-refresh rollback
+and snapshot exclusion. Require appended rows exactly once and preserve native
+index paths and measured performance.
+
+Rust is not a confirmed passing oracle for this transition. Source analysis
+suggests a possible healthy-tail cached-population issue; this is an unexecuted
+working theory requiring a separate paired runtime check before any Rust
+finding or scope decision. SOW-0159 already handles its distinct empty-file and
+recovery contracts; do not silently broaden that SOW to this healthy-position
+state machine.
+
+### Artifact Impact And Follow-Up Mapping
+
+The originating SOW is reopened here rather than replaced by a new SOW.
+Canonical/root status indexes record its paused state, and SOW-0159 and
+SOW-0108 cross-reference this finding. No positioning runtime code is changed.
+Synthetic fixtures contain no real journal data. During repair, update the
+product live-reader contract, caller guidance and compatibility skill if the
+resolved behavior requires it; preserve existing compiler/purity constraints.
+Validation, review disposition and a complete-and-clean audit remain required
+before this regression can close. This is tracked work outside the active
+header/lifetime repair; no completion is claimed.
+
+### Related Queued Directory Seek-State Finding - 2026-10-16
+
+Read-only SOW-0159 impact analysis finds an independent pre-existing getter-state
+gap: `go/journal/directory_reader.go:739` clears realtime/candidate state during
+SeekRealtimeUsec but retains the selected file index. After a successful Step,
+SeekRealtimeUsec followed by GetEntry/GetCursor can therefore expose the old
+row before movement. Rust's `rust/src/journal/src/directory.rs:201-208` clears
+its index and its getters report NoEntry until movement succeeds. The Go
+omission exists in commit `fcca32fe56f4`; SOW-0027 claimed broad facade and seek
+support but did not specify getter state between seek and movement.
+
+Track this alongside the queued healthy-position analysis here; it is not a
+consequence of native filtering and adds no SDK-first prerequisite. Before any
+runtime change, reproduce both public APIs and facades, determine the affected
+seek/refresh state combinations and obtain any unresolved behavior decision.
+
+### Additional Unfiltered Time-Search Evidence - 2026-10-17
+
+SOW-0159's directory error-position controls find another inherited unfiltered
+Go seek gap. `reader.go:firstRealtimeIndexAtOrAfter` and
+`lastRealtimeIndexAtOrBefore` use a `sort.Search` predicate that treats an ENTRY
+read error as a comparison result and then returns a nil error. A backward
+merged-directory direction change can consequently skip the damaged candidate
+without reporting its ENTRY-read failure. The same source pattern exists in
+the frozen SOW-0159 baseline. This is separate from retaining the previous row
+after an error that is actually returned.
+
+Track propagation and retry of errors from unfiltered time probes here with
+the existing seek/refresh scope. The approved SOW-0159 binary-search change is
+for explicit filtered realtime positioning, and its ordinary unfiltered time
+policy remains unchanged. Its Go directory position regression matrix excludes
+the unfiltered backward merged case for that specific reason; filtered cases
+and sequential unfiltered file transitions remain covered. Before implementing
+this item, add a direct file and directory SDK/facade reproducer, compare Rust
+and the frozen baseline, resolve any remaining behavior choice, and measure
+the existing/new binary-search paths. This SOW remains paused while SOW-0159
+is the sole implementation SOW.
+No new position contract or implementation is approved by this tracking note.
+SOW-0159 retains its distinct reverse-filter and selected-entry error decisions.