From cbb0c5ddfdd156b69193f822cd76d06beb5481fe Mon Sep 17 00:00:00 2001
From: Valentino Stoll <v@codenamev.com>
Date: Thu, 16 Jul 2026 17:47:33 -0400
Subject: [PATCH] [Fix] Skip signal-health benchmark when the dogfood DB is
 absent, not just an LFS pointer
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

CI (`.github/workflows/main.yml` -> `./bin/run-evals`) has been red since
ae077b7 [Chore] Untrack machine-local .claude state. Root cause:

- The signal-health benchmark (spec/benchmarks/health/database_signal_spec.rb)
  runs against the live committed .claude/memory.sqlite3 and its before(:all)
  guard skipped the suite only when that file EXISTED but wasn't a SQLite file
  (an unresolved git-lfs pointer on CI's lfs:false checkout).
- ae077b7 untracked the DB, so on CI it is now ABSENT entirely. File.exist?
  is false, the guard is bypassed, and the suite runs against a fresh empty DB
  created by ensure_project! — where "has more than 5 active project facts
  (sanity floor)" fails with "only 0 active project facts; suspect ingest
  broken or DB nuked". `bundle exec rspec spec/benchmarks/ --tag benchmark`
  then exits 1, so bin/run-evals reports SOME CHECKS FAILED.

Fix: skip the suite unless the DB is present AND a real SQLite file, covering
both the absent (untracked/gitignored/fresh-clone) and lfs-pointer cases. These
contracts only mean something against the live DB; they still run locally.

Reproduced with CLAUDE_PROJECT_DIR/CLAUDE_CONFIG_DIR pointed at empty dirs
(1 failure -> 10 pending after the fix); LFS-pointer case also skips; a present
live DB still runs.
---
 .../benchmarks/health/database_signal_spec.rb | 31 ++++++++++---------
 1 file changed, 17 insertions(+), 14 deletions(-)

diff --git a/spec/benchmarks/health/database_signal_spec.rb b/spec/benchmarks/health/database_signal_spec.rb
index 4e07045..2a24f44 100644
--- a/spec/benchmarks/health/database_signal_spec.rb
+++ b/spec/benchmarks/health/database_signal_spec.rb
@@ -23,22 +23,25 @@
 require_relative "../benchmark_helper"
 
 RSpec.describe "memory database signal health", :benchmark do
-  # Skip the entire suite when the committed `.claude/memory.sqlite3` is an
-  # unresolved git-lfs pointer (CI checkout without LFS, an LFS bandwidth
-  # cap, or a fresh clone before `git lfs pull`). Reading the pointer text
-  # as a SQLite file raises `Extralite::Error: file is not a database`,
-  # which would fail benchmarks on unrelated changes — and the project's
-  # LFS blobs aren't reliably present on the GitHub LFS server, so CI runs
-  # always see the pointer. The signal-health contracts only mean something
-  # against the real committed DB; run locally after `git lfs pull` to
-  # validate them before tagging a release (see docs/api_stability.md §7).
+  # Skip the entire suite unless the real committed `.claude/memory.sqlite3`
+  # is present as an actual SQLite file. Two cases make it unavailable:
+  #   1. Absent — the dogfood DB is gitignored/untracked as of the
+  #      2026-06-27 "untrack machine-local state" change (ae077b7), so CI
+  #      and fresh clones simply don't have the file.
+  #   2. An unresolved git-lfs pointer — older checkouts LFS-tracked the DB,
+  #      and the project's LFS blobs aren't reliably on the GitHub LFS
+  #      server, so LFS-less CI checkouts see the pointer text instead.
+  # In both cases the signal-health contracts have nothing real to assert
+  # against; running them would fail on unrelated changes (an empty DB trips
+  # the "≥ 5 active facts" sanity floor; a pointer raises "file is not a
+  # database"). These contracts only mean something against the live DB —
+  # run locally to validate them before tagging a release (docs/api_stability.md §7).
   before(:all) do
     project_db = ClaudeMemory::Configuration.new.project_db_path
-    if File.exist?(project_db)
-      header = File.binread(project_db, 16).to_s
-      unless header.start_with?("SQLite format 3")
-        skip "skipping: #{project_db} is not a SQLite file (likely an unresolved git-lfs pointer). Run `git lfs pull` locally to validate signal contracts."
-      end
+    live_db = File.exist?(project_db) &&
+      File.binread(project_db, 16).to_s.start_with?("SQLite format 3")
+    unless live_db
+      skip "skipping: #{project_db} is absent or not a SQLite file (dogfood DB is untracked/gitignored, or an unresolved git-lfs pointer). Run locally against the live DB to validate signal contracts."
     end
   end