From 2bbd07e426417ba43b8e2bdc612e35a160f753bd Mon Sep 17 00:00:00 2001 From: Daniel Bushman Date: Thu, 9 Jul 2026 22:22:16 +0000 Subject: [PATCH 1/2] =?UTF-8?q?docs(maproom):=20Postgres=20backend=20accur?= =?UTF-8?q?acy=20pass=20=E2=80=94=20plugin=200.10.2?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - concept-glossary.md: expand SQLite entry to cover both backends; describe shared-Postgres via MAPROOM_DATABASE_URL, note localhost:5433 as cargo-test-only sandbox, maproom status as canonical probe - troubleshooting.md GAP-004/005/006: add "SQLite backend only" banners; add Postgres Connectivity Issues section (health check, connection string, common error table, FTS-vs-embeddings note) - embedding-providers.md + adc-setup.md: add status banners noting embeddings not generated by default in shared-PG; FTS operative; docs remain valid when embeddings are enabled - README.md L42 + L114: reframe database-location as backend-agnostic (MAPROOM_DATABASE_URL / maproom status); retain SQLite path for SQLite users - maproom-researcher.md: version floor 0.1.0 → 0.2.0; add INTERIM multi-repo tasks subsection (status --json enumeration, per-repo loop, maproom-all as local convenience, --repo specs availability, no --all-repos) - plugin.json: 0.10.1 → 0.10.2 (PATCH per repo convention) - verify-docs.sh: 11/11 checks pass Co-Authored-By: Claude Fable 5 --- plugins/maproom/.claude-plugin/plugin.json | 2 +- plugins/maproom/README.md | 4 +- plugins/maproom/agents/maproom-researcher.md | 29 +++++++++- .../references/concept-glossary.md | 9 +++- .../maproom-search/references/adc-setup.md | 2 + .../references/embedding-providers.md | 2 + .../references/troubleshooting.md | 53 +++++++++++++++++++ 7 files changed, 94 insertions(+), 7 deletions(-) diff --git a/plugins/maproom/.claude-plugin/plugin.json b/plugins/maproom/.claude-plugin/plugin.json index 024a8c54..b8e100cd 100644 --- a/plugins/maproom/.claude-plugin/plugin.json +++ b/plugins/maproom/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "maproom", - "version": "0.10.1", + "version": "0.10.2", "description": "Semantic code search using the maproom CLI. Find code by concept, understand architecture, and explore relationships between code elements.", "author": { "name": "Daniel Bushman", diff --git a/plugins/maproom/README.md b/plugins/maproom/README.md index 45d60a63..e5972a70 100644 --- a/plugins/maproom/README.md +++ b/plugins/maproom/README.md @@ -39,7 +39,7 @@ Before using the Maproom plugin, ensure you have: maproom --version ``` 3. **Indexed database**: Your codebase must be scanned using `maproom scan` before searching -4. **Database location**: The maproom database is typically located at `~/.maproom/maproom.db` (can be overridden with `MAPROOM_DATABASE_URL` environment variable) +4. **Database location**: Maproom stores indexed data in the backend configured via `MAPROOM_DATABASE_URL`. SQLite users find the default database at `~/.maproom/maproom.db`; shared-Postgres users set `MAPROOM_DATABASE_URL=postgres://...`. Run `maproom status` to confirm which repos are indexed in the active backend. To verify your setup: ```bash @@ -111,7 +111,7 @@ Uses context expansion to show where validateCart is called throughout the codeb **Solution**: - Run `maproom scan` to index your codebase - Check indexing status: `maproom status` -- Verify database exists: `ls -la ~/.maproom/maproom.db` +- Verify connectivity: `maproom status` (lists indexed repos and confirms backend is reachable; SQLite users can also check `ls -la ~/.maproom/maproom.db`) ### No Results Found **Problem**: Searches return no results or irrelevant matches diff --git a/plugins/maproom/agents/maproom-researcher.md b/plugins/maproom/agents/maproom-researcher.md index ca1c2e9c..c7954b1b 100644 --- a/plugins/maproom/agents/maproom-researcher.md +++ b/plugins/maproom/agents/maproom-researcher.md @@ -47,6 +47,31 @@ maproom status Then select the most relevant repo for the research question. + +### Multi-repo tasks + +When a task spans multiple repositories, enumerate available repos first, then search each one individually: + +```bash +# Step 1: list all indexed repos (shared-Postgres environments return the full inventory) +maproom status --json +``` + +Parse the `repos[].name` values from the JSON output to build your repo list. Then issue a separate search per repo: + +```bash +# Step 2: search each repo in turn +QUERY="" +maproom search --repo --query "$QUERY" --k 10 --format agent +``` + +A local convenience wrapper (`maproom-all`) may be present in some environments; use it if available, but do not assume it exists — the per-repo loop above is portable. + +**`--repo specs` is available in shared-Postgres environments** and covers indexed specification documents. Include it explicitly when the task involves specs. + +**No server-side cross-repo search yet.** There is no `--all-repos` flag. Each search targets exactly one repo. Cross-repo synthesis is done by the agent after collecting results from each repo in sequence. + + ## Critical Rules > **Note:** Search budget constraints are enforced by a PreToolUse hook (enforce-search-cap.py) with a soft cap at 5 and hard cap at 10. @@ -80,7 +105,7 @@ maproom search --repo --query --format agent Before beginning the 4-phase workflow: -1. **CLI Version Check:** Run `maproom --version` and verify version 0.1.0 or higher. If version check fails, report error and halt execution. +1. **CLI Version Check:** Run `maproom --version` and verify version 0.2.0 or higher. If version check fails, report error and halt execution. 2. **Query Validation:** Reject null, empty, or whitespace-only queries. Valid query pattern: at least one non-whitespace character. ## Query Classification (Before Phase 1) @@ -322,4 +347,4 @@ Instead: **Security:** Do not include actual credentials, access tokens, or project IDs in your findings output. Use placeholders like `YOUR_PROJECT_ID`. ### CLI and Query Validation -See Pre-Workflow Checks section above. Version 0.1.0 or higher required; null/empty queries rejected. +See Pre-Workflow Checks section above. Version 0.2.0 or higher required; null/empty queries rejected. diff --git a/plugins/maproom/skills/maproom-guide/references/concept-glossary.md b/plugins/maproom/skills/maproom-guide/references/concept-glossary.md index 93406d08..e1797026 100644 --- a/plugins/maproom/skills/maproom-guide/references/concept-glossary.md +++ b/plugins/maproom/skills/maproom-guide/references/concept-glossary.md @@ -102,9 +102,14 @@ Different embedding providers (Google Vertex AI, OpenAI, Ollama) produce vectors ## Infrastructure Concepts -### SQLite Database +### Storage Backend -Maproom stores all indexed chunks and embeddings in a local **SQLite database** (default: `~/.maproom/maproom.db`). SQLite is a file-based database that requires no server. It supports concurrent reads via WAL (Write-Ahead Logging) mode but serializes writes. +Maproom stores all indexed chunks and embeddings in a database. Two backends are supported: + +- **SQLite (OSS default):** A file-based database at `~/.maproom/maproom.db` (can be overridden via `MAPROOM_DATABASE_URL`). No server required; concurrent reads are handled via WAL (Write-Ahead Logging) mode; writes are serialized. This is the default for standalone/open-source use. +- **PostgreSQL (shared-Postgres environments):** A server-based database configured via `MAPROOM_DATABASE_URL` (e.g. `postgres://user:pass@host/dbname`). Multiple clients share one database; all repos are stored in a single schema. In this devcontainer, `localhost:5433` hosts an in-container PG14 instance used **only** for `cargo test` (throwaway sandbox); the shared live database is reachable at `host.docker.internal:5433`. Run `maproom status` to confirm which repos are indexed in the active backend. + +In shared-Postgres environments, `maproom status` is the canonical probe for connectivity and repo inventory. SQLite WAL-specific file paths (`maproom.db-wal`, `maproom.db-shm`) apply only to the SQLite backend. ### Tree-Sitter diff --git a/plugins/maproom/skills/maproom-search/references/adc-setup.md b/plugins/maproom/skills/maproom-search/references/adc-setup.md index 44b33b25..30907dce 100644 --- a/plugins/maproom/skills/maproom-search/references/adc-setup.md +++ b/plugins/maproom/skills/maproom-search/references/adc-setup.md @@ -1,5 +1,7 @@ # Google ADC Setup for DevContainer +> **Note (shared-Postgres environments):** In shared-Postgres environments where `MAPROOM_DATABASE_URL` points to a PostgreSQL server, embeddings are not generated by default and vector search is not active. **Full-text search (`maproom search`) is fully operative without ADC.** ADC setup is only required if you intend to generate embeddings using Google Vertex AI. This guide remains valid for those cases. + Step-by-step guide for configuring Google Application Default Credentials (ADC) in a DevContainer environment. ADC is required when using Google Vertex AI as the embedding provider for maproom vector search. ## Prerequisites diff --git a/plugins/maproom/skills/maproom-search/references/embedding-providers.md b/plugins/maproom/skills/maproom-search/references/embedding-providers.md index c83dd631..6f37e167 100644 --- a/plugins/maproom/skills/maproom-search/references/embedding-providers.md +++ b/plugins/maproom/skills/maproom-search/references/embedding-providers.md @@ -1,5 +1,7 @@ # Embedding Provider Comparison +> **Note (shared-Postgres environments):** In shared-Postgres environments where `MAPROOM_DATABASE_URL` points to a PostgreSQL server, embeddings are not generated by default — `maproom status` will show `0.0%` embedding coverage. **Full-text search (`maproom search`) is fully operative and does not require embeddings.** Vector search (`maproom vector-search`) requires embeddings to be generated first; follow the provider setup below if you need it. This documentation remains valid when embeddings are enabled. + Maproom supports three embedding providers for vector search: Google Vertex AI, OpenAI, and Ollama. This guide helps you choose the right provider and configure it for your environment. ## Overview diff --git a/plugins/maproom/skills/maproom-search/references/troubleshooting.md b/plugins/maproom/skills/maproom-search/references/troubleshooting.md index 957e76ff..34bd70ae 100644 --- a/plugins/maproom/skills/maproom-search/references/troubleshooting.md +++ b/plugins/maproom/skills/maproom-search/references/troubleshooting.md @@ -537,6 +537,8 @@ This section documents boundary conditions, resource errors, and concurrency sce ### SQLite Lock Contention (GAP-006) -- HIGH PRIORITY +> **SQLite backend only.** This section applies when maproom is configured with a SQLite database. In shared-Postgres environments (`MAPROOM_DATABASE_URL=postgres://...`), PostgreSQL handles concurrent access natively and SQLite lock contention does not occur. + **Symptom:** When multiple agents or processes run `maproom search` or `maproom scan` concurrently against the same repository, commands may fail with SQLite lock errors such as `SQLITE_BUSY` or connection pool timeouts. **Root Cause:** The maproom database uses SQLite, which has limited write concurrency. Concurrent read-only operations (searches) are handled well by SQLite's WAL mode. However, concurrent write operations (scan while searching) or access to a locked database can cause contention. @@ -653,6 +655,8 @@ This section documents boundary conditions, resource errors, and concurrency sce ### Disk Full During Scan (GAP-004) +> **SQLite backend only.** Disk-full behavior described here is specific to SQLite file storage. In shared-Postgres environments (`MAPROOM_DATABASE_URL=postgres://...`), storage is managed by the PostgreSQL server and WAL files (`maproom.db-wal`, `maproom.db-shm`) do not exist on the client. + **Note:** This edge case has not been tested empirically due to the risk of destabilizing the shared development environment. Simulating disk-full conditions requires filling the filesystem, which could affect other processes and services. **Symptom (predicted):** `maproom scan` fails mid-operation when the filesystem runs out of space. The SQLite database may be left in an inconsistent state if the write-ahead log (WAL) cannot be flushed. @@ -688,6 +692,8 @@ Error: database or disk is full ### Permission Denied on Database (GAP-005) +> **SQLite backend only.** Filesystem permission errors apply to SQLite's file-based storage. In shared-Postgres environments (`MAPROOM_DATABASE_URL=postgres://...`), access control is handled by PostgreSQL roles rather than filesystem permissions; see [Postgres Connectivity Issues](#postgres-connectivity-issues) for connection failure troubleshooting. + **Symptom:** Search or scan commands fail with repeated `ERROR unable to open database file` messages followed by a connection pool timeout. The CLI retries with exponential backoff for approximately 25 seconds before giving up. **Root Cause:** The maproom database file (`~/.maproom//maproom.db`) or its directory has insufficient filesystem permissions. SQLite requires read access for searches and write access for scans. SQLite also needs access to the WAL file (`maproom.db-wal`) and shared memory file (`maproom.db-shm`) in the same directory. @@ -765,3 +771,50 @@ Caused by: ``` **Prevention:** Keep `--k` values proportional to the expected result set size. Check `maproom status` to see total chunk counts per repository. For a repository with 6,450 chunks, `--k 100` covers the top 1.5% of results. Very large `--k` values (999,999+) cause longer execution times (~25 seconds vs. <1 second) and produce large output volumes (~750 KB) without improving result quality, since all additional results are lower-relevance matches. + +--- + +## Postgres Connectivity Issues + +> **Postgres backend only.** This section applies when `MAPROOM_DATABASE_URL` is set to a `postgres://` connection string. + +When maproom is configured for a shared PostgreSQL backend, connection failures produce different error patterns than the SQLite errors documented above. + +### Container Health Check + +Before diagnosing a maproom error, verify the Postgres container is running and reachable: + +```bash +# Confirm the container is up +docker ps --filter name=maproom-postgres + +# Probe the port (expect immediate output, not a timeout) +maproom status +``` + +`maproom status` is the primary health probe. A successful response lists indexed repositories. If it hangs or errors, the container or network path is the issue — not maproom itself. + +### Connection String + +The connection string is set via `MAPROOM_DATABASE_URL`. Verify it is exported in the current shell: + +```bash +echo "$MAPROOM_DATABASE_URL" +# Expected: postgres://maproom:maproom@host.docker.internal:5433/maproom +# (exact values depend on your environment) +``` + +If `MAPROOM_DATABASE_URL` is empty, maproom falls back to SQLite defaults. Set it in your shell profile or `.env` for persistence. + +### Common Postgres Connection Errors + +| Error pattern | Likely cause | Fix | +|---|---|---| +| `connection refused` at host/port | Container not running or wrong port | Check `docker ps`; verify port matches the `postgres://` URL | +| `password authentication failed` | Wrong credentials in URL | Update `MAPROOM_DATABASE_URL` with correct user/password | +| `database "..." does not exist` | Database not provisioned | Check container init logs; re-run container setup | +| `timeout` or `no route to host` | Network path blocked (e.g. wrong hostname) | In devcontainer, use `host.docker.internal` for the host machine; use `localhost` only for in-container PG | + +### FTS vs. Embeddings in Shared-Postgres Environments + +Full-text search (`maproom search`) is fully operative against the shared Postgres backend. Vector search (`maproom vector-search`) requires embeddings to have been generated — run `maproom status` and check `embedding_percentage`; a value of `0.0%` means embeddings are absent and vector search will return no results. See [embedding-providers.md](./embedding-providers.md) for generation instructions. From 1900431b049863e6026cfb6fc311febb8d8b2ac9 Mon Sep 17 00:00:00 2001 From: Daniel Bushman Date: Thu, 9 Jul 2026 22:32:25 +0000 Subject: [PATCH 2/2] fix(maproom): resolve review MUST-FIX accuracy defects in docs-refresh MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Bump EXPECTED_VERSION in monthly-cli-verification.sh from 0.1.0 to 0.2.0 (was causing immediate DRIFT on every scheduled CI run) - Update README.md minimum version floor from 0.1.0 to 0.2.0 at all three occurrences (lines 37, 183, 202) — eliminates intra-plugin contradiction with researcher.md's 0.2.0 requirement - Fix concept-glossary.md Storage Backend entry: remove devcontainer-specific prose ("In this devcontainer"), correct shared backend from PG14 to PG16, make addresses illustrative ("e.g.") rather than normative - Fix troubleshooting.md Connection String section: replace "Expected: " with a portable placeholder; add devcontainer form as a labelled example only - Fix troubleshooting.md quick-check block: bump version comment 0.1.0→0.2.0 - Add inline note that the docker container name is environment-specific verify-docs.sh: 11/11 PASS; monthly-cli-verification.sh: no DRIFT Co-Authored-By: Claude Fable 5 --- plugins/maproom/README.md | 6 +++--- plugins/maproom/scripts/monthly-cli-verification.sh | 2 +- .../skills/maproom-guide/references/concept-glossary.md | 2 +- .../skills/maproom-search/references/troubleshooting.md | 8 ++++---- 4 files changed, 9 insertions(+), 9 deletions(-) diff --git a/plugins/maproom/README.md b/plugins/maproom/README.md index e5972a70..25e56c9c 100644 --- a/plugins/maproom/README.md +++ b/plugins/maproom/README.md @@ -34,7 +34,7 @@ See [multi-repo-guide.md](skills/maproom-search/references/multi-repo-guide.md) Before using the Maproom plugin, ensure you have: 1. **maproom CLI installed**: The plugin requires the `maproom` command-line tool to be available in your system PATH -2. **Minimum maproom version**: 0.1.0. Verify your version: +2. **Minimum maproom version**: 0.2.0. Verify your version: ```bash maproom --version ``` @@ -180,7 +180,7 @@ maproom status # Shows last scan timestamp ### Monthly CLI Verification -**Purpose:** Detect maproom CLI flag deprecation or behavior changes before agents encounter failures. The CLI is at v0.1.0 (pre-release), where breaking changes are allowed per semver. 52 command examples across plugin documentation depend on 6 CLI flags; if any flag is renamed or removed, agents will learn deprecated syntax and encounter command failures. +**Purpose:** Detect maproom CLI flag deprecation or behavior changes before agents encounter failures. The CLI is at v0.2.0 (pre-release), where breaking changes are allowed per semver. 52 command examples across plugin documentation depend on 6 CLI flags; if any flag is renamed or removed, agents will learn deprecated syntax and encounter command failures. **Automation:** This procedure is automated via GitHub Actions (see `.github/workflows/monthly-cli-verification.yml`). The workflow runs on the first Friday of each month and creates a GitHub issue if drift is detected. Manual execution is still supported for ad-hoc verification using the `workflow_dispatch` trigger or by running the script directly: ```bash @@ -199,7 +199,7 @@ bash plugins/maproom/scripts/monthly-cli-verification.sh ```bash cd plugins/maproom ``` -- [ ] Run `maproom --version` and verify the version matches the documented version (currently 0.1.0): +- [ ] Run `maproom --version` and verify the version matches the documented version (currently 0.2.0): ```bash maproom --version ``` diff --git a/plugins/maproom/scripts/monthly-cli-verification.sh b/plugins/maproom/scripts/monthly-cli-verification.sh index 219fedf7..4b2a9f1c 100755 --- a/plugins/maproom/scripts/monthly-cli-verification.sh +++ b/plugins/maproom/scripts/monthly-cli-verification.sh @@ -86,7 +86,7 @@ SEARCH_EXPECTED_FLAGS="--format --kind --lang --preview --preview-length --repo VSEARCH_EXPECTED_FLAGS="--format --kind --lang --preview --preview-length --threshold --repo --worktree --query --k" # Expected version -EXPECTED_VERSION="0.1.0" +EXPECTED_VERSION="0.2.0" # --------------------------------------------------------------------------- # Counters diff --git a/plugins/maproom/skills/maproom-guide/references/concept-glossary.md b/plugins/maproom/skills/maproom-guide/references/concept-glossary.md index e1797026..1827ba46 100644 --- a/plugins/maproom/skills/maproom-guide/references/concept-glossary.md +++ b/plugins/maproom/skills/maproom-guide/references/concept-glossary.md @@ -107,7 +107,7 @@ Different embedding providers (Google Vertex AI, OpenAI, Ollama) produce vectors Maproom stores all indexed chunks and embeddings in a database. Two backends are supported: - **SQLite (OSS default):** A file-based database at `~/.maproom/maproom.db` (can be overridden via `MAPROOM_DATABASE_URL`). No server required; concurrent reads are handled via WAL (Write-Ahead Logging) mode; writes are serialized. This is the default for standalone/open-source use. -- **PostgreSQL (shared-Postgres environments):** A server-based database configured via `MAPROOM_DATABASE_URL` (e.g. `postgres://user:pass@host/dbname`). Multiple clients share one database; all repos are stored in a single schema. In this devcontainer, `localhost:5433` hosts an in-container PG14 instance used **only** for `cargo test` (throwaway sandbox); the shared live database is reachable at `host.docker.internal:5433`. Run `maproom status` to confirm which repos are indexed in the active backend. +- **PostgreSQL (shared-Postgres environments):** A server-based database configured via `MAPROOM_DATABASE_URL` (e.g. `postgres://user:pass@host/dbname`). Multiple clients share one database; all repos are stored in a single schema. In shared-Postgres devcontainer setups, two Postgres instances are often present: an in-container instance (e.g. `localhost:5433`, PG14) used **only** for `cargo test` (throwaway sandbox), and the shared live database at a separate host/port (e.g. `host.docker.internal:5433`, PG16). Exact addresses and versions depend on your environment. Run `maproom status` to confirm which repos are indexed in the active backend. In shared-Postgres environments, `maproom status` is the canonical probe for connectivity and repo inventory. SQLite WAL-specific file paths (`maproom.db-wal`, `maproom.db-shm`) apply only to the SQLite backend. diff --git a/plugins/maproom/skills/maproom-search/references/troubleshooting.md b/plugins/maproom/skills/maproom-search/references/troubleshooting.md index 34bd70ae..aeac5e83 100644 --- a/plugins/maproom/skills/maproom-search/references/troubleshooting.md +++ b/plugins/maproom/skills/maproom-search/references/troubleshooting.md @@ -15,7 +15,7 @@ command -v maproom **2. Check CLI version** ```bash maproom --version -# Expected: >= 0.1.0 (minimum version for this documentation) +# Expected: >= 0.2.0 (minimum version for this documentation) ``` **3. Verify database status** @@ -785,7 +785,7 @@ When maproom is configured for a shared PostgreSQL backend, connection failures Before diagnosing a maproom error, verify the Postgres container is running and reachable: ```bash -# Confirm the container is up +# Confirm the container is up (replace maproom-postgres with your actual container name) docker ps --filter name=maproom-postgres # Probe the port (expect immediate output, not a timeout) @@ -800,8 +800,8 @@ The connection string is set via `MAPROOM_DATABASE_URL`. Verify it is exported i ```bash echo "$MAPROOM_DATABASE_URL" -# Expected: postgres://maproom:maproom@host.docker.internal:5433/maproom -# (exact values depend on your environment) +# Expected: postgres://user:pass@host:port/dbname (exact values depend on your environment) +# Example in a devcontainer setup: postgres://maproom:maproom@host.docker.internal:5433/maproom ``` If `MAPROOM_DATABASE_URL` is empty, maproom falls back to SQLite defaults. Set it in your shell profile or `.env` for persistence.