feat(cli,agent): list-documents CLI command, search --scope explorations, list_documents agent tool

- openkb list-documents [--kind summary|exploration] [--json]: new command
  mirroring openkb list-taxonomy, exposing the Core PR's list_documents.
- openkb search --scope gains 'explorations' (TieredWikiSearch's 4th tier).
- openkb list (print_list): deprecated in its help text in favor of
  list-taxonomy/list-documents; its Summaries/Concepts/Entities sections
  now call list_documents/list_taxonomy_items internally instead of
  duplicating directory-glob logic (Documents-registry-table and Reports
  listing stay as their own logic — they don't map onto the 5/7-kind
  content model). Output format unchanged for existing scripts.
- agent/query.py: new list_documents tool (mirrors list_taxonomy's
  browse-list style) registered on the query agent — and therefore also
  the chat agent, which builds on top of it. Instructions updated to
  recognize an explorations search hit as a previously-saved answer,
  distinct from a summaries/sources hit, and to check list_documents
  before re-synthesizing an answer that may already exist. Deferred (not
  part of this PR, see plan): replacing the internal read_file/
  get_page_content tools with the unified get_content — a separate,
  higher-risk change to an already-productive agent, to be assessed on
  its own.
- tests/test_query.py: tool count/name assertions updated for the new
  list_documents tool.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Author: Sebastian Braun

openkb/agent/query.py

@@ -14,6 +14,9 @@
     read_wiki_image,
     write_kb_file,
 )
+from openkb.agent.tools import (
+    list_documents as list_documents_impl,
+)
 from openkb.agent.tools import (
     list_taxonomy as list_taxonomy_impl,
 )
@@ -42,29 +45,39 @@
    browse list, not a keyword search. Pick the slug(s) that match the
    question's meaning by their brief, then read_file the matching
    concepts/<slug>.md or entities/<slug>.md.
-4. If index.md's one-line summaries and list_taxonomy don't surface a
-   specific detail you need (a niche term, an exact figure, an
-   author/creation-date only present in a raw source), use
+4. For "what documents/explorations exist" questions, or to check whether a
+   question was already answered before, call list_documents — same
+   browse-list style as list_taxonomy, but over summaries (one per
+   ingested document) and explorations (saved answers from a previous
+   `openkb query --save`). If a matching exploration's brief already
+   answers the current question, read and reuse it instead of
+   re-synthesizing from summaries/sources.
+5. If index.md's one-line summaries, list_taxonomy, and list_documents
+   don't surface a specific detail you need (a niche term, an exact
+   figure, an author/creation-date only present in a raw source), use
    search_wiki(query, scope) — a tiered, keyword-level full-text search
-   over summaries/sources only (concepts/entities are step 3's job, never
-   search_wiki's). This is a hybrid fallback: use it in addition to, not
-   instead of, index.md/list_taxonomy navigation. Narrow scope to
-   ["sources"] when you specifically need a source-only detail (an exact
-   field name, an author, a date) that a generated summary would likely
-   omit; leave scope unset to search all tiers.
-5. When you need detailed source document content, each summary page has a
+   over summaries/sources/explorations only (concepts/entities are step
+   3's job, never search_wiki's). This is a hybrid fallback: use it in
+   addition to, not instead of, index.md/list_taxonomy/list_documents
+   navigation. Narrow scope to ["sources"] when you specifically need a
+   source-only detail (an exact field name, an author, a date) that a
+   generated summary would likely omit; leave scope unset to search all
+   tiers. A hit from the "explorations" tier is a previously-saved answer,
+   not a document summary — treat it as its own category, distinct from a
+   "summaries"/"sources" hit for the same slug.
+6. When you need detailed source document content, each summary page has a
    `full_text` frontmatter field with the path to the original document content:
    - Short documents (doc_type: short): read_file with that path.
    - PageIndex documents (doc_type: pageindex): use get_page_content(doc_name, pages)
      with tight page ranges. The summary shows document tree structure with page
      ranges to help you target. Never fetch the whole document. A search_wiki
      hit with a "page" locator names the exact page to fetch.
-6. Source content may reference images. Short-doc .md pages link them
+7. Source content may reference images. Short-doc .md pages link them
    note-relative (e.g. ![image](images/doc/file.png), resolved from
    wiki/sources/); long-doc JSON page metadata lists them wiki-root-relative
    (e.g. sources/images/doc/file.png). Pass either form as seen to the
    get_image tool — it accepts both.
-7. Synthesize a clear, concise, well-cited answer grounded in wiki content.
+8. Synthesize a clear, concise, well-cited answer grounded in wiki content.
 
 Answer based only on wiki content. Be concise.
 Before each tool call, output one short sentence explaining the reason.
@@ -117,6 +130,31 @@ def list_taxonomy(kind: str | None = None) -> str:
         """
         return list_taxonomy_impl(wiki_root, kind=kind)
 
+    @function_tool
+    def list_documents(kind: str | None = None) -> str:
+        """List persisted summary/exploration pages with one-line briefs.
+
+        Mirrors list_taxonomy for a different pair of kinds: summaries (one
+        per ingested document) and explorations (saved answers from a
+        previous `openkb query --save`) — an exploration's brief is the
+        originally-asked question. Use this to check whether a matching
+        exploration already answers the current question before
+        re-synthesizing from summaries/sources, or to find a document's
+        slug before reading its summary/source.
+
+        Args:
+            kind: "summary" or "exploration" to restrict the list; omit for both.
+        """
+        items = list_documents_impl(wiki_root, kind=kind)
+        if not items:
+            return "No summaries or explorations found."
+        lines = []
+        for item in items:
+            wikilink = item.path[:-3] if item.path.endswith(".md") else item.path
+            brief_suffix = f" — {item.brief}" if item.brief else ""
+            lines.append(f"- [[{wikilink}]] ({item.kind}){brief_suffix}")
+        return "\n".join(lines)
+
     @function_tool
     def search_wiki(query: str, scope: list[str] | None = None) -> str:
         """Tiered full-text (BM25) keyword search over summaries/sources.
@@ -168,7 +206,7 @@ def get_image(image_path: str) -> ToolOutputImage | ToolOutputText:
     return Agent(
         name="wiki-query",
         instructions=instructions,
-        tools=[read_file, get_page_content, list_taxonomy, search_wiki, get_image],
+        tools=[read_file, get_page_content, list_taxonomy, list_documents, search_wiki, get_image],
         model=f"litellm/{model}",
         model_settings=ModelSettings(**model_settings),
     )

openkb/cli.py

@@ -2556,7 +2556,16 @@ def visualize(ctx, open_browser):
 
 
 def print_list(kb_dir: Path) -> None:
-    """Print all documents in the knowledge base. Usable from CLI and chat REPL."""
+    """Print all documents in the knowledge base. Usable from CLI and chat REPL.
+
+    Deprecated: prefer ``openkb list-taxonomy`` (concepts/entities, with
+    briefs) and ``openkb list-documents`` (summaries/explorations, with
+    briefs) for anything beyond a quick human-readable overview — this
+    command's output format is kept unchanged for existing scripts, but its
+    Summaries/Concepts/Entities sections are now thin wrappers around
+    ``list_documents``/``list_taxonomy_items`` (the same data source as
+    those newer commands) instead of duplicating directory-glob logic.
+    """
     openkb_dir = kb_dir / ".openkb"
     hashes_file = openkb_dir / "hashes.json"
     if not hashes_file.exists():
@@ -2568,7 +2577,9 @@ def print_list(kb_dir: Path) -> None:
         click.echo("No documents indexed yet.")
         return
 
-    # Display documents table with count in header
+    # Display documents table with count in header. Registry metadata (file
+    # type, page count) isn't wiki content, so it stays its own logic rather
+    # than going through list_documents/get_content.
     doc_count = len(hashes)
     click.echo(f"Documents ({doc_count}):")
     click.echo(f"  {'Name':<40} {'Type':<12} {'Pages':<8}")
@@ -2581,34 +2592,33 @@ def print_list(kb_dir: Path) -> None:
         pages_str = str(pages) if pages else ""
         click.echo(f"  {name:<40} {display:<12} {pages_str:<8}")
 
+    from openkb.agent.content import list_documents, list_taxonomy_items
+
+    wiki_root = str(kb_dir / "wiki")
+
     # Display summaries
-    summaries_dir = kb_dir / "wiki" / "summaries"
-    if summaries_dir.exists():
-        summaries = sorted(p.stem for p in summaries_dir.glob("*.md"))
-        if summaries:
-            click.echo(f"\nSummaries ({len(summaries)}):")
-            for s in summaries:
-                click.echo(f"  - {s}")
+    summaries = [i.slug for i in list_documents(wiki_root, kind="summary")]
+    if summaries:
+        click.echo(f"\nSummaries ({len(summaries)}):")
+        for s in summaries:
+            click.echo(f"  - {s}")
 
     # Display concepts
-    concepts_dir = kb_dir / "wiki" / "concepts"
-    if concepts_dir.exists():
-        concepts = sorted(p.stem for p in concepts_dir.glob("*.md"))
-        if concepts:
-            click.echo(f"\nConcepts ({len(concepts)}):")
-            for c in concepts:
-                click.echo(f"  - {c}")
+    concepts = [i.slug for i in list_taxonomy_items(wiki_root, kind="concept")]
+    if concepts:
+        click.echo(f"\nConcepts ({len(concepts)}):")
+        for c in concepts:
+            click.echo(f"  - {c}")
 
     # Display entities
-    entities_dir = kb_dir / "wiki" / "entities"
-    if entities_dir.exists():
-        entities = sorted(p.stem for p in entities_dir.glob("*.md"))
-        if entities:
-            click.echo(f"\nEntities ({len(entities)}):")
-            for e in entities:
-                click.echo(f"  - {e}")
-
-    # Display reports
+    entities = [i.slug for i in list_taxonomy_items(wiki_root, kind="entity")]
+    if entities:
+        click.echo(f"\nEntities ({len(entities)}):")
+        for e in entities:
+            click.echo(f"  - {e}")
+
+    # Display reports — reports/ has no brief/frontmatter to speak of, so a
+    # plain glob stays simpler than a dedicated list_reports() would be.
     reports_dir = kb_dir / "wiki" / "reports"
     if reports_dir.exists():
         reports = sorted(p.name for p in reports_dir.glob("*.md"))
@@ -2622,7 +2632,12 @@ def print_list(kb_dir: Path) -> None:
 @click.pass_context
 @_with_kb_lock(exclusive=False)
 def list_cmd(ctx):
-    """List all documents in the knowledge base."""
+    """List all documents in the knowledge base.
+
+    Deprecated: prefer ``openkb list-taxonomy`` (concepts/entities) and
+    ``openkb list-documents`` (summaries/explorations) for anything that
+    needs briefs, a ``--kind`` filter, or ``--json`` output.
+    """
     kb_dir = _find_kb_dir(ctx.obj.get("kb_dir_override"))
     if kb_dir is None:
         click.echo("No knowledge base found. Run `openkb init` first.")
@@ -2638,6 +2653,11 @@ def _taxonomy_items_to_json(items) -> list[dict]:
     ]
 
 
+def _document_items_to_json(items) -> list[dict]:
+    """Convert ``DocumentItem`` dataclasses to plain JSON-serializable dicts."""
+    return [{"kind": i.kind, "slug": i.slug, "path": i.path, "brief": i.brief} for i in items]
+
+
 @cli.command(name="list-taxonomy")
 @click.option(
     "--kind",
@@ -2677,6 +2697,47 @@ def list_taxonomy_cmd(ctx, kind, as_json):
         click.echo(f"[{item.kind}] {item.slug}{type_suffix}{brief_suffix}")
 
 
+@cli.command(name="list-documents")
+@click.option(
+    "--kind",
+    type=click.Choice(["summary", "exploration"]),
+    default=None,
+    help="Restrict to summaries or explorations (default: both).",
+)
+@click.option("--json", "as_json", is_flag=True, default=False, help="Output as JSON.")
+@click.pass_context
+@_with_kb_lock(exclusive=False)
+def list_documents_cmd(ctx, kind, as_json):
+    """List persisted summary/exploration pages with their one-line briefs.
+
+    Mirrors ``openkb list-taxonomy`` for a different pair of kinds:
+    summaries (one per ingested document) and explorations (saved
+    ``openkb query --save`` answers) — an exploration's brief is the
+    originally-saved question. Prefer ``openkb search`` for keyword lookups
+    over many summaries/explorations; this command is for browsing the
+    full list with its briefs.
+    """
+    kb_dir = _find_kb_dir(ctx.obj.get("kb_dir_override"))
+    if kb_dir is None:
+        click.echo("No knowledge base found. Run `openkb init` first.")
+        return
+
+    from openkb.agent.tools import list_documents
+
+    items = list_documents(str(kb_dir / "wiki"), kind=kind)
+
+    if as_json:
+        click.echo(json.dumps(_document_items_to_json(items), ensure_ascii=False, indent=2))
+        return
+
+    if not items:
+        click.echo("No summaries or explorations found.")
+        return
+    for item in items:
+        brief_suffix = f" — {item.brief}" if item.brief else ""
+        click.echo(f"[{item.kind}] {item.slug}{brief_suffix}")
+
+
 def _search_results_to_json(results: dict) -> dict:
     """Convert ``{tier: [SearchHit, ...]}`` to plain JSON-serializable dicts."""
     return {
@@ -2701,20 +2762,21 @@ def _search_results_to_json(results: dict) -> dict:
 @click.option(
     "--scope",
     default=None,
-    help="Comma-separated subset of briefs,summaries,sources (default: all three).",
+    help="Comma-separated subset of briefs,summaries,sources,explorations (default: all four).",
 )
 @click.option("--top-k", default=5, show_default=True, help="Max ranked results per tier.")
 @click.option("--json", "as_json", is_flag=True, default=False, help="Output as JSON.")
 @click.pass_context
 @_with_kb_lock(exclusive=False)
 def search_cmd(ctx, query, scope, top_k, as_json):
-    """Full-text (BM25) search over summaries/sources, tier by tier.
+    """Full-text (BM25) search over summaries/sources/explorations, tier by tier.
 
     Concepts/entities are not covered — use ``openkb list-taxonomy`` for
     those (semantic browsing, not keyword search). Each tier is scored and
     ranked independently: ``briefs`` (one-line document summaries), rich
-    ``summaries`` (full document-summary text), and ``sources`` (raw source
-    files, with a page/line locator pointing at the exact hit location).
+    ``summaries`` (full document-summary text), ``sources`` (raw source
+    files, with a page/line locator pointing at the exact hit location),
+    and ``explorations`` (saved ``openkb query --save`` answers).
     """
     kb_dir = _find_kb_dir(ctx.obj.get("kb_dir_override"))
     if kb_dir is None:
@@ -2738,7 +2800,7 @@ def search_cmd(ctx, query, scope, top_k, as_json):
         return
 
     any_hits = False
-    for tier in ("briefs", "summaries", "sources"):
+    for tier in ("briefs", "summaries", "sources", "explorations"):
         hits = results.get(tier)
         if not hits:
             continue

tests/test_query.py

@@ -17,16 +17,17 @@ def test_agent_name(self, tmp_path):
         agent = build_query_agent(str(tmp_path), "gpt-4o-mini")
         assert agent.name == "wiki-query"
 
-    def test_agent_has_five_tools(self, tmp_path):
+    def test_agent_has_six_tools(self, tmp_path):
         agent = build_query_agent(str(tmp_path), "gpt-4o-mini")
-        assert len(agent.tools) == 5
+        assert len(agent.tools) == 6
 
     def test_agent_tool_names(self, tmp_path):
         agent = build_query_agent(str(tmp_path), "gpt-4o-mini")
         names = {t.name for t in agent.tools}
         assert "read_file" in names
         assert "get_page_content" in names
         assert "list_taxonomy" in names
+        assert "list_documents" in names
         assert "search_wiki" in names
         assert "get_image" in names