diff --git a/MAINTAINERS.md b/MAINTAINERS.md index 6c0577c..dbe16ce 100644 --- a/MAINTAINERS.md +++ b/MAINTAINERS.md @@ -37,8 +37,11 @@ Mechanical only; behaviour is identical. - The setup wizard imports private helpers from `hermes_cli.memory_setup` (`_curses_select`, `_prompt`, ...). Those are Hermes internals, not API; expect to own a copy or drop the wizard hook if they move. -- Tests were not copied: the in-tree tests import `plugins.memory.supermemory` and depend on the - hermes-agent test harness. See `tests/plugins/memory/` in hermes-agent for the originals. +- The in-tree tests were not copied (they depend on the hermes-agent test harness; see + `tests/plugins/memory/` in hermes-agent). `tests/` here stubs the few host modules the provider + imports and runs against the real SDK over a mock HTTP transport; see README → Development. +- 2.0 moved to the v5 API (`supermemory>=5,<6`). `_SupermemoryClient` refuses other SDK majors at + construction; widen `_SDK_MAJOR` together with `pyproject.toml` and `plugin.yaml`. - While the in-tree copy still exists, a same-named user plugin is shadowed by it (bundled providers win on name). It takes effect the moment core drops `plugins/memory/supermemory`. diff --git a/README.md b/README.md index 43d2b38..86b4cdb 100644 --- a/README.md +++ b/README.md @@ -6,9 +6,9 @@ Semantic long-term memory with profile recall, semantic search, explicit memory ## Requirements -- `pip install supermemory` +- `supermemory>=5,<6` (the Python SDK for the v5 API; `hermes plugins install` installs it from `pyproject.toml`) - Hosted: API key from [console.supermemory.ai/keys](https://console.supermemory.ai/keys?create=true) -- Self-hosted: a running [Supermemory local](https://supermemory.ai/docs/self-hosting/overview) server and the API key it prints on first boot +- Self-hosted: a running [Supermemory local](https://supermemory.ai/docs/self-hosting/overview) server that serves the v5 API (`/ns/{namespace}/...`) and the API key it prints on first boot ## Setup @@ -57,7 +57,7 @@ Config file: `$HERMES_HOME/supermemory.json` | `max_recall_results` | `10` | Max recalled items to format into context | | `profile_frequency` | `50` | Include profile facts on first turn and every N turns | | `capture_mode` | `all` | Skip tiny or trivial turns by default | -| `search_mode` | `hybrid` | Search mode: `hybrid` (profile + memories), `memories` (memories only), `documents` (documents only) | +| `search_mode` | `hybrid` | Search mode: `hybrid` (memories + source chunks), `memories` (memories only), `chunks` (source chunks only; `documents` is accepted as the pre-v5 name) | | `entity_context` | built-in default | Extraction guidance passed to Supermemory | | `api_timeout` | `5.0` | Timeout for SDK requests | @@ -97,12 +97,32 @@ Supermemory app, so you can filter, browse, and bulk-manage them per source agen When enabled, Hermes can: - prefetch relevant memory context before each turn -- write each completed user/assistant turn to **one document per session per 4-hour window** (`customId` = `__b<0-5>`, so the API appends deltas), matching the capture shape of the other Supermemory agent integrations -- retry failed turn writes on the next turn, session end, `/reset`, or shutdown (at-least-once: if the API accepted a write but the response was lost, the same turn is appended again) +- write each completed user/assistant turn to **one document per session per 4-hour window** (document `id` = `__b<0-5>`; a v5 `POST` with an existing `id` appends the delta rather than replacing the document), matching the capture shape of the other Supermemory agent integrations +- retry failed turn writes on the next turn, session end, `/reset`, or shutdown. Delivery is **at-least-once**, not exactly-once: SDK-level retries are disabled so one hook sends each batch once, but if the API accepted a write and the response was lost (timeout, dropped connection), the provider's retry appends the same turns again. The retry buffer is in memory only (lost when the process exits) and capped at 50 turns / 256 KiB, dropping the oldest turns first - route every SDK and probe request through the configured hosted or self-hosted endpoint - expose explicit tools for search, store, forget, and profile access +## Upgrading from 1.x (SDK 3) + +2.0 talks to the namespace-first v5 API through `supermemory>=5,<6`. Tools, config keys and environment +variables are unchanged. What to know: + +- **No data migration.** The configured container tag (after the same sanitization as before) is used as the + v5 namespace, so existing memories stay addressable. +- **Self-hosted servers** must serve the v5 API. An older local server will reject the requests; the + configured `base_url` is still used for everything, never silently replaced by the hosted API. +- **Search keeps its pre-v5 behaviour.** v5 lowered the default similarity threshold to 0.3; the provider + sends `threshold: 0.6`, `rerank: "none"` and `rewriteQuery: false` explicitly. Hybrid hits that carry + their text in `chunk` (source passages) are now rendered instead of showing up empty. +- **Profile + query.** The v5 profile endpoint has no query parameter, so a query (automatic recall or + `supermemory-profile` with `query`) fetches the profile and runs a separate search concurrently, bounded by + `api_timeout`. If one of the two fails, the other is still used and the tool reports the failure under `errors`. +- **Forget is confirmed.** Forgetting by id succeeds only when the API reports that exact id as forgotten; + forgetting by query searches memories only (chunk ids are not memory ids). +- If another component pins an older SDK into the Hermes venv, the provider stays inactive and + `hermes memory status` reports the installed version instead of failing on every request. + ## Profile-Scoped Containers Use `{identity}` in the `container_tag` to scope memories per Hermes profile: @@ -134,6 +154,15 @@ When enabled: - Automatic operations (turn capture, prefetch, memory write mirroring) always use the **primary** container only - Custom container instructions are injected into the system prompt +## Development + +The tests run the provider against the real installed SDK with an in-process mock HTTP transport +(no network, no Hermes checkout needed): + +```bash +uv run --isolated --no-project --python 3.11 --with 'supermemory>=5.0.0,<6' --with pytest python -m pytest tests -q +``` + ## Support - [Supermemory Discord](https://supermemory.link/discord) diff --git a/__init__.py b/__init__.py index 104c813..5bd1658 100644 --- a/__init__.py +++ b/__init__.py @@ -2,12 +2,13 @@ from __future__ import annotations -import importlib +import importlib.metadata import json import logging import os import re import threading +import time from datetime import datetime, timezone from pathlib import Path from typing import Any, Callable, Dict, List, Optional @@ -19,7 +20,10 @@ logger = logging.getLogger(__name__) _DEFAULT_CONTAINER_TAG = "hermes" -_VALID_SEARCH_MODES = ("hybrid", "memories", "documents") +_VALID_SEARCH_MODES = ("hybrid", "memories", "chunks", "documents") # "documents" = pre-v5 name for "chunks" +_SDK_MAJOR = 5 # supermemory>=5,<6 (pyproject.toml / plugin.yaml): the namespace-first v5 API +# v5 lowered the server defaults (threshold 0.3); send the pre-v5 search behaviour explicitly so recall precision is unchanged. +_SEARCH_DEFAULTS = {"threshold": 0.6, "rerank": "none", "rewrite_query": False} _DEFAULT_BASE_URL = "https://api.supermemory.ai" _API_KEY_URL = "https://console.supermemory.ai/keys?create=true" # Strips injected / blocks before capture. @@ -168,23 +172,55 @@ def _clean_text_for_capture(text: str) -> str: return _DATA_URI_RE.sub("[image]", _INJECTED_BLOCK_RE.sub("", text or "")).strip() -def _memory_fields(item: Any, *keys: str) -> dict: - """Pick SDK result attrs into a plain dict; ``updated_at`` also accepts camelCase ``updatedAt``.""" - defaults = {"id": "", "memory": "", "similarity": None, "metadata": None} - return {k: getattr(item, "updated_at", None) or getattr(item, "updatedAt", None) if k == "updated_at" else getattr(item, k, defaults[k]) - for k in keys} +def _field(item: Any, name: str, default: Any = None) -> Any: + """Read a field from an SDK model or a plain dict (e.g. after ``model_dump``).""" + return item.get(name, default) if isinstance(item, dict) else getattr(item, name, default) + + +def _memory_text(item: Any) -> str: + """Hybrid/chunk hits leave ``memory`` empty and carry their text in ``chunk`` (or a ``chunks`` list).""" + for key in ("memory", "chunk"): + if isinstance(text := _field(item, key), str) and text.strip(): + return text + texts = (c if isinstance(c, str) else _field(c, "content") for c in _field(item, "chunks") or []) + return "\n".join(t for t in texts if isinstance(t, str) and t.strip()) + + +def _memory_fields(item: Any) -> dict: + """v5 search hit -> plain dict; ``updated_at`` comes from ``system.updatedAt`` (top-level fields kept as fallback).""" + system = _field(item, "system") + updated = next((v for v in (_field(system, "updated_at"), _field(system, "updatedAt"), + _field(item, "updated_at"), _field(item, "updatedAt")) if v), None) + return {"id": _field(item, "id", "") or "", "memory": _memory_text(item), "similarity": _field(item, "similarity"), + "updated_at": updated.isoformat() if isinstance(updated, datetime) else updated, "metadata": _field(item, "metadata")} + + +def _profile_fields(response: Any) -> dict: + """v5 profile facts are ``{id, memory}`` objects: keep their text, never the model repr.""" + profile = _field(response, "profile") + texts = {key: [t for f in (_field(profile, key) or []) if isinstance(t := f if isinstance(f, str) else _field(f, "memory"), str) and t.strip()] + for key in ("static", "dynamic")} + return {**texts, "search_results": []} + + +def _error_text(exc: BaseException) -> str: + return str(exc).strip()[:160] or type(exc).__name__ class _SupermemoryClient: def __init__(self, api_key: str, timeout: float, container_tag: str, search_mode: str = "hybrid", base_url: str = ""): - # Lazy-install the SDK on demand (honors security.allow_lazy_installs and sealed Docker - # venvs). On failure fall through so the raw import produces the canonical ImportError. - pass # dependencies come from pyproject.toml (installed by Hermes on install/enable/update) + # Dependencies come from pyproject.toml (installed by Hermes on install/enable/update). Another + # component can still pin an older SDK into the shared venv; fail here with a clear message + # instead of letting every v5 call fail later. from supermemory import Supermemory + if not (sdk_version := importlib.metadata.version("supermemory")).startswith(f"{_SDK_MAJOR}."): + raise RuntimeError(f"supermemory {sdk_version} is installed; this provider needs supermemory>={_SDK_MAJOR},<{_SDK_MAJOR + 1}") self._api_key, self._container_tag, self._timeout = api_key, container_tag, timeout self._search_mode = search_mode if search_mode in _VALID_SEARCH_MODES else "hybrid" self._base_url = _resolve_base_url(base_url) + # max_retries=0: SDK retries would re-POST an append the server may already have accepted (duplicated turns). + # Failed capture writes are retried once per provider hook instead (see _write_turns). self._client = Supermemory(api_key=api_key, base_url=self._base_url, timeout=timeout, max_retries=0, default_headers={"x-sm-source": "hermes"}) @@ -198,37 +234,63 @@ def _merge_metadata(self, metadata: Optional[dict]) -> dict: def add_memory(self, content: str, metadata: Optional[dict] = None, *, entity_context: str = "", container_tag: Optional[str] = None, custom_id: Optional[str] = None) -> dict: - kwargs: dict[str, Any] = {"content": content.strip(), "container_tags": [container_tag or self._container_tag], + # The v5 namespace is the pre-v5 container tag. POST with an existing ``id`` appends to that document + # (PATCH would replace it). task_type="memory" keeps the pre-v5 extraction pipeline explicit. + kwargs: dict[str, Any] = {"content": content.strip(), "task_type": "memory", **({"metadata": self._merge_metadata(metadata)} if metadata else {}), - **({"entity_context": _clamp_entity_context(entity_context)} if entity_context else {}), - **({"custom_id": custom_id} if custom_id else {})} - return {"id": getattr(self._client.documents.add(**kwargs), "id", "")} + **({"supporting_context": _clamp_entity_context(entity_context)} if entity_context else {}), + **({"id": custom_id} if custom_id else {})} + return {"id": _field(self._client.add(container_tag or self._container_tag, **kwargs), "id", "") or ""} def search_memories(self, query: str, *, limit: int = 5, container_tag: Optional[str] = None, search_mode: Optional[str] = None) -> list[dict]: mode = search_mode or self._search_mode - kwargs: dict[str, Any] = {"q": query, "container_tag": container_tag or self._container_tag, "limit": limit, - **({"search_mode": mode} if mode in _VALID_SEARCH_MODES else {})} - response = self._client.search.memories(**kwargs) - return [{**_memory_fields(item, "id", "memory", "similarity", "updated_at", "metadata"), "memory": getattr(item, "memory", "") or ""} - for item in (getattr(response, "results", None) or [])] - - def get_profile(self, query: Optional[str] = None, *, container_tag: Optional[str] = None) -> dict: - response = self._client.profile(container_tag=container_tag or self._container_tag, **({"q": query} if query else {})) - profile_data = getattr(response, "profile", None) - search_data = getattr(response, "search_results", None) or getattr(response, "searchResults", None) - raw_results = getattr(search_data, "results", None) or search_data or [] - return { - **{k: (getattr(profile_data, k, []) or []) if profile_data else [] for k in ("static", "dynamic")}, - "search_results": [item if isinstance(item, dict) else _memory_fields(item, "memory", "updated_at", "similarity") - for item in raw_results] if isinstance(raw_results, list) else [], - } + response = self._client.search(container_tag or self._container_tag, query=query, limit=limit, + search_mode="chunks" if mode == "documents" else mode, **_SEARCH_DEFAULTS) + return [_memory_fields(item) for item in (_field(response, "results") or [])] + + def get_profile(self, query: Optional[str] = None, *, container_tag: Optional[str] = None, limit: int = 10) -> dict: + namespace = container_tag or self._container_tag + if not query: + return _profile_fields(self._client.profile(namespace)) + # The v5 profile endpoint takes no query, so recall is two requests. The search runs on one short-lived + # helper thread while the profile is fetched here, and the join is capped at the SDK timeout, so recall + # still costs about one request of wall time. A failed or late leg is reported in "errors" and the + # other leg is still used; only when both fail does this raise. + deadline, search, errors = time.monotonic() + self._timeout, {}, {} + + def _search() -> None: + try: + search["results"] = self.search_memories(query, limit=limit, container_tag=namespace) + except Exception as exc: + search["error"] = _error_text(exc) + worker = spawn_context_thread(_search, daemon=True, name="supermemory-recall-search") + worker.start() + try: + result = _profile_fields(self._client.profile(namespace)) + except Exception as exc: + result, errors["profile"] = _profile_fields(None), _error_text(exc) + worker.join(timeout=max(0.0, deadline - time.monotonic())) + if (results := search.get("results")) is not None: + result["search_results"] = results + else: + errors["search"] = search.get("error") or "timed out" + if len(errors) == 2: + raise RuntimeError(f"profile: {errors['profile']}; search: {errors['search']}") + if errors: + logger.debug("Supermemory recall partially failed: %s", errors) + return {**result, "errors": errors} def forget_memory(self, memory_id: str, *, container_tag: Optional[str] = None) -> None: - self._client.memories.forget(container_tag=container_tag or self._container_tag, id=memory_id) + response = self._client.memories.forget(container_tag or self._container_tag, ids=[memory_id]) + # v5 reports per-id outcomes instead of raising: only an exact match for this one id counts as forgotten. + errors = [str(_field(e, "error", "")) for e in _field(response, "errors") or []] + if errors or [_field(m, "id") for m in _field(response, "matches") or []] != [memory_id]: + raise RuntimeError(f"Supermemory did not confirm forgetting {memory_id}" + (f": {'; '.join(errors)}" if errors else "")) def forget_by_query(self, query: str, *, container_tag: Optional[str] = None) -> dict: - results = self.search_memories(query, limit=5, container_tag=container_tag) + # Memories mode only: a hybrid hit can be a chunk, and a chunk id is not a forgettable memory id. + results = self.search_memories(query, limit=5, container_tag=container_tag, search_mode="memories") memory_id = results[0].get("id", "") if results else "" if not memory_id: return {"success": False, "message": "Best matching memory has no id." if results else "No matching memory found to forget."} @@ -312,6 +374,11 @@ def _tagged(resp: dict, tag: Optional[str]) -> dict: return {**resp, "container_tag": tag} if tag else resp +def _tool_result(item: dict) -> dict: + return {"id": item.get("id", ""), "content": item.get("memory", ""), + **({"similarity": pct} if (pct := _similarity_pct(item.get("similarity"))) is not None else {})} + + class SupermemoryMemoryProvider(MemoryProvider): def __init__(self): self._api_key = self._session_id = self._hermes_home = "" @@ -409,14 +476,14 @@ def prefetch(self, query: str, *, session_id: str = "") -> str: if not self._active or not self._auto_recall or not self._client or not query.strip(): return "" def _recall(): - profile = self._client.get_profile(query=query[:200]) + profile = self._client.get_profile(query=query[:200], limit=self._max_recall_results) include_profile = self._turn_count <= 1 or (self._turn_count % self._profile_frequency == 0) return _format_prefetch_context(profile["static"] if include_profile else [], profile["dynamic"] if include_profile else [], profile["search_results"], self._max_recall_results) return _quietly(_recall, "Supermemory prefetch failed", default="") def _write_turns(self, mode: str, new_turn: Optional[Dict[str, str]] = None) -> None: - """Write pending turns (+ ``new_turn``) as one documents.add per session id; custom_id = session + 4h bucket, so the + """Write pending turns (+ ``new_turn``) as one add per session id; id = session + 4h bucket, so the API appends deltas. Failed batches stay pending under their own session id, so a switch never re-homes them. Retries are at-least-once: a write the API accepted but whose response was lost is re-sent and appended again. The lock serializes the snapshot/write/replace sequence across the worker and caller threads.""" @@ -536,8 +603,7 @@ def _tool_search(self, args: dict) -> dict | str: return tool_error("query is required") limit = _clamp_number(args.get("limit", 5) or 5, 5, 1, 20, int) tag = self._tool_container_tag(args) - results = [{"id": i.get("id", ""), "content": i.get("memory", ""), **({"similarity": pct} if (pct := _similarity_pct(i.get("similarity"))) is not None else {})} - for i in self._client.search_memories(query, limit=limit, container_tag=tag)] + results = [_tool_result(i) for i in self._client.search_memories(query, limit=limit, container_tag=tag)] return _tagged({"results": results, "count": len(results)}, tag) def _tool_forget(self, args: dict) -> dict | str: @@ -552,9 +618,13 @@ def _tool_forget(self, args: dict) -> dict | str: def _tool_profile(self, args: dict) -> dict: tag = self._tool_container_tag(args) - profile = self._client.get_profile(query=str(args.get("query") or "").strip() or None, container_tag=tag) - return _tagged({"profile": "\n\n".join(_profile_sections(profile["static"], profile["dynamic"])), - "static_count": len(profile["static"]), "dynamic_count": len(profile["dynamic"])}, tag) + query = str(args.get("query") or "").strip() or None + profile = self._client.get_profile(query=query, container_tag=tag) + resp = {"profile": "\n\n".join(_profile_sections(profile["static"], profile["dynamic"])), + "static_count": len(profile["static"]), "dynamic_count": len(profile["dynamic"])} + if query: # v5 profiles take no query, so the query focuses a separate memory search + resp["results"] = [_tool_result(i) for i in profile["search_results"]] + return _tagged({**resp, **({"errors": profile["errors"]} if profile.get("errors") else {})}, tag) def handle_tool_call(self, tool_name: str, args: Dict[str, Any], **kwargs) -> str: """Handlers return a tool_error() string for bad args or a dict to JSON-encode; client failures get ``fail_prefix``.""" diff --git a/plugin.yaml b/plugin.yaml index afb2cf0..5081e7b 100644 --- a/plugin.yaml +++ b/plugin.yaml @@ -1,5 +1,5 @@ name: supermemory -version: 1.0.1 +version: 2.0.0 description: "Supermemory semantic long-term memory with profile recall, semantic search, explicit memory tools, and per-turn conversation capture." pip_dependencies: - - supermemory + - supermemory>=5.0.0,<6 diff --git a/pyproject.toml b/pyproject.toml index 2e86afa..e8dcb8e 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "hermes-supermemory" -version = "1.0.1" +version = "2.0.0" description = "Hermes Agent memory provider plugin: supermemory" requires-python = ">=3.11" license = { text = "MIT" } @@ -8,5 +8,5 @@ license = { text = "MIT" } # every `hermes update` (hermes-agent#113851). Keep upper bounds: Hermes pins its own deps exactly # and refuses a plugin whose requirements cannot resolve against them. dependencies = [ - "supermemory>=3.50.0,<4", + "supermemory>=5.0.0,<6", ] diff --git a/tests/conftest.py b/tests/conftest.py new file mode 100644 index 0000000..a39e27b --- /dev/null +++ b/tests/conftest.py @@ -0,0 +1,150 @@ +"""Offline harness: the real provider module and the real installed supermemory SDK (its request +serializers and response models), with HTTP answered by an ``httpx.MockTransport``. + +Only the Hermes host modules the provider imports are stubbed (mirroring their public signatures), +so the tests run without a hermes-agent checkout. Sockets are blocked: nothing reaches a network. +""" + +from __future__ import annotations + +import contextvars +import importlib.util +import json +import socket +import sys +import threading +import time +import types +from pathlib import Path + +import httpx +import pytest +import supermemory + +ROOT = Path(__file__).resolve().parents[1] +API_KEY = "sk-synthetic-test-key" +BASE_URL = "http://supermemory.test:6767" +STAMP = "2026-01-01T03:00:00Z" + + +def search_hit(hit_id: str = "m1", **fields) -> dict: + """One v5 SearchResult as the API serializes it (camelCase).""" + return {"id": hit_id, "metadata": {"fixture": True}, "similarity": 0.9, "isLatest": True, "isInference": False, + "system": {"createdAt": STAMP, "updatedAt": STAMP}, **fields} + + +def default_response(op: str, body: dict) -> dict: + if op == "document": + return {"id": body.get("id", "doc-generated"), "status": "queued"} + if op == "search": + return {"results": [search_hit("m1", memory="Synthetic memory"), search_hit("c1", chunk="Synthetic chunk")], + "searchTime": 1.0} + if op == "profile": + return {"profile": {"static": [{"id": "p1", "memory": "Persistent fact"}], + "dynamic": [{"id": "p2", "memory": "Recent fact"}], "buckets": {}}} + if op == "memories": + return {"count": len(body["ids"]), "errors": [], "matches": [{"id": i, "memory": "Synthetic memory"} for i in body["ids"]]} + raise AssertionError(f"unexpected operation {op!r}") + + +class Wire: + """Records every request; ``overrides[op]`` is a JSON body, an HTTP status, an exception, or a callable.""" + + def __init__(self): + self.requests: list[dict] = [] + self.overrides: dict = {} + self.delays: dict[str, float] = {} + self._lock = threading.Lock() + + def __call__(self, request: httpx.Request) -> httpx.Response: + body = json.loads(request.content) if request.content else {} + op = request.url.path.rsplit("/", 1)[-1] + with self._lock: + self.requests.append({"method": request.method, "url": str(request.url), "host": request.url.host, + "path": request.url.path, "op": op, "body": body, "headers": dict(request.headers)}) + if op in self.delays: + time.sleep(self.delays[op]) + value = self.overrides.get(op) + if callable(value) and not isinstance(value, type): + value = value(body) + if isinstance(value, Exception): + raise value + if isinstance(value, int): + return httpx.Response(value, json={"error": "synthetic failure"}) + return httpx.Response(200, json=value if value is not None else default_response(op, body)) + + def ops(self) -> list[str]: + return [r["op"] for r in self.requests] + + +def _install_host_stubs(monkeypatch, secrets: dict, home: Path) -> None: + provider_mod = types.ModuleType("agent.memory_provider") + + class MemoryProvider: # the real base is an ABC with default no-op hooks + pass + + def spawn_context_thread(target, *, name, daemon=True, args=(), kwargs=None): + ctx = contextvars.copy_context() + return threading.Thread(target=lambda *a, **k: ctx.run(target, *a, **k), args=args, kwargs=kwargs, + name=name, daemon=daemon) + + provider_mod.MemoryProvider, provider_mod.spawn_context_thread = MemoryProvider, spawn_context_thread + secret_mod = types.ModuleType("agent.secret_scope") + secret_mod.get_secret = lambda key, default="": secrets.get(key, default) + secret_mod.is_multiplex_active = lambda: False + registry_mod = types.ModuleType("tools.registry") + registry_mod.tool_error = lambda message, **extra: json.dumps({"error": str(message), **extra}) + constants_mod = types.ModuleType("hermes_constants") + constants_mod.get_hermes_home = lambda: home + utils_mod = types.ModuleType("utils") + utils_mod.atomic_json_write = lambda path, data, **kw: Path(path).write_text(json.dumps(data)) + for name, mod in {"agent": types.ModuleType("agent"), "agent.memory_provider": provider_mod, + "agent.secret_scope": secret_mod, "tools": types.ModuleType("tools"), + "tools.registry": registry_mod, "hermes_constants": constants_mod, "utils": utils_mod}.items(): + monkeypatch.setitem(sys.modules, name, mod) + + +class Harness: + def __init__(self, module, wire: Wire, secrets: dict, home: Path): + self.mod, self.wire, self.secrets, self.home = module, wire, secrets, home + + def write_config(self, **config) -> None: + (self.home / "supermemory.json").write_text(json.dumps({"base_url": BASE_URL, "api_timeout": 1.0, **config})) + + def provider(self, session_id: str = "session-a", **init_kwargs): + provider = self.mod.SupermemoryMemoryProvider() + provider.initialize(session_id, hermes_home=str(self.home), **init_kwargs) + return provider + + @staticmethod + def tool(provider, name: str, args: dict) -> dict: + return json.loads(provider.handle_tool_call(name, args)) + + +@pytest.fixture +def harness(monkeypatch, tmp_path): + def no_network(*_args, **_kwargs): + raise AssertionError("network access is forbidden in these tests") + monkeypatch.setattr(socket.socket, "connect", no_network) + monkeypatch.setattr(socket, "create_connection", no_network) + + secrets = {"SUPERMEMORY_API_KEY": API_KEY} + _install_host_stubs(monkeypatch, secrets, tmp_path) + spec = importlib.util.spec_from_file_location("hermes_supermemory_under_test", ROOT / "__init__.py") + module = importlib.util.module_from_spec(spec) + monkeypatch.setitem(sys.modules, spec.name, module) + spec.loader.exec_module(module) + + wire, real_client, clients = Wire(), supermemory.Supermemory, [] + + def client_factory(**kwargs): # the real SDK client, with only its HTTP transport replaced + client = real_client(**kwargs, http_client=httpx.Client(transport=httpx.MockTransport(wire))) + clients.append(client) + return client + monkeypatch.setattr(supermemory, "Supermemory", client_factory) + + h = Harness(module, wire, secrets, tmp_path) + h.write_config() + yield h + for client in clients: + client._http_client.close() diff --git a/tests/pytest.ini b/tests/pytest.ini new file mode 100644 index 0000000..fc91399 --- /dev/null +++ b/tests/pytest.ini @@ -0,0 +1,4 @@ +# The repository root is the plugin package itself (its __init__.py imports Hermes host modules). +# Rooting pytest here keeps it from importing that package outside the stubbed host in conftest.py. +[pytest] +addopts = -p no:cacheprovider diff --git a/tests/test_provider_sdk5.py b/tests/test_provider_sdk5.py new file mode 100644 index 0000000..889d111 --- /dev/null +++ b/tests/test_provider_sdk5.py @@ -0,0 +1,337 @@ +"""SDK5 / v5 API contract tests for the provider (synthetic data, mock HTTP transport only).""" + +from __future__ import annotations + +import importlib.metadata +import time +import tomllib +from datetime import datetime, timezone + +import httpx +import pytest + +from conftest import API_KEY, BASE_URL, ROOT, STAMP, search_hit + +CAPTURE_PATH = "/ns/hermes/document" + + +# ---- dependency declaration / SDK gate ------------------------------------------------------- + +def test_installed_sdk_is_inside_the_declared_range(): + declared = tomllib.loads((ROOT / "pyproject.toml").read_text())["project"]["dependencies"] + assert declared == ["supermemory>=5.0.0,<6"] + assert " - supermemory>=5.0.0,<6\n" in (ROOT / "plugin.yaml").read_text() + assert importlib.metadata.version("supermemory").split(".")[0] == "5" + + +@pytest.mark.parametrize("version,accepted", [("3.62.0", False), ("4.0.0", False), ("5.0.0", True), ("5.3.1", True), ("6.0.0", False)]) +def test_sdk_gate_accepts_only_the_v5_major(harness, monkeypatch, version, accepted): + monkeypatch.setattr(harness.mod.importlib.metadata, "version", lambda _name: version) + provider = harness.provider() + assert provider._active is accepted + status = harness.mod._probe_supermemory_connection(API_KEY, str(harness.home)) + assert status["ok"] is accepted + if not accepted: + assert "needs supermemory>=5,<6" in status["error"] + assert harness.wire.requests == [] + + +# ---- capture: append by stable id, isolation, retries ------------------------------------------ + +def test_turn_capture_appends_by_stable_id_with_source_metadata(harness): + provider = harness.provider() + provider.sync_turn("Hi recalledthere", "Hello") + provider.sync_turn("see data:image/png;base64,AAAA", "Nice") + provider.on_session_end([{"role": "user", "content": "already captured"}]) + provider.shutdown() + + assert [(r["method"], r["path"]) for r in harness.wire.requests] == [("POST", CAPTURE_PATH)] * 2 + first, second = (r["body"] for r in harness.wire.requests) + assert first["id"] == second["id"] and first["id"].startswith("session_a_") + assert first["content"] == "[role: user]\nHi there\n[user:end]\n[role: assistant]\nHello\n[assistant:end]" + assert "[image]" in second["content"] and "base64" not in second["content"] + assert first["taskType"] == "memory" + assert first["supportingContext"] == harness.mod._DEFAULT_ENTITY_CONTEXT + assert first["metadata"]["sm_source"] == "hermes" + assert first["metadata"]["type"] == "conversation" and first["metadata"]["session_id"] == "session-a" + assert not {"containerTags", "customId", "dreaming", "entityContext"} & set(first) + + +@pytest.mark.parametrize("failure", [503, 409, 429, httpx.ReadTimeout("synthetic lost acknowledgement"), + httpx.ConnectError("synthetic refused")]) +def test_failed_capture_is_sent_once_and_retried_only_by_the_provider(harness, failure): + provider = harness.provider() + harness.wire.overrides["document"] = failure + provider.sync_turn("remember me", "ok") + assert len(harness.wire.requests) == 1 # SDK retries are off: no hidden duplicate appends + assert len(provider._pending_turns) == 1 + sent = harness.wire.requests[0]["body"] + + del harness.wire.overrides["document"] + provider.on_session_end([]) + assert len(harness.wire.requests) == 2 and harness.wire.requests[1]["body"]["content"] == sent["content"] + assert harness.wire.requests[1]["body"]["id"] == sent["id"] + assert provider._pending_turns == [] + provider.shutdown() + assert len(harness.wire.requests) == 2 # nothing left to resend after a successful retry + + +def test_pending_retry_carries_its_original_session(harness): + provider = harness.provider() + harness.wire.overrides["document"] = 503 + provider.sync_turn("old session turn", "ok") + del harness.wire.overrides["document"] + provider.on_session_switch("session-b", reset=True) + provider.sync_turn("new session turn", "ok") + sessions = [r["body"]["metadata"]["session_id"] for r in harness.wire.requests] + assert sessions == ["session-a", "session-a", "session-b"] + assert provider._pending_turns == [] + + +def test_pending_buffer_stays_bounded(harness): + provider = harness.provider() + harness.wire.overrides["document"] = 503 + for i in range(harness.mod._MAX_PENDING_TURNS + 3): + provider.sync_turn(f"turn {i}", "ok") + assert len(provider._pending_turns) == harness.mod._MAX_PENDING_TURNS + assert provider._pending_turns[0]["user"] == "turn 3" # oldest dropped first + + +@pytest.mark.parametrize("context", ["cron", "flush", "subagent"]) +def test_non_primary_contexts_never_write(harness, context): + provider = harness.provider(agent_context=context) + provider.sync_turn("user", "assistant") + provider.on_memory_write("add", "memory", "mirrored fact") + provider.on_session_switch("session-b") + provider.on_session_end([]) + provider.shutdown() + assert harness.wire.requests == [] + + +def test_auto_capture_and_auto_recall_can_be_disabled(harness): + harness.write_config(auto_capture=False, auto_recall=False) + provider = harness.provider() + provider.sync_turn("not stored", "not stored") + assert provider.prefetch("anything") == "" + provider.shutdown() + assert harness.wire.requests == [] + + +def test_explicit_store_and_memory_mirror_use_the_v5_add(harness): + provider = harness.provider() + result = harness.tool(provider, "supermemory-save", {"content": "I prefer tea", "metadata": {"source": "x"}}) + assert result == {"saved": True, "id": "doc-generated", "preview": "I prefer tea"} + body = harness.wire.requests[-1]["body"] + assert harness.wire.requests[-1]["path"] == CAPTURE_PATH + assert body["metadata"] == {"sm_source": "hermes", "type": "preference"} and "id" not in body + + provider.on_memory_write("add", "user", "mirrored fact") + provider.shutdown() + assert harness.wire.requests[-1]["body"]["metadata"] == {"sm_source": "hermes", "target": "user", "type": "explicit_memory"} + + +# ---- namespaces ---------------------------------------------------------------------------------- + +def test_container_tag_template_and_env_become_the_namespace(harness): + harness.write_config(container_tag="team-{identity}") + provider = harness.provider(agent_identity="coder") + harness.tool(provider, "supermemory_search", {"query": "q"}) + assert harness.wire.requests[-1]["path"] == "/ns/team_coder/search" # same sanitized tag as before v5 + + harness.secrets["SUPERMEMORY_CONTAINER_TAG"] = "from-env" + provider = harness.provider() + harness.tool(provider, "supermemory_search", {"query": "q"}) + assert harness.wire.requests[-1]["path"] == "/ns/from_env/search" + + +def test_custom_containers_are_whitelisted_and_ignored_when_disabled(harness): + provider = harness.provider() + for name, args in [("supermemory_store", {"content": "x"}), ("supermemory_search", {"query": "q"}), + ("supermemory_profile", {}), ("supermemory_forget", {"id": "m1"})]: + assert "error" not in harness.tool(provider, name, {**args, "container_tag": "project_alpha"}) + assert {r["path"].split("/")[2] for r in harness.wire.requests} == {"hermes"} + + harness.write_config(enable_custom_container_tags=True, custom_containers=["project_alpha"]) + provider = harness.provider() + count = len(harness.wire.requests) + result = harness.tool(provider, "supermemory_search", {"query": "q", "container_tag": "project_alpha"}) + assert result["container_tag"] == "project_alpha" + assert harness.wire.requests[-1]["path"] == "/ns/project_alpha/search" + assert "error" in harness.tool(provider, "supermemory_search", {"query": "q", "container_tag": "project_beta"}) + assert len(harness.wire.requests) == count + 1 + provider.sync_turn("auto capture", "stays primary") + assert harness.wire.requests[-1]["path"] == CAPTURE_PATH + + +# ---- search ---------------------------------------------------------------------------------------- + +@pytest.mark.parametrize("mode,wire_mode", [("hybrid", "hybrid"), ("memories", "memories"), ("chunks", "chunks"), ("documents", "chunks")]) +def test_search_modes_and_pre_v5_defaults_on_the_wire(harness, mode, wire_mode): + harness.write_config(search_mode=mode) + provider = harness.provider() + result = harness.tool(provider, "supermemory-search", {"query": "tea", "limit": 7}) + request = harness.wire.requests[-1] + assert (request["method"], request["path"]) == ("POST", "/ns/hermes/search") + assert request["body"] == {"query": "tea", "limit": 7, "searchMode": wire_mode, + "threshold": 0.6, "rerank": "none", "rewriteQuery": False} + assert result == {"count": 2, "results": [{"id": "m1", "content": "Synthetic memory", "similarity": 90}, + {"id": "c1", "content": "Synthetic chunk", "similarity": 90}]} + + +@pytest.mark.parametrize("fields,text", [ + ({"memory": "memory text", "chunk": "chunk text"}, "memory text"), + ({"memory": " ", "chunk": "chunk text"}, "chunk text"), + ({"memory": None, "chunk": None, "chunks": [{"content": "first"}, {"content": " "}, {"content": "second"}]}, "first\nsecond"), + ({}, ""), +]) +def test_hit_text_falls_back_from_memory_to_chunk_text(harness, fields, text): + harness.wire.overrides["search"] = {"results": [search_hit("h1", **fields)], "searchTime": 1.0} + provider = harness.provider() + assert harness.tool(provider, "supermemory_search", {"query": "q"})["results"][0]["content"] == text + hit = provider._client.search_memories("q")[0] + assert hit["updated_at"] == STAMP and hit["metadata"] == {"fixture": True} + + +# ---- profile / recall ---------------------------------------------------------------------------- + +def test_profile_without_query_is_one_request(harness): + provider = harness.provider() + result = harness.tool(provider, "supermemory_profile", {}) + assert harness.wire.ops() == ["profile"] and harness.wire.requests[0]["body"] == {} + assert result == {"profile": "## User Profile (Persistent)\n- Persistent fact\n\n## Recent Context\n- Recent fact", + "static_count": 1, "dynamic_count": 1} + + +def test_profile_with_query_adds_a_separate_search(harness): + provider = harness.provider() + result = harness.tool(provider, "supermemory_profile", {"query": "tea"}) + assert sorted(harness.wire.ops()) == ["profile", "search"] + assert next(r["body"] for r in harness.wire.requests if r["op"] == "profile") == {} + assert next(r["body"]["query"] for r in harness.wire.requests if r["op"] == "search") == "tea" + assert "ProfileFact" not in result["profile"] and "- Persistent fact" in result["profile"] + assert [r["content"] for r in result["results"]] == ["Synthetic memory", "Synthetic chunk"] + assert "errors" not in result + + +def test_prefetch_renders_profile_and_chunk_hits_deduped(harness): + harness.wire.overrides["search"] = {"results": [search_hit("m1", memory="Persistent fact"), search_hit("c1", chunk="Synthetic chunk")], + "searchTime": 1.0} + harness.write_config(max_recall_results=4) + provider = harness.provider() + text = provider.prefetch("x" * 300) + assert text.startswith("") and text.endswith("") + assert text.count("Persistent fact") == 1 and "Recent fact" in text and "Synthetic chunk" in text + search = next(r["body"] for r in harness.wire.requests if r["op"] == "search") + assert len(search["query"]) == 200 and search["limit"] == 4 + + provider.on_turn_start(2, "next") + text = provider.prefetch("next") + assert "User Profile (Persistent)" not in text and "Relevant Memories" in text + + +@pytest.mark.parametrize("failed,kept", [("profile", "Synthetic memory"), ("search", "Persistent fact")]) +@pytest.mark.parametrize("failure", [401, 503, httpx.ReadTimeout("synthetic timeout")]) +def test_one_failed_recall_leg_keeps_the_other(harness, failed, kept, failure): + harness.wire.overrides[failed] = failure + provider = harness.provider() + assert kept in provider.prefetch("query") + assert len(harness.wire.requests) == 2 # one request per leg, no SDK retries + result = harness.tool(provider, "supermemory_profile", {"query": "query"}) + assert set(result["errors"]) == {failed} + + +def test_both_recall_legs_failing_is_an_error(harness): + harness.wire.overrides.update(profile=503, search=503) + provider = harness.provider() + assert provider.prefetch("query") == "" + assert "Profile failed" in harness.tool(provider, "supermemory_profile", {"query": "q"})["error"] + assert not harness.mod._probe_supermemory_connection(API_KEY, str(harness.home))["ok"] + + +def test_slow_search_leg_is_bounded_by_the_timeout(harness): + harness.write_config(api_timeout=0.5) + harness.wire.delays["search"] = 1.5 # MockTransport ignores httpx timeouts, so only the provider's bound applies + provider = harness.provider() + started = time.monotonic() + text = provider.prefetch("query") + assert time.monotonic() - started < 1.2 + assert "Persistent fact" in text and "Synthetic memory" not in text + + +# ---- forget ---------------------------------------------------------------------------------------- + +def test_forget_by_exact_id(harness): + provider = harness.provider() + assert harness.tool(provider, "supermemory-forget", {"id": "m1"}) == {"forgotten": True, "id": "m1"} + request = harness.wire.requests[-1] + assert (request["method"], request["path"], request["body"]) == ("DELETE", "/ns/hermes/memories", {"ids": ["m1"]}) + + +@pytest.mark.parametrize("response", [ + {"count": 0, "errors": [], "matches": []}, + {"count": 0, "errors": [{"id": "m1", "error": "not found"}], "matches": []}, + {"count": 1, "errors": [], "matches": [{"id": "other", "memory": "x"}]}, + {"count": 2, "errors": [], "matches": [{"id": "m1", "memory": "x"}, {"id": "m2", "memory": "y"}]}, +]) +def test_forget_never_claims_success_without_an_exact_match(harness, response): + harness.wire.overrides["memories"] = response + result = harness.tool(harness.provider(), "supermemory_forget", {"id": "m1"}) + assert result["error"].startswith("Forget failed: Supermemory did not confirm forgetting m1") + + +def test_forget_by_query_searches_memories_only(harness): + harness.write_config(search_mode="hybrid") + provider = harness.provider() + result = harness.tool(provider, "supermemory_forget", {"query": "synthetic"}) + assert result == {"success": True, "message": 'Forgot: "Synthetic memory"', "id": "m1"} + search, forget = harness.wire.requests[-2:] + assert search["body"]["searchMode"] == "memories" + assert forget["body"] == {"ids": ["m1"]} + + harness.wire.overrides["search"] = {"results": [], "searchTime": 1.0} + assert harness.tool(provider, "supermemory_forget", {"query": "nothing"})["success"] is False + + +# ---- endpoint, auth, status ------------------------------------------------------------------------ + +def test_every_request_uses_the_configured_endpoint_and_key(harness): + provider = harness.provider() + provider.sync_turn("u", "a") + provider.prefetch("q") + harness.tool(provider, "supermemory_forget", {"id": "m1"}) + assert {r["host"] for r in harness.wire.requests} == {"supermemory.test"} + assert all(r["url"].startswith(BASE_URL + "/ns/") for r in harness.wire.requests) + assert {r["headers"]["authorization"] for r in harness.wire.requests} == {f"Bearer {API_KEY}"} + assert {r["headers"]["x-sm-source"] for r in harness.wire.requests} == {"hermes"} + + +def test_base_url_falls_back_to_the_scoped_env_then_hosted_default(harness): + harness.write_config(base_url="") + harness.secrets["SUPERMEMORY_BASE_URL"] = "http://env-local.test:6767/" + harness.tool(harness.provider(), "supermemory_search", {"query": "q"}) + del harness.secrets["SUPERMEMORY_BASE_URL"] + harness.tool(harness.provider(), "supermemory_search", {"query": "q"}) + assert [r["host"] for r in harness.wire.requests] == ["env-local.test", "api.supermemory.ai"] + + +def test_unreachable_local_endpoint_fails_closed(harness): + harness.wire.overrides.update({op: httpx.ConnectError("synthetic refused") for op in ("document", "search", "profile")}) + provider = harness.provider() + provider.sync_turn("u", "a") + assert provider.prefetch("q") == "" + assert "error" in harness.tool(provider, "supermemory_search", {"query": "q"}) + assert not harness.mod._probe_supermemory_connection(API_KEY, str(harness.home))["ok"] + assert {r["host"] for r in harness.wire.requests} == {"supermemory.test"} # never retried against the hosted API + assert len(provider._pending_turns) == 1 + + +def test_status_probe_counts_profile_facts(harness): + status = harness.mod._probe_supermemory_connection(API_KEY, str(harness.home)) + assert status["ok"] and status["profile_facts"] == 2 and status["container_tag"] == "hermes" + assert harness.wire.ops() == ["profile"] + + +def test_capture_bucket_id_is_unchanged(harness): + custom_id = harness.mod._capture_custom_id + assert custom_id("session-a", datetime(2026, 1, 1, 3, 59, tzinfo=timezone.utc)) == "session_a_2026-01-01_b0" + assert custom_id("session-a", datetime(2026, 1, 1, 4, 0, tzinfo=timezone.utc)) == "session_a_2026-01-01_b1"