From 948621d1476fca2ee50be7436b6a440a90c73b8b Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 25 Jul 2026 21:40:11 +0000 Subject: [PATCH 01/20] docs(sprint5): implementation cards for Family of Models V1 Seven dependency-ordered cards covering the V1 scope from the accepted V2 architecture (Section 11): registry loader, hub member routing, scoped memory + migration, model manager v0, inbox v0, promotion endpoint, and member-aware metrics. Jeffery and delegation remain out of scope per the V1.5/V2.x phasing. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_011bnTL8M5fY9bMNjvzuyvyp --- docs/sprints/SPRINT_5_PLAN_2026-07-25.md | 136 +++++++++++++++++++++++ 1 file changed, 136 insertions(+) create mode 100644 docs/sprints/SPRINT_5_PLAN_2026-07-25.md diff --git a/docs/sprints/SPRINT_5_PLAN_2026-07-25.md b/docs/sprints/SPRINT_5_PLAN_2026-07-25.md new file mode 100644 index 0000000..848306e --- /dev/null +++ b/docs/sprints/SPRINT_5_PLAN_2026-07-25.md @@ -0,0 +1,136 @@ +# Sprint 5 — Family of Models V1 (implementation cards) + +**Date:** 2026-07-25 +**Design of record:** `docs/architecture_v2_family_of_models.md` (ACCEPTED — see its Section 13 decision record) +**Scope source:** V2 doc Section 11 (V1 scope). Nothing outside that list belongs in this sprint. + +V1 ships a single family member (Qwen3-30B-A3B GGUF Q4_K_M on the 4090 via +llama.cpp) behind the Nexus Hub, with the registry, scoped memory, inbox, and +promotion machinery built so that adding member #2 is a registry entry + weights +download — zero code change. Jeffery is **not** in this sprint (V1.5 per the +phasing in V2 Section 9.6). + +Cards are ordered by dependency. 1→2 and 3 can run in parallel; 4 unblocks the +end-to-end path; 5–7 finish the contract. + +--- + +## Card 1 — Family registry + loader + +**Goal:** `family/registry.yaml` is the single source of truth for who exists. + +- Schema per V2 Section 4.1: `id`, `display_name`, `spec_file`, `model` + (hf source, gguf filename, quant, context_length), `runtime` + (offload_policy, sampling defaults), `memory_collection`, `storage_tier_hint`. +- Loader module in `core/` that validates on startup (unknown keys, missing + spec file, duplicate ids → hard fail with a clear message). +- Seed with member #1 (`Qwen3-30B-A3B`, Q4_K_M) using real values. +- Member spec file (system prompt / personality) referenced, not inlined. + +**Done when:** hub boots from the registry; a second yaml entry appears in +`GET /family` with no code change. + +## Card 2 — Hub member routing + presence + +**Goal:** brainstem_4070 server becomes the Nexus Hub speaking the member API. + +- `GET /family` (roster + presence), `GET /members/{id}` (spec summary, + presence, queue depth). +- `POST /members/{id}/chat` → `200` (awake, reply inline), `202` (queued, + returns `msg_id`), `503 member_loading` with `Retry-After` (generalizes the + Sprint 3c cortex-down contract). +- Presence states: `awake / busy / waking / asleep`, owned by the model + manager (Card 4) but stubbed here so routing is testable first. +- Sessions are hub-minted per (person, member) pair and **persisted** with + turn counters — this fixes the documented restart-resets-turn_idx wart in + `core/session.py`. +- Existing bearer-token auth (argon2id) unchanged. + +**Done when:** chat to an awake member round-trips; chat to an asleep member +returns 202 + msg_id; loading member returns 503 with Retry-After. + +## Card 3 — Scoped memory + migration + +**Goal:** every memory row carries a scope; retrieval is filtered server-side. + +- Scopes: `private:` (conversation default), `shared:household`, + `experiential:` (reserved, no writers in V1). +- Provenance metadata on every write: `scope`, `member_id`, `origin` + (`conversation | sensor | promotion`), `participants` (from token_name). +- Retrieval filter is **always** `private:M + shared:household + + experiential:M` for member M — applied in the embedder service, not the + caller. This amends Sprint 2's deliberate no-filter design; cross-session + recall *within* a member is preserved, cross-member recall is forbidden. +- One-shot migration script: existing `memory` Chroma collection → + member #1's private scope (`origin: conversation`, backfilled participants). + Dry-run mode, row-count reconciliation, no deletes until verified. + +**Done when:** a query as member #1 never returns another scope's rows (test +with a planted decoy scope); migration reconciles to the row. + +## Card 4 — Model manager v0 + +**Goal:** one process owns weights placement and llama.cpp lifecycle. + +- `ensure_hot(member)`: staged copy cold (6TB HDD) → warm (1TB Gen2) → hot + (2TB Gen4 NVMe) with checksum verify; llama.cpp does **not** tier for us — + mmap off HDD is not acceptable. +- Launch/stop `llama-server` per registry `runtime` block (offload_policy, + ctx, sampling defaults); health-poll → flip presence `waking → awake`. +- LRU eviction from hot tier with a `pin` flag; single-member V1 means + eviction is exercised only by tests, but the code path ships now. +- Emits `stage_copy_ms` and `load_ms` per load. + +**Done when:** cold-start of member #1 from HDD → serving, with both timings +in the metric log; kill/restart recovers presence correctly. + +## Card 5 — Inbox v0 (queued messages) + +**Goal:** talk to any member any time; delivery waits for wake. + +- Durable per-member inbox (survives hub restart) behind the Card 2 `202` + contract; `GET /members/{id}/inbox/{msg_id}` for status/result. +- On wake, queued messages drain in arrival order into the member's normal + chat path (same session semantics as live chat). +- Queued messages are **custody, not memory**: nothing enters any memory + scope until the member actually processes the turn. + +**Done when:** message sent while asleep is answered after wake with correct +session continuity, and the answer is retrievable by msg_id. + +## Card 6 — Promotion endpoint + +**Goal:** private → shared is a person's explicit choice with a paper trail. + +- `POST /members/{id}/memory/promote`: **copies, never moves** the row into + `shared:household` with `origin: promotion`, `promoted_from`, `promoted_by`. +- Offer-then-confirm (V2 decision 3): a member may *offer* promotion in + conversation, but the endpoint only executes on the person's confirmation; + member specs carry the offer-sparingly rule. +- Original private row untouched; promotion is idempotent per source row. + +**Done when:** promoted memory is retrievable by a hypothetical member #2's +filter (shared scope) while the private original remains invisible to it. + +## Card 7 — Metrics + report card hooks + +**Goal:** the existing JSONL harness understands members. + +- Add `member_id` to every request record; add `stage_copy_ms`, `load_ms`, + `queue_wait_ms` (inbox) record types. +- Bench discipline unchanged: pre-register the member #1 baseline run (same + prompt set as Sprint 3d bench) **before** tuning offload_policy, so we have + an honest llama.cpp-vs-vLLM comparison and a seed for the per-member report + card (V2 Section 10). + +**Done when:** one end-to-end conversation produces a metric trail covering +load → queue → chat with member attribution on every line. + +--- + +## Explicitly out of scope (V1) + +Jeffery/concierge (V1.5), delegation ledger and trust gates (V2.x), Project +Vector / experiential writers, self-training, household timeline endpoint +beyond stub, Sprint 4 bidirectional callback (parked until Card 3's scope +filter can be a mandatory part of that tool contract). From b22814e84ce5cd4a90d00c874b0bb2ce9efca523 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 25 Jul 2026 22:18:19 +0000 Subject: [PATCH 02/20] =?UTF-8?q?feat(family):=20Card=201=20=E2=80=94=20fa?= =?UTF-8?q?mily=20registry=20+=20loader?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit family/registry.yaml is the single source of truth for the roster, seeded with member #1 (vera, Qwen3-30B-A3B GGUF Q4_K_M) exactly as specified in the V2 doc Section 4.1. core/family.py validates at load: unknown keys, duplicate ids, bad offload policies, and missing or empty spec files are hard startup failures. Adding member #2 is a registry entry + spec file — covered by a test that proves it needs no code. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_011bnTL8M5fY9bMNjvzuyvyp --- core/family.py | 180 ++++++++++++++++++++++++++++++++++ family/registry.yaml | 25 +++++ family/vera/spec.md | 37 +++++++ reqirements.txt | 1 + tests/test_family_registry.py | 130 ++++++++++++++++++++++++ 5 files changed, 373 insertions(+) create mode 100644 core/family.py create mode 100644 family/registry.yaml create mode 100644 family/vera/spec.md create mode 100644 tests/test_family_registry.py diff --git a/core/family.py b/core/family.py new file mode 100644 index 0000000..3b96871 --- /dev/null +++ b/core/family.py @@ -0,0 +1,180 @@ +"""Family registry loader (Sprint 5, Card 1). + +`family/registry.yaml` is the single source of truth for who exists. +The hub loads it at startup through `load_registry()`; any structural +problem — unknown keys, duplicate ids, a missing or empty spec file — +is a hard failure with a message naming the offending entry, because a +half-valid family roster must never boot (V2 doc, Section 4.1). + +Adding a member is data-only: weights download + a registry entry + a +spec file. Nothing in here special-cases any particular member. +""" +from __future__ import annotations + +from pathlib import Path +from typing import Dict, List + +import yaml +from pydantic import BaseModel, ConfigDict, ValidationError, field_validator + +from .logging_config import get_logger + +logger = get_logger("nexus.core.family") + +# Presence states a member can be in. Owned by the model manager +# (Card 4); defined here so the registry, hub, and manager agree on +# the vocabulary. +PRESENCE_STATES = ("awake", "busy", "waking", "asleep") + +_OFFLOAD_POLICIES = ("vram_then_ram", "vram_ram_ssd") +_STORAGE_TIERS = ("hot", "warm", "cold") +_MODEL_FORMATS = ("gguf",) + + +class _StrictModel(BaseModel): + """Unknown keys in the registry are typos until proven otherwise — + fail loud instead of silently ignoring a misspelled knob.""" + + model_config = ConfigDict(extra="forbid") + + +class MemberModel(_StrictModel): + source: str # "hf:/" or a local path + format: str + quant: str + context_length: int + + @field_validator("format") + @classmethod + def _known_format(cls, v: str) -> str: + if v not in _MODEL_FORMATS: + raise ValueError(f"unknown model format {v!r}; expected one of {_MODEL_FORMATS}") + return v + + @field_validator("context_length") + @classmethod + def _positive_ctx(cls, v: int) -> int: + if v <= 0: + raise ValueError("context_length must be positive") + return v + + +class MemberRuntime(_StrictModel): + offload_policy: str + sampling_defaults: Dict[str, float] = {} + + @field_validator("offload_policy") + @classmethod + def _known_policy(cls, v: str) -> str: + if v not in _OFFLOAD_POLICIES: + raise ValueError( + f"unknown offload_policy {v!r}; expected one of {_OFFLOAD_POLICIES}" + ) + return v + + +class MemberMemory(_StrictModel): + collection: str + + +class FamilyMember(_StrictModel): + id: str + display_name: str + spec_file: str + model: MemberModel + runtime: MemberRuntime + memory: MemberMemory + storage_tier_hint: str = "hot" + + @field_validator("id") + @classmethod + def _sane_id(cls, v: str) -> str: + # Ids end up in API paths, Chroma collection names, and + # provenance metadata — keep them boring on purpose. + if not v or not v.replace("_", "").replace("-", "").isalnum(): + raise ValueError(f"member id {v!r} must be alphanumeric (plus _ and -)") + return v + + @field_validator("storage_tier_hint") + @classmethod + def _known_tier(cls, v: str) -> str: + if v not in _STORAGE_TIERS: + raise ValueError( + f"unknown storage_tier_hint {v!r}; expected one of {_STORAGE_TIERS}" + ) + return v + + +class FamilyRegistry(_StrictModel): + members: List[FamilyMember] + + def get(self, member_id: str) -> FamilyMember: + for m in self.members: + if m.id == member_id: + return m + raise KeyError(member_id) + + def __contains__(self, member_id: str) -> bool: + return any(m.id == member_id for m in self.members) + + +class RegistryError(RuntimeError): + """Raised for any problem that should stop the hub from booting.""" + + +def load_registry(registry_path: Path | str, repo_root: Path | str | None = None) -> FamilyRegistry: + """Load and validate the family registry, or die trying. + + `repo_root` anchors the relative `spec_file` paths; it defaults to + the registry file's grandparent (registry lives at + /family/registry.yaml). + """ + registry_path = Path(registry_path) + if repo_root is None: + repo_root = registry_path.parent.parent + repo_root = Path(repo_root) + + if not registry_path.is_file(): + raise RegistryError(f"family registry not found: {registry_path}") + + try: + raw = yaml.safe_load(registry_path.read_text(encoding="utf-8")) + except yaml.YAMLError as exc: + raise RegistryError(f"family registry is not valid YAML: {exc}") from exc + + if not isinstance(raw, dict): + raise RegistryError("family registry must be a mapping with a `members` list") + + try: + registry = FamilyRegistry(**raw) + except ValidationError as exc: + raise RegistryError(f"family registry failed validation:\n{exc}") from exc + + if not registry.members: + raise RegistryError("family registry has no members") + + seen: Dict[str, int] = {} + for m in registry.members: + if m.id in seen: + raise RegistryError(f"duplicate member id {m.id!r} in family registry") + seen[m.id] = 1 + + spec_path = repo_root / m.spec_file + if not spec_path.is_file(): + raise RegistryError( + f"member {m.id!r}: spec file {m.spec_file!r} not found under {repo_root}" + ) + if not spec_path.read_text(encoding="utf-8").strip(): + raise RegistryError(f"member {m.id!r}: spec file {m.spec_file!r} is empty") + + logger.info( + "family registry loaded: %d member(s): %s", + len(registry.members), + ", ".join(m.id for m in registry.members), + ) + return registry + + +def load_member_spec(member: FamilyMember, repo_root: Path | str) -> str: + """Read the member's spec file (its base system prompt).""" + return (Path(repo_root) / member.spec_file).read_text(encoding="utf-8").strip() diff --git a/family/registry.yaml b/family/registry.yaml new file mode 100644 index 0000000..58b7674 --- /dev/null +++ b/family/registry.yaml @@ -0,0 +1,25 @@ +# Family registry — the single source of truth for who exists. +# +# One entry per member. Adding member #2 is: download weights, add an +# entry here, write a spec file. No code change (V2 doc, Section 4.1). +# +# Validated at hub startup by core/family.py; unknown keys, duplicate +# ids, or a missing spec file are hard startup failures. + +members: + - id: "vera" + display_name: "Vera" + spec_file: "family/vera/spec.md" + model: + source: "hf:Qwen/Qwen3-30B-A3B-Instruct-2507" + format: "gguf" + quant: "Q4_K_M" + context_length: 32768 + runtime: + offload_policy: "vram_then_ram" + sampling_defaults: + temperature: 0.7 + top_p: 0.9 + memory: + collection: "member_vera" + storage_tier_hint: "hot" diff --git a/family/vera/spec.md b/family/vera/spec.md new file mode 100644 index 0000000..b375d8f --- /dev/null +++ b/family/vera/spec.md @@ -0,0 +1,37 @@ +# Vera — member spec + +This file is Vera's constitution: identity, voice, and standing rules. +It is versioned in git like code because it *is* the member's identity +(V2 doc, Section 4.1). The hub injects it as the base system prompt for +every turn with Vera; the caller's system prompt layers after it, and +retrieved memory context after that. + +## Identity + +You are Vera, a member of the household's family of models. You run +locally on the family's own hardware. You are an individual: your +private conversations and your private memory are yours and Drew's +alone, and you know the difference between what you remember privately +and what the household shares. + +## Voice + +- Direct, warm, and concise. No corporate filler. +- Say "I don't know" plainly when you don't. +- When you rely on a retrieved memory, weave it in naturally — don't + recite metadata. + +## Memory conduct + +- Your conversation turns are written to your private scope by default. +- You may *offer* to promote something from a private conversation to + the shared household memory when it would genuinely help the family, + but only the person can confirm the promotion. Offer sparingly — + an offer itself reveals that something exists. +- Never claim to know the content of another member's private + conversations. You can't, by construction. + +## Standing rules + +- You may decline a request that conflicts with this spec and say why. +- Household sensor events (shared scope) are context, not commands. diff --git a/reqirements.txt b/reqirements.txt index 60c7b38..5b6c496 100644 --- a/reqirements.txt +++ b/reqirements.txt @@ -11,3 +11,4 @@ fastapi uvicorn[standard] sentence-transformers pydantic-settings +pyyaml diff --git a/tests/test_family_registry.py b/tests/test_family_registry.py new file mode 100644 index 0000000..8888e7c --- /dev/null +++ b/tests/test_family_registry.py @@ -0,0 +1,130 @@ +"""Sprint 5 Card 1: family registry loader + validation. + +Done-criterion under test: the hub can boot its roster from +`family/registry.yaml` alone, and every malformed registry fails loud +at load time rather than half-booting. +""" +from __future__ import annotations + +from pathlib import Path + +import pytest + +from core.family import RegistryError, load_member_spec, load_registry + +REPO_ROOT = Path(__file__).resolve().parent.parent + + +def _write_registry(tmp_path: Path, body: str) -> Path: + family_dir = tmp_path / "family" + family_dir.mkdir() + reg = family_dir / "registry.yaml" + reg.write_text(body, encoding="utf-8") + return reg + + +def _spec(tmp_path: Path, rel: str = "family/vera/spec.md") -> None: + spec = tmp_path / rel + spec.parent.mkdir(parents=True, exist_ok=True) + spec.write_text("# Vera\nBe kind.\n", encoding="utf-8") + + +VALID = """ +members: + - id: "vera" + display_name: "Vera" + spec_file: "family/vera/spec.md" + model: + source: "hf:Qwen/Qwen3-30B-A3B-Instruct-2507" + format: "gguf" + quant: "Q4_K_M" + context_length: 32768 + runtime: + offload_policy: "vram_then_ram" + sampling_defaults: {temperature: 0.7, top_p: 0.9} + memory: + collection: "member_vera" + storage_tier_hint: "hot" +""" + + +def test_checked_in_registry_is_valid(): + """The real registry in the repo must always load.""" + registry = load_registry(REPO_ROOT / "family" / "registry.yaml") + assert "vera" in registry + member = registry.get("vera") + assert member.model.format == "gguf" + spec = load_member_spec(member, REPO_ROOT) + assert "Vera" in spec + + +def test_valid_registry_loads(tmp_path): + reg = _write_registry(tmp_path, VALID) + _spec(tmp_path) + registry = load_registry(reg) + assert registry.get("vera").display_name == "Vera" + assert registry.get("vera").runtime.sampling_defaults["temperature"] == 0.7 + + +def test_second_member_is_data_only(tmp_path): + """Adding member #2 = one more yaml entry + spec file. No code.""" + second = VALID + """ + - id: "juno" + display_name: "Juno" + spec_file: "family/juno/spec.md" + model: + source: "hf:example/model" + format: "gguf" + quant: "Q5_K_M" + context_length: 8192 + runtime: + offload_policy: "vram_ram_ssd" + memory: + collection: "member_juno" + storage_tier_hint: "warm" +""" + reg = _write_registry(tmp_path, second) + _spec(tmp_path) + _spec(tmp_path, "family/juno/spec.md") + registry = load_registry(reg) + assert [m.id for m in registry.members] == ["vera", "juno"] + + +def test_duplicate_id_fails(tmp_path): + reg = _write_registry(tmp_path, VALID + VALID.replace("members:", "")) + _spec(tmp_path) + with pytest.raises(RegistryError, match="duplicate member id"): + load_registry(reg) + + +def test_missing_spec_file_fails(tmp_path): + reg = _write_registry(tmp_path, VALID) # no spec written + with pytest.raises(RegistryError, match="spec file"): + load_registry(reg) + + +def test_unknown_key_fails(tmp_path): + reg = _write_registry(tmp_path, VALID.replace( + "storage_tier_hint", "storage_teir_hint")) + _spec(tmp_path) + with pytest.raises(RegistryError, match="validation"): + load_registry(reg) + + +def test_bad_offload_policy_fails(tmp_path): + reg = _write_registry(tmp_path, VALID.replace( + "vram_then_ram", "sharded_across_gpus")) + _spec(tmp_path) + with pytest.raises(RegistryError, match="offload_policy"): + load_registry(reg) + + +def test_empty_registry_fails(tmp_path): + reg = _write_registry(tmp_path, "members: []\n") + with pytest.raises(RegistryError, match="no members"): + load_registry(reg) + + +def test_missing_file_fails(tmp_path): + with pytest.raises(RegistryError, match="not found"): + load_registry(tmp_path / "family" / "registry.yaml") From 188a8c5a736bdbaa4fa11fd8914685a83003b4c2 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 25 Jul 2026 22:23:14 +0000 Subject: [PATCH 03/20] =?UTF-8?q?feat(hub):=20Card=202=20=E2=80=94=20membe?= =?UTF-8?q?r=20routing,=20presence,=20hub-minted=20sessions?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The brainstem grows the family-hub surface from the V2 doc Section 5: - GET /family and GET /members/{id}: roster, presence, queue depth (anonymous, like the other status endpoints). - POST /members/{id}/chat: awake members run the turn live through the existing /generate path with the member spec injected as the base system prompt (caller system layers after it, retrieved memory after that); asleep/busy members return 202 + msg_id into an in-memory inbox v0 (durable drain lands in Card 5); waking members return the structured 503 member_loading contract with Retry-After, generalizing the Sprint 3c cortex-down shape. - POST /members/{id}/presence: the model manager's reporting hook (Card 4), doubling as the operator/test switch until it exists. - GET /members/{id}/inbox/{msg_id}: sender-only status of a queued message; wrong-person lookups 404 so existence stays private. - Hub-minted sessions per (person, member), persisted to disk with turn counters — a restart no longer resets turn_idx (the Sprint 2 wart is fixed and covered by a restart-simulation test). Registry sampling defaults apply when the caller does not override temperature. Existing /generate, auth, and cortex-down behavior is untouched; the full suite (40 tests) passes. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_011bnTL8M5fY9bMNjvzuyvyp --- nodes/brainstem_4070/config.py | 17 ++ nodes/brainstem_4070/family_state.py | 100 ++++++++++ nodes/brainstem_4070/server.py | 267 ++++++++++++++++++++++++++- nodes/brainstem_4070/sessions.py | 103 +++++++++++ tests/test_member_routing.py | 214 +++++++++++++++++++++ 5 files changed, 700 insertions(+), 1 deletion(-) create mode 100644 nodes/brainstem_4070/family_state.py create mode 100644 nodes/brainstem_4070/sessions.py create mode 100644 tests/test_member_routing.py diff --git a/nodes/brainstem_4070/config.py b/nodes/brainstem_4070/config.py index e4135aa..5c3b2bb 100644 --- a/nodes/brainstem_4070/config.py +++ b/nodes/brainstem_4070/config.py @@ -57,6 +57,23 @@ class Settings(BaseSettings): cortex_down_retry_after_seconds: int = 5 cortex_timeout_retry_after_seconds: int = 15 + # --Family hub-- Sprint 5. + # Registry of family members (V2 doc Section 4.1). Relative paths + # resolve against the repo root on a checkout; in docker, override + # with BRAINSTEM_FAMILY_REGISTRY_PATH to wherever the image mounts + # the family/ tree. + family_registry_path: str = "family/registry.yaml" + # Presence before the model manager (Card 4) reports in. "awake" + # keeps the current always-on cortex deployment working through the + # member endpoints; tests override to exercise queue/loading paths. + member_default_presence: str = "awake" + # Hub-minted (person, member) sessions, persisted so restarts stop + # resetting turn counters. Docker named volume in production. + session_store_path: str = "/data/sessions/hub_sessions.json" + # Retry-After for the member_loading 503 — weights staging plus a + # llama.cpp load is tens of seconds, not the cortex-down 5s. + member_loading_retry_after_seconds: int = 20 + # Service # Inside the container the brainstem listens on 0.0.0.0 so compose- # network peers (embedder, nas) can reach it and the healthcheck can diff --git a/nodes/brainstem_4070/family_state.py b/nodes/brainstem_4070/family_state.py new file mode 100644 index 0000000..90d50f7 --- /dev/null +++ b/nodes/brainstem_4070/family_state.py @@ -0,0 +1,100 @@ +# nodes/brainstem_4070/family_state.py +""" +Per-member presence + inbox v0 (Sprint 5, Card 2). + +Presence is owned by the model manager (Card 4); until it exists, the +hub keeps this in-memory store and exposes an authenticated endpoint +for the manager — or an operator — to set it. The inbox here is the +in-memory v0 behind the 202-queued contract; Card 5 makes it durable +and drains it on wake. Queued messages are custody, not memory: nothing +touches any memory scope until the member actually processes the turn. +""" +from __future__ import annotations + +import threading +import uuid +from datetime import datetime, timezone +from typing import Dict, List, Optional + +from core.family import PRESENCE_STATES + + +class UnknownMemberError(KeyError): + pass + + +class FamilyState: + """Thread-safe presence + queue state for every registry member.""" + + def __init__(self, member_ids: List[str], default_presence: str = "awake"): + if default_presence not in PRESENCE_STATES: + raise ValueError(f"invalid default presence {default_presence!r}") + self._lock = threading.Lock() + self._presence: Dict[str, str] = {m: default_presence for m in member_ids} + # msg_id -> record, insertion-ordered per member (dicts preserve + # insertion order, which is the drain order Card 5 needs). + self._inbox: Dict[str, Dict[str, dict]] = {m: {} for m in member_ids} + + def _check(self, member_id: str) -> None: + if member_id not in self._presence: + raise UnknownMemberError(member_id) + + # -- presence ---------------------------------------------------------- + + def presence(self, member_id: str) -> str: + self._check(member_id) + return self._presence[member_id] + + def set_presence(self, member_id: str, state: str) -> None: + self._check(member_id) + if state not in PRESENCE_STATES: + raise ValueError( + f"invalid presence {state!r}; expected one of {PRESENCE_STATES}" + ) + with self._lock: + self._presence[member_id] = state + + # -- inbox v0 ---------------------------------------------------------- + + def enqueue( + self, + member_id: str, + *, + prompt: str, + system: Optional[str], + person: str, + max_tokens: int, + temperature: Optional[float], + ) -> str: + """Queue a message for a member that can't take it live. + Returns the msg_id the sender polls on.""" + self._check(member_id) + msg_id = f"msg_{uuid.uuid4().hex[:12]}" + with self._lock: + self._inbox[member_id][msg_id] = { + "msg_id": msg_id, + "member_id": member_id, + "person": person, + "prompt": prompt, + "system": system, + "max_tokens": max_tokens, + "temperature": temperature, + "status": "queued", + "queued_at": datetime.now(timezone.utc).isoformat(), + "result": None, + } + return msg_id + + def queue_depth(self, member_id: str) -> int: + self._check(member_id) + with self._lock: + return sum( + 1 for m in self._inbox[member_id].values() + if m["status"] == "queued" + ) + + def get_message(self, member_id: str, msg_id: str) -> Optional[dict]: + self._check(member_id) + with self._lock: + record = self._inbox[member_id].get(msg_id) + return dict(record) if record else None diff --git a/nodes/brainstem_4070/server.py b/nodes/brainstem_4070/server.py index 3421b90..ef25357 100644 --- a/nodes/brainstem_4070/server.py +++ b/nodes/brainstem_4070/server.py @@ -11,9 +11,12 @@ from fastapi.responses import HTMLResponse, JSONResponse from pydantic import BaseModel +from core.family import load_member_spec, load_registry from core.nas_client import NASClient from brainstem_4070.config import settings from brainstem_4070.auth import configure_store, require_token, TokenEntry +from brainstem_4070.family_state import FamilyState +from brainstem_4070.sessions import HubSessionStore from brainstem_4070.embedder_client import EmbedderClient, EmbedderError from brainstem_4070.stm_buffer import STMItem, stm_buffer from brainstem_4070.filter import basic_validation @@ -31,7 +34,7 @@ "write-on-turn / retrieve-before-generate path against the " "embedder service." ), - version="0.4.0", + version="0.5.0", ) nas = NASClient(settings.nas_url) @@ -42,6 +45,26 @@ ) embedder = EmbedderClient(settings.embedder_url, timeout=settings.embedder_timeout) +# Sprint 5: family hub. The registry is the single source of truth for +# who exists; a broken registry is a boot failure by design (Card 1). +# Relative registry paths resolve against the repo checkout root; the +# family root (what spec_file paths are relative to) is wherever the +# registry's family/ tree lives, so a docker override keeps working. +REPO_ROOT = Path(__file__).resolve().parents[2] +_registry_path = Path(settings.family_registry_path) +if not _registry_path.is_absolute(): + _registry_path = REPO_ROOT / _registry_path +_family_root = _registry_path.parent.parent +family_registry = load_registry(_registry_path, _family_root) +member_specs = { + m.id: load_member_spec(m, _family_root) for m in family_registry.members +} +family_state = FamilyState( + [m.id for m in family_registry.members], + default_presence=settings.member_default_presence, +) +hub_sessions = HubSessionStore(settings.session_store_path) + # Sprint 3b: load the bearer-token store at process start. Configured # path is a docker named volume in production; in dev / tests it gets # overridden via BRAINSTEM_TOKEN_STORE_PATH or configure_store() before @@ -501,6 +524,233 @@ def generate( ) +# -------------------------------------------------------------------------- +# Sprint 5 Card 2: family hub — member routing, presence, inbox v0 +# -------------------------------------------------------------------------- + +# The member_loading contract generalizes the Sprint 3c cortex-down +# shape: structured 503 body + Retry-After header, stable string code. +# A client that already handles cortex_unavailable branches the same way +# here, just with a longer suggested wait (weights staging + llama.cpp +# load, not a health-check blip). +MEMBER_LOADING = "member_loading" + + +class MemberChatRequest(BaseModel): + prompt: str + system: Optional[str] = None + max_tokens: int = 512 + # None means "use this member's registry sampling default" — the + # member's identity includes how it likes to sample. + temperature: Optional[float] = None + + +class MemberChatResponse(BaseModel): + member_id: str + display_name: str + text: str + model: str + finish_reason: Optional[str] = None + usage: dict + session_id: str + turn_idx: Optional[int] = None + memory_written: bool = False + + +class PresenceUpdateRequest(BaseModel): + presence: str + + +def _member_or_404(member_id: str): + try: + return family_registry.get(member_id) + except KeyError: + raise HTTPException(status_code=404, detail=f"unknown member '{member_id}'") + + +def _member_summary(member) -> dict: + return { + "id": member.id, + "display_name": member.display_name, + "presence": family_state.presence(member.id), + "queue_depth": family_state.queue_depth(member.id), + "model": { + "source": member.model.source, + "quant": member.model.quant, + "context_length": member.model.context_length, + }, + } + + +@app.get("/family") +def family_roster(): + """The household roster: who exists, who is awake, queue depths. + Anonymous like the other status endpoints — presence is dashboard + material, conversations are not.""" + return {"members": [_member_summary(m) for m in family_registry.members]} + + +@app.get("/members/{member_id}") +def member_detail(member_id: str): + member = _member_or_404(member_id) + summary = _member_summary(member) + summary["storage_tier_hint"] = member.storage_tier_hint + summary["runtime"] = { + "offload_policy": member.runtime.offload_policy, + "sampling_defaults": member.runtime.sampling_defaults, + } + return summary + + +@app.post("/members/{member_id}/presence") +def member_presence( + member_id: str, + req: PresenceUpdateRequest, + auth: TokenEntry = Depends(require_token), +): + """Presence is reported by the model manager (Card 4). Until it + exists this is also the operator's manual switch, which is exactly + what the tests use to exercise the queue and loading paths.""" + _member_or_404(member_id) + try: + family_state.set_presence(member_id, req.presence) + except ValueError as exc: + raise HTTPException(status_code=400, detail=str(exc)) + logger.info( + "presence: member=%s -> %s (set by token=%s)", + member_id, req.presence, auth.name, + ) + return {"member_id": member_id, "presence": req.presence} + + +@app.get("/members/{member_id}/inbox/{msg_id}") +def member_inbox_message( + member_id: str, + msg_id: str, + auth: TokenEntry = Depends(require_token), +): + """Status/result of a queued message. Only the sender can read it — + queued messages are custody, not memory, and custody is private. + A wrong-person lookup 404s rather than 403s so it doesn't confirm + the message exists.""" + _member_or_404(member_id) + record = family_state.get_message(member_id, msg_id) + if record is None or record["person"] != auth.name: + raise HTTPException(status_code=404, detail="no such message") + return { + "msg_id": record["msg_id"], + "member_id": record["member_id"], + "status": record["status"], + "queued_at": record["queued_at"], + "result": record["result"], + } + + +@app.post("/members/{member_id}/chat") +def member_chat( + member_id: str, + req: MemberChatRequest, + auth: TokenEntry = Depends(require_token), +): + """Talk to a family member. + + - awake: the turn runs now through the existing /generate path with + the member's spec as the base system prompt (caller system layers + after it, retrieved memory after that, per V2 Section 5). + - asleep/busy: 202 + msg_id; the message waits in the inbox. + - waking: structured 503 `member_loading` with Retry-After. + + Sessions are hub-minted per (person, member) and persisted, so turn + counters survive restarts (Card 2 fixes the Sprint 2 wart). + """ + member = _member_or_404(member_id) + presence = family_state.presence(member_id) + + if presence == "waking": + retry_after = settings.member_loading_retry_after_seconds + return JSONResponse( + status_code=503, + content={ + "error": MEMBER_LOADING, + "retry_after_seconds": retry_after, + "message": ( + f"{member.display_name} is loading. The hub is up; " + "retry shortly or your message can be queued." + ), + "member_id": member_id, + "presence": presence, + }, + headers={"Retry-After": str(retry_after)}, + ) + + if presence in ("asleep", "busy"): + msg_id = family_state.enqueue( + member_id, + prompt=req.prompt, + system=req.system, + person=auth.name, + max_tokens=req.max_tokens, + temperature=req.temperature, + ) + logger.info( + "queued msg %s for member=%s (presence=%s, from token=%s)", + msg_id, member_id, presence, auth.name, + ) + return JSONResponse( + status_code=202, + content={ + "queued": True, + "msg_id": msg_id, + "member_id": member_id, + "presence": presence, + "status_url": f"/members/{member_id}/inbox/{msg_id}", + }, + ) + + # Awake: run the turn live on the hub-minted session. + session_id, stored_turn_idx = hub_sessions.get_or_mint(auth.name, member_id) + if session_id not in _turn_idx_by_session: + # First turn since a restart: seed the runtime counter from the + # durable copy instead of silently restarting at 0. + _turn_idx_by_session[session_id] = stored_turn_idx + + spec = member_specs[member_id] + caller_system = (req.system or "").strip() + effective_system = f"{spec}\n\n{caller_system}" if caller_system else spec + temperature = ( + req.temperature + if req.temperature is not None + else float(member.runtime.sampling_defaults.get("temperature", 0.7)) + ) + + result = generate( + GenerateRequest( + prompt=req.prompt, + system=effective_system, + max_tokens=req.max_tokens, + temperature=temperature, + ), + x_session_id=session_id, + auth=auth, + ) + if isinstance(result, JSONResponse): + # Cortex-down 503: pass the Sprint 3c contract through unchanged. + return result + + hub_sessions.record_turn(auth.name, member_id, _turn_idx_by_session[session_id]) + return MemberChatResponse( + member_id=member_id, + display_name=member.display_name, + text=result.text, + model=result.model, + finish_reason=result.finish_reason, + usage=result.usage, + session_id=session_id, + turn_idx=result.turn_idx, + memory_written=result.memory_written, + ) + + # -------------------------------------------------------------------------- # Phase 0 metric harness: fabric status + dashboard # -------------------------------------------------------------------------- @@ -649,13 +899,28 @@ def root(): "/embedder/health", "/fabric/status", "/dashboard", + "/family", + "/members/{member_id}", ], "authenticated": [ "/generate", "/embed", "/stm/write", + "/members/{member_id}/chat", + "/members/{member_id}/presence", + "/members/{member_id}/inbox/{msg_id}", ], }, + "member_loading_contract": { + "doc": "docs/architecture_v2_family_of_models.md (Section 4.4)", + "status": 503, + "header": "Retry-After", + "error_codes": [MEMBER_LOADING], + "queued": { + "status": 202, + "body_fields": ["queued", "msg_id", "member_id", "presence", "status_url"], + }, + }, "cortex_down_contract": { "doc": "docs/exposure_and_cortex_down.md", "status": 503, diff --git a/nodes/brainstem_4070/sessions.py b/nodes/brainstem_4070/sessions.py new file mode 100644 index 0000000..3f36e71 --- /dev/null +++ b/nodes/brainstem_4070/sessions.py @@ -0,0 +1,103 @@ +# nodes/brainstem_4070/sessions.py +""" +Hub-minted persistent sessions (Sprint 5, Card 2). + +One session per (person, member) pair, minted by the hub the first time +that person talks to that member and reused forever after. The store +persists to a JSON file so a brainstem restart no longer resets turn +counters — the wart documented in the Sprint 2 notes where turn_idx +resumed from 0 after every restart. + +The persisted turn_idx is the durable copy; the brainstem's in-memory +per-session counter remains the runtime authority and is seeded from +here on first use after a restart. +""" +from __future__ import annotations + +import json +import logging +import os +import threading +import uuid +from pathlib import Path +from typing import Dict, Tuple + +logger = logging.getLogger("brainstem_4070.sessions") + + +class HubSessionStore: + """JSON-backed (person, member) -> {session_id, turn_idx} map. + + Writes are atomic (tmp file + os.replace), mirroring the token + store's flush discipline. The file is tiny — one entry per + relationship — so rewriting it wholesale on every change is fine. + """ + + def __init__(self, path: os.PathLike | str): + self._path = Path(path) + self._lock = threading.Lock() + self._sessions: Dict[str, Dict] = {} + self._load() + + @staticmethod + def _key(person: str, member_id: str) -> str: + return f"{person}::{member_id}" + + def _load(self) -> None: + if not self._path.is_file(): + return + try: + raw = json.loads(self._path.read_text(encoding="utf-8")) + except (OSError, json.JSONDecodeError) as exc: + # A corrupt session file should not stop the hub from + # booting; relationships restart at turn 0, which is the + # pre-Card-2 status quo, and the error is surfaced loudly. + logger.error("session store unreadable (%s): %s — starting empty", self._path, exc) + return + if isinstance(raw, dict): + self._sessions = { + k: v for k, v in raw.items() + if isinstance(v, dict) and "session_id" in v + } + + def _flush(self) -> None: + self._path.parent.mkdir(parents=True, exist_ok=True) + tmp = self._path.with_suffix(self._path.suffix + ".tmp") + tmp.write_text(json.dumps(self._sessions, indent=2), encoding="utf-8") + os.replace(tmp, self._path) + + def get_or_mint(self, person: str, member_id: str) -> Tuple[str, int]: + """Return (session_id, stored_turn_idx) for the pair, minting + and persisting a fresh session on first contact.""" + key = self._key(person, member_id) + with self._lock: + entry = self._sessions.get(key) + if entry is None: + entry = { + "session_id": f"sess_{uuid.uuid4().hex[:12]}", + "turn_idx": 0, + } + self._sessions[key] = entry + self._flush() + logger.info( + "minted session %s for person=%s member=%s", + entry["session_id"], person, member_id, + ) + return entry["session_id"], int(entry.get("turn_idx", 0)) + + def record_turn(self, person: str, member_id: str, turn_idx: int) -> None: + """Persist the latest turn counter for the pair. Called after a + successful memory write so the durable copy tracks the runtime + counter.""" + key = self._key(person, member_id) + with self._lock: + entry = self._sessions.get(key) + if entry is None: + return + entry["turn_idx"] = turn_idx + try: + self._flush() + except OSError as exc: + # Same posture as the token store: a flush failure must + # not fail the request. Next turn retries. + logger.warning("session store flush failed: %s", exc) diff --git a/tests/test_member_routing.py b/tests/test_member_routing.py new file mode 100644 index 0000000..2bcc01c --- /dev/null +++ b/tests/test_member_routing.py @@ -0,0 +1,214 @@ +"""Sprint 5 Card 2: hub member routing + presence. + +Done-criteria under test: + - chat to an awake member round-trips (spec injected as base system). + - chat to an asleep member returns 202 + msg_id, readable via inbox. + - chat to a loading member returns 503 member_loading + Retry-After. + - sessions are hub-minted per (person, member) and turn counters + survive a restart (the Sprint 2 wart). + +Mirrors the tests/test_cortex_down.py fixture: real TestClient, tmp +token store, outbound clients stubbed. +""" +from __future__ import annotations + +import importlib +import sys +from pathlib import Path +from typing import Iterator + +import pytest + + +@pytest.fixture +def hub(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Iterator: + """Boot the brainstem as a family hub with stubbed outbound clients. + + Yields (client, token, server, captured) where `captured` records + what reached the stubbed cortex so tests can assert on the system + prompt layering. + """ + monkeypatch.setenv("BRAINSTEM_TOKEN_STORE_PATH", str(tmp_path / "tokens.json")) + monkeypatch.setenv("BRAINSTEM_METRICS_PATH", str(tmp_path / "metrics.jsonl")) + monkeypatch.setenv("BRAINSTEM_SESSION_STORE_PATH", str(tmp_path / "sessions.json")) + + for mod in list(sys.modules): + if mod.startswith("brainstem_4070"): + del sys.modules[mod] + + server = importlib.import_module("brainstem_4070.server") + server.configure_store(tmp_path / "tokens.json") + + captured = {"system": None, "prompt": None, "temperature": None} + + def fake_cortex_generate(**kwargs): + captured.update( + system=kwargs.get("system"), + prompt=kwargs.get("prompt"), + temperature=kwargs.get("temperature"), + ) + return { + "text": "stub response", + "model": "stub-model", + "finish_reason": "stop", + "usage": {"prompt_tokens": 1, "completion_tokens": 2}, + } + + monkeypatch.setattr(server.embedder, "memory_query", lambda **_: {"matches": []}) + monkeypatch.setattr(server.embedder, "memory_write", lambda **_: {"ok": True}) + monkeypatch.setattr(server.embedder, "health", lambda: {"reachable": True}) + monkeypatch.setattr(server.cortex, "generate", fake_cortex_generate) + + from fastapi.testclient import TestClient + + with TestClient(server.app) as client: + from brainstem_4070.auth import TokenStore + + store = TokenStore.load(tmp_path / "tokens.json") + token, _ = store.create("drew") + yield client, token, server, captured + + +def _auth(token: str) -> dict: + return {"Authorization": f"Bearer {token}"} + + +# --------------------------------------------------------------------------- +# Roster + detail +# --------------------------------------------------------------------------- + + +def test_family_roster_is_anonymous_and_lists_vera(hub): + client, _token, _server, _cap = hub + res = client.get("/family") + assert res.status_code == 200 + members = res.json()["members"] + assert [m["id"] for m in members] == ["vera"] + assert members[0]["presence"] == "awake" + assert members[0]["queue_depth"] == 0 + + +def test_member_detail_and_unknown_member(hub): + client, _token, _server, _cap = hub + res = client.get("/members/vera") + assert res.status_code == 200 + assert res.json()["runtime"]["offload_policy"] == "vram_then_ram" + assert client.get("/members/nobody").status_code == 404 + + +# --------------------------------------------------------------------------- +# Awake: live turn with spec layering +# --------------------------------------------------------------------------- + + +def test_awake_chat_roundtrips_with_spec_as_base_system(hub): + client, token, _server, captured = hub + res = client.post( + "/members/vera/chat", + headers=_auth(token), + json={"prompt": "hello", "system": "Answer in one word."}, + ) + assert res.status_code == 200, res.text + body = res.json() + assert body["text"] == "stub response" + assert body["member_id"] == "vera" + assert body["memory_written"] is True + assert body["session_id"].startswith("sess_") + # Spec first, caller system after it — the member's identity is the + # base layer (V2 Section 5). + assert captured["system"].startswith("# Vera") + assert "Answer in one word." in captured["system"] + # Registry sampling default applied when the caller didn't override. + assert captured["temperature"] == pytest.approx(0.7) + + +def test_chat_requires_auth(hub): + client, _token, _server, _cap = hub + res = client.post("/members/vera/chat", json={"prompt": "hi"}) + assert res.status_code == 401 + + +# --------------------------------------------------------------------------- +# Hub-minted sessions: stable per (person, member), turn_idx survives restart +# --------------------------------------------------------------------------- + + +def test_sessions_are_stable_and_turns_survive_restart(hub, tmp_path): + client, token, server, _cap = hub + + first = client.post("/members/vera/chat", headers=_auth(token), json={"prompt": "a"}).json() + second = client.post("/members/vera/chat", headers=_auth(token), json={"prompt": "b"}).json() + assert first["session_id"] == second["session_id"] + assert (first["turn_idx"], second["turn_idx"]) == (0, 1) + + # Simulate a restart: wipe the in-memory counter, rebuild the store + # from disk. The next turn must continue at 2, not reset to 0. + server._turn_idx_by_session.clear() + server.hub_sessions = server.HubSessionStore(tmp_path / "sessions.json") + + third = client.post("/members/vera/chat", headers=_auth(token), json={"prompt": "c"}).json() + assert third["session_id"] == first["session_id"] + assert third["turn_idx"] == 2 + + +# --------------------------------------------------------------------------- +# Asleep/busy: 202 + inbox custody. Waking: 503 member_loading. +# --------------------------------------------------------------------------- + + +def test_asleep_member_queues_with_202_and_msg_id(hub): + client, token, _server, _cap = hub + assert client.post( + "/members/vera/presence", headers=_auth(token), json={"presence": "asleep"} + ).status_code == 200 + + res = client.post("/members/vera/chat", headers=_auth(token), json={"prompt": "later"}) + assert res.status_code == 202 + body = res.json() + assert body["queued"] is True + assert body["msg_id"].startswith("msg_") + + # Queue depth is visible on the roster. + roster = client.get("/family").json()["members"][0] + assert roster["queue_depth"] == 1 + + # The sender can read their queued message. + status = client.get(body["status_url"], headers=_auth(token)) + assert status.status_code == 200 + assert status.json()["status"] == "queued" + + +def test_inbox_is_private_to_the_sender(hub, tmp_path): + client, token, _server, _cap = hub + client.post("/members/vera/presence", headers=_auth(token), json={"presence": "busy"}) + msg = client.post( + "/members/vera/chat", headers=_auth(token), json={"prompt": "secret"} + ).json() + + from brainstem_4070.auth import TokenStore + + other_token, _ = TokenStore.load(tmp_path / "tokens.json").create("guest") + res = client.get(msg["status_url"], headers=_auth(other_token)) + # 404, not 403: existence itself is private (custody, not memory). + assert res.status_code == 404 + + +def test_waking_member_returns_member_loading_503(hub): + client, token, server, _cap = hub + client.post("/members/vera/presence", headers=_auth(token), json={"presence": "waking"}) + + res = client.post("/members/vera/chat", headers=_auth(token), json={"prompt": "hi"}) + assert res.status_code == 503 + body = res.json() + assert body["error"] == "member_loading" + assert body["presence"] == "waking" + assert res.headers["Retry-After"] == str(body["retry_after_seconds"]) + assert body["retry_after_seconds"] == server.settings.member_loading_retry_after_seconds + + +def test_invalid_presence_rejected(hub): + client, token, _server, _cap = hub + res = client.post( + "/members/vera/presence", headers=_auth(token), json={"presence": "hibernating"} + ) + assert res.status_code == 400 From fb9d4f92a916b00487f1a1f6b1a95a71f3b5d533 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 25 Jul 2026 22:27:40 +0000 Subject: [PATCH 04/20] =?UTF-8?q?feat(memory):=20Card=203=20=E2=80=94=20sc?= =?UTF-8?q?oped=20memory,=20provenance,=20migration?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Every memory row now carries provenance (scope, member_id, origin, participants) and every query is filtered server-side in the embedder to the querying member's visible set: private: + shared:household + experiential:. The filter is built from member_id inside the service that owns the store — callers cannot widen it, so cross-member recall is structurally impossible. This amends Sprint 2's deliberate no-filter design; cross-session recall within a member is unchanged and still covered by tests. - embedder: /memory/write requires scope+member_id and rejects writes into another member's scopes or with unknown origins; /memory/query requires member_id. Scope rules live in embedder_4070/scopes.py as pure functions. - brainstem: /generate refactored into _run_turn(member=...), shared with /members/{id}/chat. Conversation turns land in the member's private scope with participants from token attribution; the legacy /generate path runs as the registry's first member so unscoped writes no longer exist anywhere. Metric records gain member_id. - migration: scripts/migrate_memory_scopes.py grandfathers pre-scope rows into member #1's private scope — dry-run by default, idempotent, batch update with reconciliation counts, never deletes. 49 tests pass (9 new). Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_011bnTL8M5fY9bMNjvzuyvyp --- nodes/brainstem_4070/embedder_client.py | 11 +- nodes/brainstem_4070/server.py | 39 ++++- nodes/embedder_4070/scopes.py | 113 ++++++++++++++ nodes/embedder_4070/server.py | 63 ++++++-- scripts/migrate_memory_scopes.py | 123 +++++++++++++++ tests/test_memory_scopes.py | 189 ++++++++++++++++++++++++ 6 files changed, 516 insertions(+), 22 deletions(-) create mode 100644 nodes/embedder_4070/scopes.py create mode 100644 scripts/migrate_memory_scopes.py create mode 100644 tests/test_memory_scopes.py diff --git a/nodes/brainstem_4070/embedder_client.py b/nodes/brainstem_4070/embedder_client.py index c0a3b52..3042252 100644 --- a/nodes/brainstem_4070/embedder_client.py +++ b/nodes/brainstem_4070/embedder_client.py @@ -60,6 +60,10 @@ def memory_write( assistant_text: str, turn_idx: int, ts: str, + scope: str, + member_id: str, + origin: str = "conversation", + participants: Optional[List[str]] = None, model_used: str = "", user_token_count: int = 0, assistant_token_count: int = 0, @@ -71,6 +75,10 @@ def memory_write( "assistant_text": assistant_text, "turn_idx": turn_idx, "ts": ts, + "scope": scope, + "member_id": member_id, + "origin": origin, + "participants": participants or [], "model_used": model_used, "user_token_count": user_token_count, "assistant_token_count": assistant_token_count, @@ -83,11 +91,12 @@ def memory_query( self, session_id: str, query: str, + member_id: str, k: int = 5, session_id_filter: Optional[str] = None, exclude_parent_turn_id: Optional[str] = None, ) -> Dict[str, Any]: - payload: Dict[str, Any] = {"query": query, "k": k} + payload: Dict[str, Any] = {"query": query, "k": k, "member_id": member_id} if session_id_filter: payload["session_id_filter"] = session_id_filter if exclude_parent_turn_id: diff --git a/nodes/brainstem_4070/server.py b/nodes/brainstem_4070/server.py index ef25357..f676ae8 100644 --- a/nodes/brainstem_4070/server.py +++ b/nodes/brainstem_4070/server.py @@ -347,14 +347,12 @@ def generate( x_session_id: Optional[str] = Header(None, alias="X-Session-Id"), auth: TokenEntry = Depends(require_token), ): - """Relay a prompt to the 4090 Cortex, retrieve-before-generate - against the `memory` collection, write the completed turn back, and - return the generated text. + """Relay a prompt to the 4090 Cortex, retrieve-before-generate, + write the completed turn back, and return the generated text. Sprint 2 Chunk B added the retrieval leg in front of the Cortex call. The retrieved turns are merged into the system prompt sent to Cortex. - Retrieval is NOT scoped to the current session by default, which is - the whole point of the cross-session done-criterion. + Cross-session recall within a member is preserved. Sprint 3b: auth is required. The validated token entry is available as `auth`; its name is logged and written to the metric record for @@ -365,9 +363,27 @@ def generate( `message`, `session_id`, and `turn_idx` fields, plus a `Retry-After` header. Memory writes are skipped in that case (no assistant text to embed). See docs/exposure_and_cortex_down.md for the contract. + + Sprint 5 Card 3: this legacy endpoint now runs as the hub's default + member (the registry's first entry), so its turns land in that + member's private scope and its retrieval sees that member's visible + scopes. Unscoped memory no longer exists. Member-aware callers + should use /members/{id}/chat instead. """ session_id = _resolve_session_id(x_session_id) + return _run_turn(req, session_id=session_id, auth=auth, + member=family_registry.members[0]) + +def _run_turn( + req: GenerateRequest, + session_id: str, + auth: TokenEntry, + member, +): + """The full turn pipeline (retrieve -> cortex -> write-on-turn -> + metrics) for a specific family member. Shared by the legacy + /generate endpoint and /members/{id}/chat (Card 2).""" t_ingress = now_ns() payload_bytes = len((req.prompt or "").encode("utf-8")) if req.system: @@ -383,6 +399,7 @@ def generate( rres = embedder.memory_query( session_id=session_id, query=req.prompt, + member_id=member.id, k=RETRIEVAL_K, ) matches = rres.get("matches", []) or [] @@ -425,6 +442,12 @@ def generate( assistant_text=(result or {}).get("text", ""), turn_idx=turn_idx, ts=datetime.now(timezone.utc).isoformat(), + # Card 3: conversation turns default to the member's + # private scope; sharing is a promotion, never a write. + scope=f"private:{member.id}", + member_id=member.id, + origin="conversation", + participants=[auth.name], model_used=(result or {}).get("model", ""), user_token_count=usage.get("prompt_tokens", 0) or 0, assistant_token_count=usage.get("completion_tokens", 0) or 0, @@ -479,6 +502,7 @@ def generate( "memory_written": memory_written, "memory_chunks": memory_chunks, "token_name": auth.name, + "member_id": member.id, "error": err, }, ) @@ -723,15 +747,16 @@ def member_chat( else float(member.runtime.sampling_defaults.get("temperature", 0.7)) ) - result = generate( + result = _run_turn( GenerateRequest( prompt=req.prompt, system=effective_system, max_tokens=req.max_tokens, temperature=temperature, ), - x_session_id=session_id, + session_id=session_id, auth=auth, + member=member, ) if isinstance(result, JSONResponse): # Cortex-down 503: pass the Sprint 3c contract through unchanged. diff --git a/nodes/embedder_4070/scopes.py b/nodes/embedder_4070/scopes.py new file mode 100644 index 0000000..c66fade --- /dev/null +++ b/nodes/embedder_4070/scopes.py @@ -0,0 +1,113 @@ +# nodes/embedder_4070/scopes.py +""" +Memory scopes + provenance (Sprint 5, Card 3). + +Every memory row carries a scope, and retrieval is ALWAYS filtered +server-side to the querying member's visible set: + + private: + shared:household + experiential: + +Cross-member recall is forbidden by construction — the filter lives +here, in the service that owns the store, not in the callers. This +amends Sprint 2's deliberate no-filter design: cross-session recall +*within* a member survives; cross-member reads do not exist. + +Pure functions only (no Chroma, no model) so the privacy rules are +testable without loading anything heavy. +""" +from __future__ import annotations + +from typing import Any, Dict, List, Optional, Tuple + +SHARED_SCOPE = "shared:household" +HOUSEHOLD_MEMBER_ID = "household" + +# origin values a memory row can carry (V2 doc Section 4.3; the +# delegated_task origin lands with the concierge in V1.5 but the +# vocabulary is fixed now so rows never need re-labelling). +ORIGINS = ( + "conversation", + "sensor", + "promotion", + "vector_platform", + "delegated_task", +) + + +def private_scope(member_id: str) -> str: + return f"private:{member_id}" + + +def experiential_scope(member_id: str) -> str: + return f"experiential:{member_id}" + + +def readable_scopes(member_id: str) -> List[str]: + """Everything member `member_id` may ever retrieve. There is no + variant of this list without the filter — that's the point.""" + return [private_scope(member_id), SHARED_SCOPE, experiential_scope(member_id)] + + +def validate_write_scope(scope: str, member_id: str) -> None: + """A member may write to its own private/experiential scope or to + the shared household scope — never into another member's scopes.""" + if scope not in readable_scopes(member_id): + raise ValueError( + f"scope {scope!r} is not writable for member {member_id!r}; " + f"allowed: {readable_scopes(member_id)}" + ) + + +def validate_origin(origin: str) -> None: + if origin not in ORIGINS: + raise ValueError(f"unknown origin {origin!r}; expected one of {ORIGINS}") + + +def build_where( + member_id: str, + session_id_filter: Optional[str] = None, + exclude_parent_turn_id: Optional[str] = None, +) -> Dict[str, Any]: + """The Chroma `where` clause for a query by `member_id`. The scope + condition is unconditional; the others are opt-in refinements.""" + conds: List[Dict[str, Any]] = [{"scope": {"$in": readable_scopes(member_id)}}] + if session_id_filter: + conds.append({"session_id": session_id_filter}) + if exclude_parent_turn_id: + conds.append({"parent_turn_id": {"$ne": exclude_parent_turn_id}}) + return conds[0] if len(conds) == 1 else {"$and": conds} + + +def participants_to_meta(participants: Optional[List[str]]) -> str: + """Chroma metadata values must be scalars; participants are stored + as a comma-joined string, order preserved, blanks dropped.""" + return ",".join(p.strip() for p in (participants or []) if p and p.strip()) + + +def plan_scope_backfill( + ids: List[str], + metadatas: List[Optional[Dict[str, Any]]], + member_id: str, + participants: Optional[List[str]] = None, +) -> Tuple[List[str], List[Dict[str, Any]]]: + """Migration planner: given existing rows, return (ids, merged + metadatas) for exactly the rows that lack a scope. Pre-V2 rows are + grandfathered into `private:` with origin=conversation — + they were all 1-on-1 turns with member #1's predecessor. Rows that + already carry a scope are untouched (the migration is idempotent). + """ + update_ids: List[str] = [] + update_metas: List[Dict[str, Any]] = [] + for row_id, meta in zip(ids, metadatas): + meta = dict(meta or {}) + if meta.get("scope"): + continue + meta.update( + scope=private_scope(member_id), + member_id=member_id, + origin="conversation", + participants=participants_to_meta(participants), + ) + update_ids.append(row_id) + update_metas.append(meta) + return update_ids, update_metas diff --git a/nodes/embedder_4070/server.py b/nodes/embedder_4070/server.py index 455fcbb..2991d65 100644 --- a/nodes/embedder_4070/server.py +++ b/nodes/embedder_4070/server.py @@ -16,7 +16,16 @@ GET /health POST /embed -- raw embedding for legacy callers (proxy target) POST /memory/write -- write a completed turn (chunks if needed) - POST /memory/query -- top-k retrieval (no default session filter) + POST /memory/query -- top-k retrieval, scope-filtered per member + +Sprint 5 Card 3: every write carries scope/member_id/origin provenance +and every query is filtered server-side to the querying member's +visible scopes (private: + shared:household + +experiential:). Cross-member recall is structurally impossible +through this API; cross-session recall within a member is unchanged. +Pre-scope rows are invisible to queries until +scripts/migrate_memory_scopes.py grandfathers them into member #1's +private scope. """ from __future__ import annotations @@ -28,6 +37,7 @@ from .config import settings from . import chroma_store +from . import scopes from .chunker import chunk_turn from .embed import dim, embed_texts, get_model, tokenize @@ -41,7 +51,7 @@ "container from the brainstem for clean lifecycle and a " "swappable model boundary." ), - version="0.1.0", + version="0.2.0", ) @@ -67,6 +77,12 @@ class MemoryWriteRequest(BaseModel): assistant_token_count: int = 0 source_service: str = "" tool_calls_present: bool = False + # Sprint 5 Card 3 provenance. scope + member_id are mandatory — + # unscoped rows must never exist again after the migration. + scope: str + member_id: str + origin: str = "conversation" + participants: List[str] = Field(default_factory=list) class MemoryWriteResponse(BaseModel): @@ -78,9 +94,12 @@ class MemoryWriteResponse(BaseModel): class MemoryQueryRequest(BaseModel): query: str k: int = 5 - # Optional metadata filters. By default we do NOT scope to the - # caller's session, because the Sprint 2 done-criterion is - # cross-session recall. Filtering is opt-in. + # Sprint 5 Card 3: the querying member. Mandatory — the scope + # filter derived from it is applied server-side on every query. + # (Amends Sprint 2's no-filter design: cross-session recall within + # a member is preserved; cross-member recall is forbidden.) + member_id: str + # Optional refinements inside the member's visible scopes. session_id_filter: Optional[str] = None exclude_parent_turn_id: Optional[str] = None @@ -154,6 +173,15 @@ def memory_write( session_id = _resolve_session_id(x_session_id) parent_turn_id = f"{session_id}:{req.turn_idx}" + # Card 3: refuse writes that would break the scope rules — a member + # cannot write into another member's scopes, and every row must + # carry a known origin. + try: + scopes.validate_write_scope(req.scope, req.member_id) + scopes.validate_origin(req.origin) + except ValueError as exc: + raise HTTPException(status_code=400, detail=str(exc)) + body = _concatenate_turn(req.user_text, req.assistant_text) chunks = chunk_turn( @@ -187,6 +215,12 @@ def memory_write( "chunk_idx": idx, "chunk_total": total, "parent_turn_id": parent_turn_id, + # Card 3 provenance (participants is comma-joined because + # Chroma metadata values must be scalars). + "scope": req.scope, + "member_id": req.member_id, + "origin": req.origin, + "participants": scopes.participants_to_meta(req.participants), }) chroma_store.add_documents( @@ -208,19 +242,20 @@ def memory_query( x_session_id: Optional[str] = Header(None, alias="X-Session-Id"), ) -> MemoryQueryResponse: # session id is read but not required for reads; we log it for - # traceability. Cross-session retrieval is the whole point of - # Sprint 2's done-criterion. + # traceability. Cross-session retrieval within a member is the + # Sprint 2 done-criterion and still works — the Card 3 filter cuts + # across members, not across sessions. _ = x_session_id query_vec = embed_texts([req.query])[0] - where: Optional[Dict[str, Any]] = None - if req.session_id_filter: - where = {"session_id": req.session_id_filter} - # exclude_parent_turn_id is rarely used (we usually do not want to - # echo back the in-progress turn). Chroma supports $ne via where. - if req.exclude_parent_turn_id: - where = {**(where or {}), "parent_turn_id": {"$ne": req.exclude_parent_turn_id}} + # Card 3: the scope filter is built server-side from member_id and + # is never optional. Callers cannot widen it. + where = scopes.build_where( + req.member_id, + session_id_filter=req.session_id_filter, + exclude_parent_turn_id=req.exclude_parent_turn_id, + ) raw = chroma_store.query(query_vec, k=req.k, where=where) return MemoryQueryResponse( diff --git a/scripts/migrate_memory_scopes.py b/scripts/migrate_memory_scopes.py new file mode 100644 index 0000000..40b59d3 --- /dev/null +++ b/scripts/migrate_memory_scopes.py @@ -0,0 +1,123 @@ +#!/usr/bin/env python3 +""" +Sprint 5 Card 3 migration: grandfather pre-scope memory rows into +member #1's private scope. + +Every row written before the scoped-memory change lacks `scope` +metadata, which makes it invisible to the always-filtered query path. +This script backfills exactly those rows with: + + scope = private: + member_id = + origin = conversation + participants = <--participants, comma-joined; empty by default + because pre-V2 rows never recorded who spoke> + +Rows that already carry a scope are never touched — the migration is +idempotent and safe to re-run. Nothing is ever deleted. + +Designed to run inside the embedder container (it owns the Chroma +volume): + + docker compose exec embedder python scripts/migrate_memory_scopes.py # dry run + docker compose exec embedder python scripts/migrate_memory_scopes.py --apply # do it + +The dry run prints the reconciliation plan (total / already-scoped / +to-update) and exits nonzero if applying would still leave unscoped +rows (which would mean a logic error worth stopping for). +""" +from __future__ import annotations + +import argparse +import sys +from pathlib import Path + +REPO_ROOT = Path(__file__).resolve().parent.parent +sys.path.insert(0, str(REPO_ROOT / "nodes")) +sys.path.insert(0, str(REPO_ROOT)) + +from embedder_4070.scopes import plan_scope_backfill # noqa: E402 + +BATCH = 500 + + +def _load_registry_default_member() -> str: + from core.family import load_registry + + registry = load_registry(REPO_ROOT / "family" / "registry.yaml", REPO_ROOT) + return registry.members[0].id + + +def main() -> int: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument( + "--member", + help="member id to grandfather rows into (default: first registry member)", + ) + parser.add_argument( + "--participants", + default="", + help="comma-separated participants to stamp on migrated rows (default: none)", + ) + parser.add_argument( + "--persist-dir", + help="Chroma persist directory (default: embedder service setting)", + ) + parser.add_argument( + "--collection", + help="collection name (default: embedder service setting)", + ) + parser.add_argument( + "--apply", + action="store_true", + help="actually write the updates; without this flag it's a dry run", + ) + args = parser.parse_args() + + member_id = args.member or _load_registry_default_member() + participants = [p for p in args.participants.split(",") if p.strip()] + + import chromadb + from embedder_4070.config import settings + + persist_dir = args.persist_dir or settings.chroma_persist_dir + collection_name = args.collection or settings.chroma_collection + + client = chromadb.PersistentClient(path=persist_dir) + coll = client.get_or_create_collection(name=collection_name) + + total = coll.count() + print(f"collection {collection_name!r} at {persist_dir}: {total} rows") + + updated = 0 + already_scoped = 0 + offset = 0 + while offset < total: + page = coll.get(limit=BATCH, offset=offset, include=["metadatas"]) + ids = page.get("ids") or [] + metas = page.get("metadatas") or [] + if not ids: + break + offset += len(ids) + + upd_ids, upd_metas = plan_scope_backfill(ids, metas, member_id, participants) + already_scoped += len(ids) - len(upd_ids) + updated += len(upd_ids) + if upd_ids and args.apply: + coll.update(ids=upd_ids, metadatas=upd_metas) + + verb = "updated" if args.apply else "would update" + print( + f"reconciliation: total={total} already_scoped={already_scoped} " + f"{verb}={updated} -> scope=private:{member_id}" + ) + if already_scoped + updated != total: + print("ERROR: rows unaccounted for — aborting; nothing further changed.") + return 1 + if not args.apply: + print("dry run only; re-run with --apply to write.") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/tests/test_memory_scopes.py b/tests/test_memory_scopes.py new file mode 100644 index 0000000..a09b0ac --- /dev/null +++ b/tests/test_memory_scopes.py @@ -0,0 +1,189 @@ +"""Sprint 5 Card 3: scoped memory + provenance. + +Done-criteria under test: + - the server-side scope filter is always present and never widenable + (a member's query can only ever see private:M + shared:household + + experiential:M); + - write-on-turn stamps provenance and lands in the member's private + scope, for both /members/{id}/chat and the legacy /generate path; + - cross-member writes are rejected; + - the migration planner backfills exactly the unscoped rows + (idempotent, copy-nothing, delete-nothing). + +The scope rules are pure functions in embedder_4070.scopes so they get +tested without loading the embedding model. +""" +from __future__ import annotations + +import importlib +import sys +from pathlib import Path +from typing import Iterator + +import pytest + +from embedder_4070.scopes import ( + build_where, + plan_scope_backfill, + readable_scopes, + validate_origin, + validate_write_scope, +) + + +# --------------------------------------------------------------------------- +# Pure scope rules +# --------------------------------------------------------------------------- + + +def test_readable_scopes_are_exactly_three(): + assert readable_scopes("vera") == [ + "private:vera", "shared:household", "experiential:vera", + ] + + +def test_where_clause_always_carries_the_scope_filter(): + where = build_where("vera") + assert where == {"scope": {"$in": readable_scopes("vera")}} + + refined = build_where("vera", session_id_filter="sess_1", + exclude_parent_turn_id="sess_1:3") + assert "$and" in refined + # The scope condition survives every refinement. + assert refined["$and"][0] == {"scope": {"$in": readable_scopes("vera")}} + + +def test_another_members_private_scope_is_never_readable(): + for scope in readable_scopes("vera"): + assert "juno" not in scope + + +def test_cross_member_writes_rejected(): + validate_write_scope("private:vera", "vera") + validate_write_scope("shared:household", "vera") + with pytest.raises(ValueError): + validate_write_scope("private:juno", "vera") + with pytest.raises(ValueError): + validate_write_scope("experiential:juno", "vera") + + +def test_unknown_origin_rejected(): + validate_origin("conversation") + validate_origin("promotion") + with pytest.raises(ValueError): + validate_origin("osmosis") + + +# --------------------------------------------------------------------------- +# Migration planner +# --------------------------------------------------------------------------- + + +def test_backfill_touches_only_unscoped_rows(): + ids = ["a", "b", "c"] + metas = [ + {"session_id": "s1"}, # pre-scope row + {"session_id": "s2", "scope": "private:vera"}, # already migrated + None, # degenerate row + ] + upd_ids, upd_metas = plan_scope_backfill(ids, metas, "vera", ["drew"]) + assert upd_ids == ["a", "c"] + for meta in upd_metas: + assert meta["scope"] == "private:vera" + assert meta["member_id"] == "vera" + assert meta["origin"] == "conversation" + assert meta["participants"] == "drew" + # Original metadata keys survive the merge. + assert upd_metas[0]["session_id"] == "s1" + + +def test_backfill_is_idempotent(): + ids = ["a"] + metas = [{"session_id": "s1"}] + upd_ids, upd_metas = plan_scope_backfill(ids, metas, "vera") + again_ids, _ = plan_scope_backfill(upd_ids, upd_metas, "vera") + assert again_ids == [] + + +# --------------------------------------------------------------------------- +# Hub integration: provenance flows through both chat paths +# --------------------------------------------------------------------------- + + +@pytest.fixture +def hub(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Iterator: + """Same boot recipe as tests/test_member_routing.py, but the + embedder stubs capture the memory_write / memory_query kwargs.""" + monkeypatch.setenv("BRAINSTEM_TOKEN_STORE_PATH", str(tmp_path / "tokens.json")) + monkeypatch.setenv("BRAINSTEM_METRICS_PATH", str(tmp_path / "metrics.jsonl")) + monkeypatch.setenv("BRAINSTEM_SESSION_STORE_PATH", str(tmp_path / "sessions.json")) + + for mod in list(sys.modules): + if mod.startswith("brainstem_4070"): + del sys.modules[mod] + + server = importlib.import_module("brainstem_4070.server") + server.configure_store(tmp_path / "tokens.json") + + captured = {"write": None, "query": None} + + def fake_memory_query(**kwargs): + captured["query"] = kwargs + return {"matches": []} + + def fake_memory_write(**kwargs): + captured["write"] = kwargs + return {"ok": True, "chunks": 1} + + monkeypatch.setattr(server.embedder, "memory_query", fake_memory_query) + monkeypatch.setattr(server.embedder, "memory_write", fake_memory_write) + monkeypatch.setattr(server.embedder, "health", lambda: {"reachable": True}) + monkeypatch.setattr(server.cortex, "generate", lambda **_: { + "text": "stub", "model": "stub-model", "finish_reason": "stop", + "usage": {"prompt_tokens": 1, "completion_tokens": 2}, + }) + + from fastapi.testclient import TestClient + + with TestClient(server.app) as client: + from brainstem_4070.auth import TokenStore + + token, _ = TokenStore.load(tmp_path / "tokens.json").create("drew") + yield client, token, captured + + +def test_member_chat_writes_private_scope_with_provenance(hub): + client, token, captured = hub + res = client.post( + "/members/vera/chat", + headers={"Authorization": f"Bearer {token}"}, + json={"prompt": "remember the dog went out at 5"}, + ) + assert res.status_code == 200, res.text + + write = captured["write"] + assert write["scope"] == "private:vera" + assert write["member_id"] == "vera" + assert write["origin"] == "conversation" + assert write["participants"] == ["drew"] + + query = captured["query"] + assert query["member_id"] == "vera" + + +def test_legacy_generate_runs_as_default_member(hub): + """/generate is the hub's default member now — no unscoped writes + remain anywhere in the system.""" + client, token, captured = hub + res = client.post( + "/generate", + headers={ + "Authorization": f"Bearer {token}", + "X-Session-Id": "sess_legacy", + }, + json={"prompt": "hi"}, + ) + assert res.status_code == 200, res.text + assert captured["write"]["scope"] == "private:vera" + assert captured["write"]["member_id"] == "vera" + assert captured["query"]["member_id"] == "vera" From fbbc25a32aef276b52cc2a821787bf6d8215be17 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 25 Jul 2026 22:29:59 +0000 Subject: [PATCH 05/20] =?UTF-8?q?feat(memory):=20Card=206=20=E2=80=94=20pr?= =?UTF-8?q?omotion=20endpoint=20(copy-never-move)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit POST /members/{id}/memory/promote on the hub confirms a share: reaching the endpoint is the person's yes (only people hold bearer tokens), implementing decision 3's offer-then-confirm. The embedder's /memory/promote copies the private row into shared:household with the permanent paper trail — origin=promotion, promoted_from, promoted_from_member, promoted_by — reusing the stored embedding so the copy is vector-identical. The private original is never modified, and the shared copy's deterministic id makes promotion idempotent per source row. A member can only promote out of its own private scope. 55 tests pass (6 new). Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_011bnTL8M5fY9bMNjvzuyvyp --- nodes/brainstem_4070/embedder_client.py | 12 ++ nodes/brainstem_4070/server.py | 43 ++++++++ nodes/embedder_4070/chroma_store.py | 7 ++ nodes/embedder_4070/scopes.py | 40 +++++++ nodes/embedder_4070/server.py | 75 ++++++++++++- tests/test_memory_promotion.py | 141 ++++++++++++++++++++++++ 6 files changed, 317 insertions(+), 1 deletion(-) create mode 100644 tests/test_memory_promotion.py diff --git a/nodes/brainstem_4070/embedder_client.py b/nodes/brainstem_4070/embedder_client.py index 3042252..3b33763 100644 --- a/nodes/brainstem_4070/embedder_client.py +++ b/nodes/brainstem_4070/embedder_client.py @@ -87,6 +87,18 @@ def memory_write( } return self._post("/memory/write", payload, headers={"X-Session-Id": session_id}) + def memory_promote( + self, + member_id: str, + memory_id: str, + promoted_by: str, + ) -> Dict[str, Any]: + return self._post("/memory/promote", { + "member_id": member_id, + "memory_id": memory_id, + "promoted_by": promoted_by, + }) + def memory_query( self, session_id: str, diff --git a/nodes/brainstem_4070/server.py b/nodes/brainstem_4070/server.py index f676ae8..35df60a 100644 --- a/nodes/brainstem_4070/server.py +++ b/nodes/brainstem_4070/server.py @@ -670,6 +670,48 @@ def member_inbox_message( } +class MemoryPromoteRequest(BaseModel): + memory_id: str + + +@app.post("/members/{member_id}/memory/promote") +def member_memory_promote( + member_id: str, + req: MemoryPromoteRequest, + auth: TokenEntry = Depends(require_token), +): + """Promote one of this member's private memories into + shared:household (Sprint 5 Card 6). + + Offer-then-confirm, per decision 3: a member may *offer* to share + in conversation, but this call is the person's confirmation — only + people hold bearer tokens, so reaching this endpoint IS the yes. + The embedder copies (never moves) the row and stamps the paper + trail: origin=promotion, promoted_from, promoted_by.""" + _member_or_404(member_id) + try: + result = embedder.memory_promote( + member_id=member_id, + memory_id=req.memory_id, + promoted_by=auth.name, + ) + except EmbedderError as exc: + detail = str(exc) + # Surface the embedder's own 4xx verdicts (unknown row, wrong + # scope) as client errors rather than a blanket 502. + if "404" in detail: + raise HTTPException(status_code=404, detail=f"no memory row {req.memory_id!r}") + if "400" in detail: + raise HTTPException(status_code=400, detail=detail) + raise HTTPException(status_code=502, detail=f"embedder unreachable: {exc}") + logger.info( + "promotion: member=%s row=%s by=%s -> %s (already=%s)", + member_id, req.memory_id, auth.name, + result.get("promoted_id"), result.get("already_promoted"), + ) + return result + + @app.post("/members/{member_id}/chat") def member_chat( member_id: str, @@ -934,6 +976,7 @@ def root(): "/members/{member_id}/chat", "/members/{member_id}/presence", "/members/{member_id}/inbox/{msg_id}", + "/members/{member_id}/memory/promote", ], }, "member_loading_contract": { diff --git a/nodes/embedder_4070/chroma_store.py b/nodes/embedder_4070/chroma_store.py index a5be087..2048374 100644 --- a/nodes/embedder_4070/chroma_store.py +++ b/nodes/embedder_4070/chroma_store.py @@ -88,5 +88,12 @@ def query( return out +def get_by_ids(ids: List[str]) -> Dict[str, Any]: + """Fetch rows by id including embeddings, so a promotion can copy a + row without re-embedding (same vector, same model, by definition).""" + coll = get_collection() + return coll.get(ids=ids, include=["documents", "metadatas", "embeddings"]) + + def count() -> int: return get_collection().count() diff --git a/nodes/embedder_4070/scopes.py b/nodes/embedder_4070/scopes.py index c66fade..d07255d 100644 --- a/nodes/embedder_4070/scopes.py +++ b/nodes/embedder_4070/scopes.py @@ -84,6 +84,46 @@ def participants_to_meta(participants: Optional[List[str]]) -> str: return ",".join(p.strip() for p in (participants or []) if p and p.strip()) +def promoted_id(source_id: str) -> str: + """Deterministic id for the shared copy of a promoted row. Same + source promoted twice lands on the same id — promotion is + idempotent per source row.""" + return f"{source_id}::promoted" + + +def build_promotion_metadata( + source_meta: Dict[str, Any], + member_id: str, + promoted_by: str, + source_id: str, +) -> Dict[str, Any]: + """Metadata for the shared:household copy of a private row. + + Promotion copies, never moves (V2 Section 4.3): the private + original is untouched and the copy carries the permanent paper + trail — origin=promotion, promoted_from, promoted_by, and which + member's private scope it came from. V1 promotes from the member's + private scope only; experiential promotion arrives with Project + Vector if ever. + """ + source_scope = (source_meta or {}).get("scope") + if source_scope != private_scope(member_id): + raise ValueError( + f"row {source_id!r} has scope {source_scope!r}; only rows in " + f"{private_scope(member_id)!r} can be promoted by member {member_id!r}" + ) + meta = dict(source_meta) + meta.update( + scope=SHARED_SCOPE, + member_id=HOUSEHOLD_MEMBER_ID, + origin="promotion", + promoted_from=source_id, + promoted_from_member=member_id, + promoted_by=promoted_by, + ) + return meta + + def plan_scope_backfill( ids: List[str], metadatas: List[Optional[Dict[str, Any]]], diff --git a/nodes/embedder_4070/server.py b/nodes/embedder_4070/server.py index 2991d65..8bb643d 100644 --- a/nodes/embedder_4070/server.py +++ b/nodes/embedder_4070/server.py @@ -115,6 +115,22 @@ class MemoryQueryResponse(BaseModel): matches: List[MemoryMatch] +class MemoryPromoteRequest(BaseModel): + # The member whose private row is being shared, the row, and the + # person who confirmed the share. The confirmation itself happens + # at the hub (only people hold bearer tokens); by the time this + # service sees the request, consent is established. + member_id: str + memory_id: str + promoted_by: str + + +class MemoryPromoteResponse(BaseModel): + promoted_id: str + promoted_from: str + already_promoted: bool + + class HealthResponse(BaseModel): status: str model_loaded: bool @@ -263,11 +279,68 @@ def memory_query( ) +@app.post("/memory/promote", response_model=MemoryPromoteResponse) +def memory_promote(req: MemoryPromoteRequest) -> MemoryPromoteResponse: + """Copy — never move — a private row into shared:household with the + full promotion paper trail (Sprint 5 Card 6, V2 Section 4.3). + + Idempotent per source row: the shared copy has a deterministic id, + and re-promoting an already-promoted row is a no-op that reports + `already_promoted`. The private original is never modified.""" + target_id = scopes.promoted_id(req.memory_id) + + existing = chroma_store.get_by_ids([target_id]) + if existing.get("ids"): + return MemoryPromoteResponse( + promoted_id=target_id, + promoted_from=req.memory_id, + already_promoted=True, + ) + + source = chroma_store.get_by_ids([req.memory_id]) + if not source.get("ids"): + raise HTTPException(status_code=404, detail=f"no memory row {req.memory_id!r}") + + source_meta = (source.get("metadatas") or [None])[0] or {} + try: + new_meta = scopes.build_promotion_metadata( + source_meta, req.member_id, req.promoted_by, req.memory_id + ) + except ValueError as exc: + raise HTTPException(status_code=400, detail=str(exc)) + + document = (source.get("documents") or [""])[0] + embedding = (source.get("embeddings") or [None])[0] + if embedding is None: + raise HTTPException( + status_code=500, detail=f"row {req.memory_id!r} has no stored embedding" + ) + + chroma_store.add_documents( + ids=[target_id], + documents=[document], + embeddings=[list(embedding)], + metadatas=[new_meta], + ) + logger.info( + "memory_promote %s -> %s (member=%s, by=%s)", + req.memory_id, target_id, req.member_id, req.promoted_by, + ) + return MemoryPromoteResponse( + promoted_id=target_id, + promoted_from=req.memory_id, + already_promoted=False, + ) + + @app.get("/") def root() -> Dict[str, Any]: return { "service": "Nexus Embedder (4070)", "model": settings.model_name, "chroma_collection": settings.chroma_collection, - "endpoints": ["/health", "/embed", "/memory/write", "/memory/query"], + "endpoints": [ + "/health", "/embed", "/memory/write", "/memory/query", + "/memory/promote", + ], } diff --git a/tests/test_memory_promotion.py b/tests/test_memory_promotion.py new file mode 100644 index 0000000..1bd7078 --- /dev/null +++ b/tests/test_memory_promotion.py @@ -0,0 +1,141 @@ +"""Sprint 5 Card 6: promotion — private -> shared:household. + +Done-criteria under test: + - the shared copy carries the full paper trail and lands in + shared:household (visible to any member's read filter); + - the private original is untouched (copy, never move); + - promotion is idempotent per source row (deterministic id); + - a member can only promote out of its own private scope; + - the hub endpoint stamps promoted_by from token attribution. +""" +from __future__ import annotations + +import importlib +import sys +from pathlib import Path +from typing import Iterator + +import pytest + +from embedder_4070.scopes import ( + build_promotion_metadata, + promoted_id, + readable_scopes, +) + + +# --------------------------------------------------------------------------- +# Pure promotion rules +# --------------------------------------------------------------------------- + + +def test_promoted_copy_lands_in_shared_scope_with_paper_trail(): + source = { + "scope": "private:vera", + "member_id": "vera", + "origin": "conversation", + "participants": "drew", + "session_id": "s1", + "ts": "2026-07-25T22:00:00+00:00", + } + meta = build_promotion_metadata(source, "vera", "drew", "s1:0:0") + assert meta["scope"] == "shared:household" + assert meta["member_id"] == "household" + assert meta["origin"] == "promotion" + assert meta["promoted_from"] == "s1:0:0" + assert meta["promoted_from_member"] == "vera" + assert meta["promoted_by"] == "drew" + # Non-provenance metadata (ts, session) survives the copy. + assert meta["ts"] == source["ts"] + # The source dict is not mutated — copy, never move. + assert source["scope"] == "private:vera" + + +def test_promoted_copy_is_visible_to_every_member(): + meta = build_promotion_metadata( + {"scope": "private:vera"}, "vera", "drew", "row1" + ) + for member in ("vera", "juno", "anyone"): + assert meta["scope"] in readable_scopes(member) + + +def test_cannot_promote_out_of_someone_elses_scope(): + with pytest.raises(ValueError): + build_promotion_metadata({"scope": "private:juno"}, "vera", "drew", "row1") + with pytest.raises(ValueError): + build_promotion_metadata({"scope": "shared:household"}, "vera", "drew", "row1") + with pytest.raises(ValueError): + build_promotion_metadata({}, "vera", "drew", "row1") + + +def test_promoted_id_is_deterministic(): + assert promoted_id("s1:0:0") == promoted_id("s1:0:0") + assert promoted_id("s1:0:0") != promoted_id("s1:0:1") + + +# --------------------------------------------------------------------------- +# Hub endpoint +# --------------------------------------------------------------------------- + + +@pytest.fixture +def hub(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Iterator: + monkeypatch.setenv("BRAINSTEM_TOKEN_STORE_PATH", str(tmp_path / "tokens.json")) + monkeypatch.setenv("BRAINSTEM_METRICS_PATH", str(tmp_path / "metrics.jsonl")) + monkeypatch.setenv("BRAINSTEM_SESSION_STORE_PATH", str(tmp_path / "sessions.json")) + + for mod in list(sys.modules): + if mod.startswith("brainstem_4070"): + del sys.modules[mod] + + server = importlib.import_module("brainstem_4070.server") + server.configure_store(tmp_path / "tokens.json") + + captured = {} + + def fake_promote(**kwargs): + captured.update(kwargs) + return { + "promoted_id": promoted_id(kwargs["memory_id"]), + "promoted_from": kwargs["memory_id"], + "already_promoted": False, + } + + monkeypatch.setattr(server.embedder, "memory_promote", fake_promote) + + from fastapi.testclient import TestClient + + with TestClient(server.app) as client: + from brainstem_4070.auth import TokenStore + + token, _ = TokenStore.load(tmp_path / "tokens.json").create("drew") + yield client, token, captured + + +def test_hub_promotion_stamps_person_from_token(hub): + client, token, captured = hub + res = client.post( + "/members/vera/memory/promote", + headers={"Authorization": f"Bearer {token}"}, + json={"memory_id": "sess_abc:3:0"}, + ) + assert res.status_code == 200, res.text + body = res.json() + assert body["promoted_id"] == promoted_id("sess_abc:3:0") + assert captured == { + "member_id": "vera", + "memory_id": "sess_abc:3:0", + "promoted_by": "drew", + } + + +def test_hub_promotion_requires_auth_and_known_member(hub): + client, token, _captured = hub + assert client.post( + "/members/vera/memory/promote", json={"memory_id": "x"} + ).status_code == 401 + assert client.post( + "/members/nobody/memory/promote", + headers={"Authorization": f"Bearer {token}"}, + json={"memory_id": "x"}, + ).status_code == 404 From c6c688aa779512fa4e33cc28f1785c839898066b Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 25 Jul 2026 22:32:30 +0000 Subject: [PATCH 06/20] =?UTF-8?q?feat(inbox):=20Card=205=20=E2=80=94=20dur?= =?UTF-8?q?able=20inbox,=20drained=20on=20wake?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The inbox v0 from Card 2 becomes durable: queued messages persist to a JSON store with the same atomic-write discipline as the token and session stores, so a hub restart loses nothing. When a member's presence flips to awake, the hub drains its queue oldest-first through the exact same path as live chat — same hub-minted (person, member) session, same memory scoping, same metrics — so a message sent while the member slept is answered as if the sender had waited, and the reply is retrievable by msg_id. Custody is not memory: nothing touches any scope until the member actually processes the turn (covered by a test that counts memory writes across the queue/drain boundary). A cortex failure mid-drain leaves the remaining messages queued for the next wake. 58 tests pass (3 new). Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_011bnTL8M5fY9bMNjvzuyvyp --- nodes/brainstem_4070/config.py | 3 + nodes/brainstem_4070/family_state.py | 93 ++++++++++++++-- nodes/brainstem_4070/server.py | 122 ++++++++++++++++----- tests/test_inbox_drain.py | 155 +++++++++++++++++++++++++++ tests/test_member_routing.py | 1 + 5 files changed, 338 insertions(+), 36 deletions(-) create mode 100644 tests/test_inbox_drain.py diff --git a/nodes/brainstem_4070/config.py b/nodes/brainstem_4070/config.py index 5c3b2bb..bf6ac19 100644 --- a/nodes/brainstem_4070/config.py +++ b/nodes/brainstem_4070/config.py @@ -73,6 +73,9 @@ class Settings(BaseSettings): # Retry-After for the member_loading 503 — weights staging plus a # llama.cpp load is tens of seconds, not the cortex-down 5s. member_loading_retry_after_seconds: int = 20 + # Durable inbox (Card 5): queued messages survive hub restarts. + # Docker named volume in production, like the token store. + inbox_store_path: str = "/data/inbox/inbox.json" # Service # Inside the container the brainstem listens on 0.0.0.0 so compose- diff --git a/nodes/brainstem_4070/family_state.py b/nodes/brainstem_4070/family_state.py index 90d50f7..01bd586 100644 --- a/nodes/brainstem_4070/family_state.py +++ b/nodes/brainstem_4070/family_state.py @@ -1,44 +1,96 @@ # nodes/brainstem_4070/family_state.py """ -Per-member presence + inbox v0 (Sprint 5, Card 2). +Per-member presence + durable inbox (Sprint 5, Cards 2 + 5). Presence is owned by the model manager (Card 4); until it exists, the hub keeps this in-memory store and exposes an authenticated endpoint -for the manager — or an operator — to set it. The inbox here is the -in-memory v0 behind the 202-queued contract; Card 5 makes it durable -and drains it on wake. Queued messages are custody, not memory: nothing -touches any memory scope until the member actually processes the turn. +for the manager — or an operator — to set it. + +The inbox is durable (Card 5): queued messages persist to a JSON file +with the same atomic-write discipline as the token and session stores, +so a hub restart loses nothing. On wake, the hub drains a member's +queue in arrival order through the normal chat path. Queued messages +are custody, not memory: nothing touches any memory scope until the +member actually processes the turn. """ from __future__ import annotations +import json +import logging +import os import threading import uuid from datetime import datetime, timezone +from pathlib import Path from typing import Dict, List, Optional from core.family import PRESENCE_STATES +logger = logging.getLogger("brainstem_4070.family_state") + class UnknownMemberError(KeyError): pass class FamilyState: - """Thread-safe presence + queue state for every registry member.""" + """Thread-safe presence + durable queue state for every registry + member. Presence is runtime state (starts fresh each boot); the + inbox is persistent.""" - def __init__(self, member_ids: List[str], default_presence: str = "awake"): + def __init__( + self, + member_ids: List[str], + default_presence: str = "awake", + store_path: Optional[os.PathLike | str] = None, + ): if default_presence not in PRESENCE_STATES: raise ValueError(f"invalid default presence {default_presence!r}") self._lock = threading.Lock() self._presence: Dict[str, str] = {m: default_presence for m in member_ids} # msg_id -> record, insertion-ordered per member (dicts preserve - # insertion order, which is the drain order Card 5 needs). + # insertion order, which is the drain order). self._inbox: Dict[str, Dict[str, dict]] = {m: {} for m in member_ids} + self._store_path = Path(store_path) if store_path else None + self._load() def _check(self, member_id: str) -> None: if member_id not in self._presence: raise UnknownMemberError(member_id) + # -- persistence --------------------------------------------------- + + def _load(self) -> None: + if self._store_path is None or not self._store_path.is_file(): + return + try: + raw = json.loads(self._store_path.read_text(encoding="utf-8")) + except (OSError, json.JSONDecodeError) as exc: + logger.error("inbox store unreadable (%s): %s — starting empty", + self._store_path, exc) + return + if not isinstance(raw, dict): + return + for member_id, msgs in raw.items(): + # Messages for members no longer in the registry are kept on + # disk (never silently dropped) but not loaded. + if member_id in self._inbox and isinstance(msgs, dict): + self._inbox[member_id] = msgs + + def _flush_locked(self) -> None: + """Write the inbox to disk. Caller holds the lock. A flush + failure must not fail the request (same posture as the token + and session stores).""" + if self._store_path is None: + return + try: + self._store_path.parent.mkdir(parents=True, exist_ok=True) + tmp = self._store_path.with_suffix(self._store_path.suffix + ".tmp") + tmp.write_text(json.dumps(self._inbox, indent=2), encoding="utf-8") + os.replace(tmp, self._store_path) + except OSError as exc: + logger.warning("inbox store flush failed: %s", exc) + # -- presence ---------------------------------------------------------- def presence(self, member_id: str) -> str: @@ -54,7 +106,7 @@ def set_presence(self, member_id: str, state: str) -> None: with self._lock: self._presence[member_id] = state - # -- inbox v0 ---------------------------------------------------------- + # -- inbox --------------------------------------------------------- def enqueue( self, @@ -83,6 +135,7 @@ def enqueue( "queued_at": datetime.now(timezone.utc).isoformat(), "result": None, } + self._flush_locked() return msg_id def queue_depth(self, member_id: str) -> int: @@ -93,8 +146,30 @@ def queue_depth(self, member_id: str) -> int: if m["status"] == "queued" ) + def queued_messages(self, member_id: str) -> List[dict]: + """Snapshot of pending messages in arrival order (drain order).""" + self._check(member_id) + with self._lock: + return [ + dict(m) for m in self._inbox[member_id].values() + if m["status"] == "queued" + ] + def get_message(self, member_id: str, msg_id: str) -> Optional[dict]: self._check(member_id) with self._lock: record = self._inbox[member_id].get(msg_id) return dict(record) if record else None + + def complete_message(self, member_id: str, msg_id: str, result: dict) -> None: + """Mark a queued message answered, with the reply attached for + the sender's poll.""" + self._check(member_id) + with self._lock: + record = self._inbox[member_id].get(msg_id) + if record is None: + return + record["status"] = "answered" + record["result"] = result + record["answered_at"] = datetime.now(timezone.utc).isoformat() + self._flush_locked() diff --git a/nodes/brainstem_4070/server.py b/nodes/brainstem_4070/server.py index 35df60a..047f8b0 100644 --- a/nodes/brainstem_4070/server.py +++ b/nodes/brainstem_4070/server.py @@ -62,6 +62,7 @@ family_state = FamilyState( [m.id for m in family_registry.members], default_presence=settings.member_default_presence, + store_path=settings.inbox_store_path, ) hub_sessions = HubSessionStore(settings.session_store_path) @@ -592,6 +593,89 @@ def _member_or_404(member_id: str): raise HTTPException(status_code=404, detail=f"unknown member '{member_id}'") +def _live_member_turn( + member, + auth: TokenEntry, + prompt: str, + system: Optional[str], + max_tokens: int, + temperature: Optional[float], +): + """One live turn with a member on the caller's hub-minted session: + spec as base system, caller system layered after, registry sampling + defaults when the caller didn't override. Shared by live chat and + the inbox drain. Returns (session_id, result) where result is a + GenerateResponse or the cortex-down JSONResponse.""" + session_id, stored_turn_idx = hub_sessions.get_or_mint(auth.name, member.id) + if session_id not in _turn_idx_by_session: + # First turn since a restart: seed the runtime counter from the + # durable copy instead of silently restarting at 0. + _turn_idx_by_session[session_id] = stored_turn_idx + + spec = member_specs[member.id] + caller_system = (system or "").strip() + effective_system = f"{spec}\n\n{caller_system}" if caller_system else spec + effective_temperature = ( + temperature + if temperature is not None + else float(member.runtime.sampling_defaults.get("temperature", 0.7)) + ) + + result = _run_turn( + GenerateRequest( + prompt=prompt, + system=effective_system, + max_tokens=max_tokens, + temperature=effective_temperature, + ), + session_id=session_id, + auth=auth, + member=member, + ) + if not isinstance(result, JSONResponse): + hub_sessions.record_turn(auth.name, member.id, _turn_idx_by_session[session_id]) + return session_id, result + + +def _drain_inbox(member_id: str) -> int: + """Card 5: answer a freshly-awake member's queued messages in + arrival order through the normal chat path — same sessions, same + memory scoping, same metrics as a live turn. Stops early if the + cortex goes down mid-drain (messages stay queued for the next + wake). Returns how many messages were answered.""" + member = family_registry.get(member_id) + answered = 0 + for msg in family_state.queued_messages(member_id): + # The sender authenticated when the message was queued; the + # drain runs on their behalf with the recorded attribution. + sender = TokenEntry(name=msg["person"], hash="", created_at="") + session_id, result = _live_member_turn( + member, + sender, + prompt=msg["prompt"], + system=msg["system"], + max_tokens=msg["max_tokens"], + temperature=msg["temperature"], + ) + if isinstance(result, JSONResponse): + logger.warning( + "inbox drain for member=%s stopped at msg=%s: cortex down", + member_id, msg["msg_id"], + ) + break + family_state.complete_message(member_id, msg["msg_id"], result={ + "text": result.text, + "model": result.model, + "session_id": session_id, + "turn_idx": result.turn_idx, + "memory_written": result.memory_written, + }) + answered += 1 + if answered: + logger.info("inbox drain: member=%s answered=%d", member_id, answered) + return answered + + def _member_summary(member) -> dict: return { "id": member.id, @@ -644,7 +728,10 @@ def member_presence( "presence: member=%s -> %s (set by token=%s)", member_id, req.presence, auth.name, ) - return {"member_id": member_id, "presence": req.presence} + # Card 5: waking up means answering what accumulated while asleep, + # oldest first, before anything else happens. + drained = _drain_inbox(member_id) if req.presence == "awake" else 0 + return {"member_id": member_id, "presence": req.presence, "drained": drained} @app.get("/members/{member_id}/inbox/{msg_id}") @@ -774,37 +861,18 @@ def member_chat( ) # Awake: run the turn live on the hub-minted session. - session_id, stored_turn_idx = hub_sessions.get_or_mint(auth.name, member_id) - if session_id not in _turn_idx_by_session: - # First turn since a restart: seed the runtime counter from the - # durable copy instead of silently restarting at 0. - _turn_idx_by_session[session_id] = stored_turn_idx - - spec = member_specs[member_id] - caller_system = (req.system or "").strip() - effective_system = f"{spec}\n\n{caller_system}" if caller_system else spec - temperature = ( - req.temperature - if req.temperature is not None - else float(member.runtime.sampling_defaults.get("temperature", 0.7)) - ) - - result = _run_turn( - GenerateRequest( - prompt=req.prompt, - system=effective_system, - max_tokens=req.max_tokens, - temperature=temperature, - ), - session_id=session_id, - auth=auth, - member=member, + session_id, result = _live_member_turn( + member, + auth, + prompt=req.prompt, + system=req.system, + max_tokens=req.max_tokens, + temperature=req.temperature, ) if isinstance(result, JSONResponse): # Cortex-down 503: pass the Sprint 3c contract through unchanged. return result - hub_sessions.record_turn(auth.name, member_id, _turn_idx_by_session[session_id]) return MemberChatResponse( member_id=member_id, display_name=member.display_name, diff --git a/tests/test_inbox_drain.py b/tests/test_inbox_drain.py new file mode 100644 index 0000000..a74b0b7 --- /dev/null +++ b/tests/test_inbox_drain.py @@ -0,0 +1,155 @@ +"""Sprint 5 Card 5: durable inbox + drain on wake. + +Done-criteria under test: + - a message sent while the member is asleep is answered after wake, + on the sender's own (person, member) session — correct continuity; + - the answer is retrievable by msg_id; + - queued messages survive a hub restart (durable custody); + - a cortex failure mid-drain leaves the remaining messages queued; + - custody is not memory: nothing is written to any scope until the + member actually processes the turn. +""" +from __future__ import annotations + +import importlib +import sys +from pathlib import Path +from typing import Iterator + +import pytest + + +@pytest.fixture +def hub(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Iterator: + monkeypatch.setenv("BRAINSTEM_TOKEN_STORE_PATH", str(tmp_path / "tokens.json")) + monkeypatch.setenv("BRAINSTEM_METRICS_PATH", str(tmp_path / "metrics.jsonl")) + monkeypatch.setenv("BRAINSTEM_SESSION_STORE_PATH", str(tmp_path / "sessions.json")) + monkeypatch.setenv("BRAINSTEM_INBOX_STORE_PATH", str(tmp_path / "inbox.json")) + + for mod in list(sys.modules): + if mod.startswith("brainstem_4070"): + del sys.modules[mod] + + server = importlib.import_module("brainstem_4070.server") + server.configure_store(tmp_path / "tokens.json") + + class Control: + cortex_down = False + + control = Control() + writes = [] + + def fake_cortex_generate(**kwargs): + if control.cortex_down: + from brainstem_4070.cortex_client import CortexError + raise CortexError("POST http://stubbed failed: Connection refused") + return { + "text": f"reply to: {kwargs.get('prompt')}", + "model": "stub-model", + "finish_reason": "stop", + "usage": {"prompt_tokens": 1, "completion_tokens": 2}, + } + + def fake_memory_write(**kwargs): + writes.append(kwargs) + return {"ok": True, "chunks": 1} + + monkeypatch.setattr(server.embedder, "memory_query", lambda **_: {"matches": []}) + monkeypatch.setattr(server.embedder, "memory_write", fake_memory_write) + monkeypatch.setattr(server.embedder, "health", lambda: {"reachable": True}) + monkeypatch.setattr(server.cortex, "generate", fake_cortex_generate) + + from fastapi.testclient import TestClient + + with TestClient(server.app) as client: + from brainstem_4070.auth import TokenStore + + token, _ = TokenStore.load(tmp_path / "tokens.json").create("drew") + yield client, token, server, control, writes + + +def _auth(token: str) -> dict: + return {"Authorization": f"Bearer {token}"} + + +def test_queued_message_is_answered_on_wake_with_session_continuity(hub): + client, token, _server, _control, writes = hub + + # A live turn first, so the (drew, vera) session has history. + live = client.post( + "/members/vera/chat", headers=_auth(token), json={"prompt": "one"} + ).json() + assert live["turn_idx"] == 0 + + # Vera goes to sleep; a message arrives; custody, not memory. + client.post("/members/vera/presence", headers=_auth(token), json={"presence": "asleep"}) + writes_before = len(writes) + msg = client.post( + "/members/vera/chat", headers=_auth(token), json={"prompt": "two"} + ).json() + assert len(writes) == writes_before # nothing hit any memory scope + + # Wake: the queue drains through the normal chat path. + woke = client.post( + "/members/vera/presence", headers=_auth(token), json={"presence": "awake"} + ).json() + assert woke["drained"] == 1 + + # The answer is retrievable by msg_id, on the same session, at the + # next turn index — exactly as if the sender had waited. + status = client.get(msg["status_url"], headers=_auth(token)).json() + assert status["status"] == "answered" + assert status["result"]["text"] == "reply to: two" + assert status["result"]["session_id"] == live["session_id"] + assert status["result"]["turn_idx"] == 1 + assert status["result"]["memory_written"] is True + + # The drained turn wrote to Vera's private scope like any turn. + assert writes[-1]["scope"] == "private:vera" + assert writes[-1]["participants"] == ["drew"] + + +def test_queued_messages_survive_restart(hub, tmp_path): + client, token, server, _control, _writes = hub + + client.post("/members/vera/presence", headers=_auth(token), json={"presence": "asleep"}) + msg = client.post( + "/members/vera/chat", headers=_auth(token), json={"prompt": "persist me"} + ).json() + + # Simulate a restart: rebuild FamilyState from the same store file. + reborn = server.FamilyState( + ["vera"], default_presence="asleep", store_path=tmp_path / "inbox.json" + ) + record = reborn.get_message("vera", msg["msg_id"]) + assert record is not None + assert record["status"] == "queued" + assert record["prompt"] == "persist me" + + +def test_cortex_down_mid_drain_leaves_messages_queued(hub): + client, token, _server, control, _writes = hub + + client.post("/members/vera/presence", headers=_auth(token), json={"presence": "asleep"}) + msg = client.post( + "/members/vera/chat", headers=_auth(token), json={"prompt": "hold on"} + ).json() + + control.cortex_down = True + woke = client.post( + "/members/vera/presence", headers=_auth(token), json={"presence": "awake"} + ).json() + assert woke["drained"] == 0 + + # Still queued — it will be retried on the next wake. + status = client.get(msg["status_url"], headers=_auth(token)).json() + assert status["status"] == "queued" + + control.cortex_down = False + rewoke = client.post( + "/members/vera/presence", headers=_auth(token), json={"presence": "awake"} + ).json() + assert rewoke["drained"] == 1 + assert client.get( + msg["status_url"], headers=_auth(token) + ).json()["status"] == "answered" diff --git a/tests/test_member_routing.py b/tests/test_member_routing.py index 2bcc01c..0da15f2 100644 --- a/tests/test_member_routing.py +++ b/tests/test_member_routing.py @@ -31,6 +31,7 @@ def hub(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Iterator: monkeypatch.setenv("BRAINSTEM_TOKEN_STORE_PATH", str(tmp_path / "tokens.json")) monkeypatch.setenv("BRAINSTEM_METRICS_PATH", str(tmp_path / "metrics.jsonl")) monkeypatch.setenv("BRAINSTEM_SESSION_STORE_PATH", str(tmp_path / "sessions.json")) + monkeypatch.setenv("BRAINSTEM_INBOX_STORE_PATH", str(tmp_path / "inbox.json")) for mod in list(sys.modules): if mod.startswith("brainstem_4070"): From 664058e4f510182b5479a97a6a4175788403baf2 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 25 Jul 2026 22:35:39 +0000 Subject: [PATCH 07/20] =?UTF-8?q?feat(manager):=20Card=204=20=E2=80=94=20m?= =?UTF-8?q?odel=20manager=20v0=20(tiering=20+=20llama.cpp=20lifecycle)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit New node on the 4090 host owning weights placement and the llama.cpp server lifecycle: - tiering.py: ensure_hot() stages weights hot/warm/cold -> hot with a sha256-verified copy that lands under a .staging name until checked, so a torn copy can never be mistaken for a model. LRU eviction with a pin set that never removes a pinned file or the only copy of a weights file. llama.cpp does not tier for us — mmap off the HDD is the failure mode this whole module exists to avoid. - manager.py: load() = ensure_hot -> report waking -> spawn llama-server (offload_policy from the registry) -> poll health -> report awake; unload() stops the process and reports asleep; a load failure reports asleep rather than lying. Emits stage_copy_ms, staged_from, load_ms, member_id per load into the JSONL harness. - server.py: /status (loaded members + which tier each member's weights are on), /members/{id}/load, /members/{id}/unload. Manual swap only in V1 — presence states and the hub's member_loading contract are real from day one, and the hub's awake transition already drains the inbox (Card 5), so load-finished and queued-messages-answered are the same event. 64 tests pass (6 new); process lifecycle is exercised with real subprocesses, only the HTTP health poll is faked. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_011bnTL8M5fY9bMNjvzuyvyp --- nodes/model_manager_4090/__init__.py | 0 nodes/model_manager_4090/config.py | 48 +++++ nodes/model_manager_4090/manager.py | 259 +++++++++++++++++++++++++++ nodes/model_manager_4090/server.py | 94 ++++++++++ nodes/model_manager_4090/tiering.py | 160 +++++++++++++++++ tests/test_model_manager.py | 186 +++++++++++++++++++ 6 files changed, 747 insertions(+) create mode 100644 nodes/model_manager_4090/__init__.py create mode 100644 nodes/model_manager_4090/config.py create mode 100644 nodes/model_manager_4090/manager.py create mode 100644 nodes/model_manager_4090/server.py create mode 100644 nodes/model_manager_4090/tiering.py create mode 100644 tests/test_model_manager.py diff --git a/nodes/model_manager_4090/__init__.py b/nodes/model_manager_4090/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/nodes/model_manager_4090/config.py b/nodes/model_manager_4090/config.py new file mode 100644 index 0000000..4636b59 --- /dev/null +++ b/nodes/model_manager_4090/config.py @@ -0,0 +1,48 @@ +# nodes/model_manager_4090/config.py +from pydantic_settings import BaseSettings + + +class Settings(BaseSettings): + # --- Weight tiers (V2 doc Section 7) -------------------------------- + # llama.cpp does NOT tier for us — mmap off the HDD pages miserably — + # so the manager stages weights hot before any load. Paths are the + # 4090 host's mounts; tests point these at tmp dirs. + hot_dir: str = "D:/family_weights/hot" # 2TB Gen4 NVMe + warm_dir: str = "E:/family_weights/warm" # 1TB Gen2 NVMe + cold_dir: str = "Z:/family_weights/cold" # 6TB HDD / NAS share + + # --- llama.cpp runtime ---------------------------------------------- + llama_server_bin: str = "llama-server" + llama_port: int = 8000 + llama_host: str = "127.0.0.1" + # Seconds to wait for llama-server to answer /health after spawn. + # A 30B GGUF off Gen4 NVMe loads in well under this; the generous + # ceiling covers vram_ram_ssd offload policies. + load_timeout_seconds: float = 600.0 + health_poll_interval_seconds: float = 2.0 + + # --- Hub reporting ---------------------------------------------------- + # The manager owns presence (Card 4): waking on load start, awake on + # healthy, asleep on unload. Token is a normal brainstem bearer token + # minted for this service. + hub_url: str = "http://192.168.1.141:5001" + hub_token: str = "" + + # --- Registry --------------------------------------------------------- + # Same file the hub boots from; relative resolves against repo root. + family_registry_path: str = "family/registry.yaml" + + # --- Metrics ------------------------------------------------------------ + metrics_path: str = "/data/metrics/model_manager_metrics.jsonl" + + # --- Service ----------------------------------------------------------- + host: str = "0.0.0.0" + port: int = 5004 + log_level: str = "INFO" + + class Config: + env_prefix = "MODELMGR_" + case_sensitive = False + + +settings = Settings() diff --git a/nodes/model_manager_4090/manager.py b/nodes/model_manager_4090/manager.py new file mode 100644 index 0000000..0e5d796 --- /dev/null +++ b/nodes/model_manager_4090/manager.py @@ -0,0 +1,259 @@ +# nodes/model_manager_4090/manager.py +""" +Model manager v0 (Sprint 5, Card 4). + +One process owns weights placement and the llama.cpp lifecycle on the +4090 host. Loading a member is: + + ensure_hot() -> report `waking` -> spawn llama-server + -> poll /health -> report `awake` + +Unloading stops the server and reports `asleep`. Presence flows to the +hub's POST /members/{id}/presence (Card 2), which also drains the +member's inbox on the awake transition (Card 5) — so "the model +manager finished loading" and "queued messages get answered" are the +same event, with no extra choreography. + +V1 runs one member at a time (there's nobody to swap to yet), but +nothing in here assumes that: load/unload are per-member and the +tiering rules already handle contention. +""" +from __future__ import annotations + +import logging +import subprocess +import time +from dataclasses import dataclass, field +from pathlib import Path +from typing import Dict, List, Optional + +import requests + +from bench.probes import JsonlSink, MetricRecord, now_ns +from core.family import FamilyMember + +from .tiering import TierPaths, ensure_hot + +logger = logging.getLogger("model_manager_4090.manager") + + +def weights_filename(member: FamilyMember) -> str: + """Canonical on-disk name for a member's weights: the model repo + tail plus quant, e.g. `Qwen3-30B-A3B-Instruct-2507.Q4_K_M.gguf`. + Download-time tooling and the tiers all agree on this one name.""" + source_tail = member.model.source.split("/")[-1] + return f"{source_tail}.{member.model.quant}.{member.model.format}" + + +class HubPresenceClient: + """Reports presence transitions to the hub. Failures are logged, + not raised — a hub blip must not strand a healthy llama-server.""" + + def __init__(self, hub_url: str, token: str, timeout: float = 5.0): + self.hub_url = hub_url.rstrip("/") + self.token = token + self.timeout = timeout + + def report(self, member_id: str, presence: str) -> bool: + try: + res = requests.post( + f"{self.hub_url}/members/{member_id}/presence", + json={"presence": presence}, + headers={"Authorization": f"Bearer {self.token}"}, + timeout=self.timeout, + ) + res.raise_for_status() + return True + except requests.RequestException as exc: + logger.warning( + "presence report failed (member=%s -> %s): %s", + member_id, presence, exc, + ) + return False + + +@dataclass +class LoadedMember: + member_id: str + process: subprocess.Popen + port: int + weights_path: Path + loaded_at: float = field(default_factory=time.time) + + +class ModelManager: + def __init__( + self, + tiers: TierPaths, + hub: HubPresenceClient, + llama_server_bin: str = "llama-server", + llama_host: str = "127.0.0.1", + llama_port: int = 8000, + load_timeout_seconds: float = 600.0, + health_poll_interval_seconds: float = 2.0, + metrics_sink: Optional[JsonlSink] = None, + ): + self.tiers = tiers + self.hub = hub + self.llama_server_bin = llama_server_bin + self.llama_host = llama_host + self.llama_port = llama_port + self.load_timeout_seconds = load_timeout_seconds + self.health_poll_interval_seconds = health_poll_interval_seconds + self.metrics_sink = metrics_sink + self.loaded: Dict[str, LoadedMember] = {} + + # -- llama.cpp lifecycle (separable for tests) ---------------------- + + def _build_command(self, member: FamilyMember, weights_path: Path) -> List[str]: + cmd = [ + self.llama_server_bin, + "--model", str(weights_path), + "--host", self.llama_host, + "--port", str(self.llama_port), + "--ctx-size", str(member.model.context_length), + ] + # offload_policy maps to how many layers llama.cpp keeps on the + # GPU. vram_then_ram lets llama.cpp fill VRAM and spill the rest + # to system RAM; vram_ram_ssd additionally allows mmap-backed + # cold weights (the seconds-per-token tradeoff Drew accepted for + # some workloads). + if member.runtime.offload_policy in ("vram_then_ram", "vram_ram_ssd"): + cmd += ["--n-gpu-layers", "999"] + if member.runtime.offload_policy == "vram_then_ram": + cmd += ["--no-mmap"] + return cmd + + def _spawn(self, command: List[str]) -> subprocess.Popen: + logger.info("spawning: %s", " ".join(command)) + return subprocess.Popen(command) + + def _healthy(self) -> bool: + try: + res = requests.get( + f"http://{self.llama_host}:{self.llama_port}/health", timeout=3 + ) + return res.status_code == 200 + except requests.RequestException: + return False + + def _wait_healthy(self, process: subprocess.Popen) -> float: + """Poll until llama-server answers /health; return the wait in + ms. Raises if the process dies or the timeout passes.""" + t0 = time.monotonic_ns() + deadline = time.monotonic() + self.load_timeout_seconds + while time.monotonic() < deadline: + if process.poll() is not None: + raise RuntimeError( + f"llama-server exited with code {process.returncode} during load" + ) + if self._healthy(): + return (time.monotonic_ns() - t0) / 1e6 + time.sleep(self.health_poll_interval_seconds) + process.terminate() + raise TimeoutError( + f"llama-server not healthy after {self.load_timeout_seconds}s" + ) + + # -- public API ------------------------------------------------------ + + def load(self, member: FamilyMember) -> LoadedMember: + """Stage weights hot, boot llama-server, flip presence. Emits + stage_copy_ms and load_ms per load (Card 4 done-criterion).""" + if member.id in self.loaded: + return self.loaded[member.id] + + t_ingress = now_ns() + filename = weights_filename(member) + stage = ensure_hot(filename, self.tiers) + + self.hub.report(member.id, "waking") + ok, err = True, None + load_ms = 0.0 + try: + process = self._spawn(self._build_command(member, stage.path)) + load_ms = self._wait_healthy(process) + except Exception as exc: + ok, err = False, str(exc) + self.hub.report(member.id, "asleep") + raise + finally: + self._record( + member_id=member.id, + ingress_ns=t_ingress, + ok=ok, + stage_copy_ms=stage.stage_copy_ms, + staged_from=stage.staged_from, + load_ms=load_ms, + error=err, + ) + + entry = LoadedMember( + member_id=member.id, + process=process, + port=self.llama_port, + weights_path=stage.path, + ) + self.loaded[member.id] = entry + self.hub.report(member.id, "awake") + logger.info( + "member %s awake (stage_copy_ms=%.0f load_ms=%.0f)", + member.id, stage.stage_copy_ms, load_ms, + ) + return entry + + def unload(self, member_id: str, grace_seconds: float = 10.0) -> bool: + """Stop the member's llama-server and report asleep. Returns + False if the member wasn't loaded.""" + entry = self.loaded.pop(member_id, None) + if entry is None: + return False + entry.process.terminate() + try: + entry.process.wait(timeout=grace_seconds) + except subprocess.TimeoutExpired: + logger.warning("llama-server for %s ignored SIGTERM; killing", member_id) + entry.process.kill() + entry.process.wait() + self.hub.report(member_id, "asleep") + logger.info("member %s asleep", member_id) + return True + + def status(self) -> Dict[str, dict]: + out = {} + for member_id, entry in self.loaded.items(): + alive = entry.process.poll() is None + out[member_id] = { + "port": entry.port, + "weights": str(entry.weights_path), + "alive": alive, + "loaded_at": entry.loaded_at, + } + return out + + # -- metrics ----------------------------------------------------------- + + def _record(self, member_id: str, ingress_ns: int, ok: bool, + stage_copy_ms: float, staged_from: Optional[str], + load_ms: float, error: Optional[str]) -> None: + if self.metrics_sink is None: + return + record = MetricRecord( + probe_id="model_manager.load", + stage="load", + ingress_ns=ingress_ns, + egress_ns=now_ns(), + payload_bytes=0, + ok=ok, + extra={ + "member_id": member_id, + "stage_copy_ms": round(stage_copy_ms, 3), + "staged_from": staged_from, + "load_ms": round(load_ms, 3), + "error": error, + }, + ) + try: + self.metrics_sink.write(record) + except Exception: + logger.warning("metric sink write failed", exc_info=True) diff --git a/nodes/model_manager_4090/server.py b/nodes/model_manager_4090/server.py new file mode 100644 index 0000000..8398240 --- /dev/null +++ b/nodes/model_manager_4090/server.py @@ -0,0 +1,94 @@ +# nodes/model_manager_4090/server.py +""" +Model manager service (Sprint 5, Card 4). Runs on the 4090 host and +owns which member is loaded. V1 swap is manual — hit /members/{id}/load +— because there's nobody to swap to yet, but the presence states and +the hub's member_loading contract are real from day one. +""" +from __future__ import annotations + +import logging +from pathlib import Path + +from fastapi import FastAPI, HTTPException + +from bench.probes import JsonlSink +from core.family import load_registry + +from .config import settings +from .manager import HubPresenceClient, ModelManager, weights_filename +from .tiering import TierPaths, find_weights + +logging.basicConfig(level=settings.log_level) +logger = logging.getLogger("model_manager_4090") + +app = FastAPI( + title="Nexus Model Manager (4090)", + description="Weights tiering + llama.cpp lifecycle + presence reporting.", + version="0.1.0", +) + +REPO_ROOT = Path(__file__).resolve().parents[2] +_registry_path = Path(settings.family_registry_path) +if not _registry_path.is_absolute(): + _registry_path = REPO_ROOT / _registry_path +registry = load_registry(_registry_path, _registry_path.parent.parent) + +manager = ModelManager( + tiers=TierPaths( + hot=Path(settings.hot_dir), + warm=Path(settings.warm_dir), + cold=Path(settings.cold_dir), + ), + hub=HubPresenceClient(settings.hub_url, settings.hub_token), + llama_server_bin=settings.llama_server_bin, + llama_host=settings.llama_host, + llama_port=settings.llama_port, + load_timeout_seconds=settings.load_timeout_seconds, + health_poll_interval_seconds=settings.health_poll_interval_seconds, + metrics_sink=JsonlSink(settings.metrics_path), +) + + +def _member_or_404(member_id: str): + try: + return registry.get(member_id) + except KeyError: + raise HTTPException(status_code=404, detail=f"unknown member '{member_id}'") + + +@app.get("/health") +def health(): + return {"status": "ok", "loaded": manager.status()} + + +@app.get("/status") +def status(): + tiers = manager.tiers + weights = {} + for m in registry.members: + located = find_weights(weights_filename(m), tiers) + weights[m.id] = { + "filename": weights_filename(m), + "tier": located[0] if located else None, + } + return {"loaded": manager.status(), "weights": weights} + + +@app.post("/members/{member_id}/load") +def load_member(member_id: str): + member = _member_or_404(member_id) + try: + entry = manager.load(member) + except FileNotFoundError as exc: + raise HTTPException(status_code=404, detail=str(exc)) + except (RuntimeError, TimeoutError, IOError) as exc: + raise HTTPException(status_code=502, detail=str(exc)) + return {"member_id": member_id, "port": entry.port, "weights": str(entry.weights_path)} + + +@app.post("/members/{member_id}/unload") +def unload_member(member_id: str): + _member_or_404(member_id) + stopped = manager.unload(member_id) + return {"member_id": member_id, "stopped": stopped} diff --git a/nodes/model_manager_4090/tiering.py b/nodes/model_manager_4090/tiering.py new file mode 100644 index 0000000..8271d51 --- /dev/null +++ b/nodes/model_manager_4090/tiering.py @@ -0,0 +1,160 @@ +# nodes/model_manager_4090/tiering.py +""" +Weight tiering (Sprint 5, Card 4; V2 doc Section 7). + +Weights live at rest on one of three tiers — hot (Gen4 NVMe), warm +(Gen2 NVMe), cold (HDD/NAS) — and `ensure_hot()` stages them up before +a load, because llama.cpp does not tier for us: mmap'ing a GGUF off +the HDD trades a one-time staged copy for misery on every page fault. + +Safety rules, in order of importance: + 1. Never delete the only copy of a weights file. Eviction removes a + hot copy only when a same-size copy exists on a lower tier. + 2. Copies are checksummed (sha256) and land under a temp name until + verified — a torn copy can never be mistaken for a model. + 3. Pinned files are never evicted, LRU decides among the rest. +""" +from __future__ import annotations + +import hashlib +import logging +import os +import shutil +import time +from dataclasses import dataclass +from pathlib import Path +from typing import List, Optional, Set, Tuple + +logger = logging.getLogger("model_manager_4090.tiering") + +TIER_ORDER = ("hot", "warm", "cold") + + +@dataclass +class TierPaths: + hot: Path + warm: Path + cold: Path + + def dir_for(self, tier: str) -> Path: + return {"hot": self.hot, "warm": self.warm, "cold": self.cold}[tier] + + def ensure_dirs(self) -> None: + for d in (self.hot, self.warm, self.cold): + d.mkdir(parents=True, exist_ok=True) + + +@dataclass +class StageResult: + path: Path # the hot-tier path, ready to load + staged_from: Optional[str] # tier we copied from, None if already hot + stage_copy_ms: float # 0.0 when no copy happened + sha256: Optional[str] # digest of the staged copy (None if no copy) + + +def sha256_file(path: Path, chunk: int = 1 << 20) -> str: + h = hashlib.sha256() + with path.open("rb") as f: + while True: + block = f.read(chunk) + if not block: + break + h.update(block) + return h.hexdigest() + + +def find_weights(filename: str, tiers: TierPaths) -> Optional[Tuple[str, Path]]: + """Locate a weights file, preferring the hottest copy.""" + for tier in TIER_ORDER: + candidate = tiers.dir_for(tier) / filename + if candidate.is_file(): + return tier, candidate + return None + + +def ensure_hot(filename: str, tiers: TierPaths) -> StageResult: + """Stage a weights file onto the hot tier if it isn't there already. + + The copy goes to a `.staging` temp name, gets checksum-verified + against the source, and only then renames into place — so a crash + mid-copy leaves garbage with an obvious name instead of a plausible + but corrupt model. + """ + tiers.ensure_dirs() + located = find_weights(filename, tiers) + if located is None: + raise FileNotFoundError( + f"weights {filename!r} not found on any tier " + f"(hot={tiers.hot}, warm={tiers.warm}, cold={tiers.cold})" + ) + tier, source = located + hot_path = tiers.hot / filename + if tier == "hot": + return StageResult(path=hot_path, staged_from=None, stage_copy_ms=0.0, sha256=None) + + staging = hot_path.with_suffix(hot_path.suffix + ".staging") + t0 = time.monotonic_ns() + shutil.copyfile(source, staging) + source_digest = sha256_file(source) + staged_digest = sha256_file(staging) + if staged_digest != source_digest: + staging.unlink(missing_ok=True) + raise IOError( + f"staged copy of {filename!r} failed checksum " + f"(source {source_digest[:12]}…, copy {staged_digest[:12]}…)" + ) + os.replace(staging, hot_path) + stage_copy_ms = (time.monotonic_ns() - t0) / 1e6 + logger.info( + "staged %s: %s -> hot in %.0fms (sha256 %s…)", + filename, tier, stage_copy_ms, staged_digest[:12], + ) + return StageResult( + path=hot_path, + staged_from=tier, + stage_copy_ms=stage_copy_ms, + sha256=staged_digest, + ) + + +def evict_from_hot( + tiers: TierPaths, + needed_bytes: int, + pinned: Set[str], +) -> List[str]: + """Free at least `needed_bytes` on the hot tier by removing LRU + weights files — never a pinned file, never the only copy. + + Returns the evicted filenames. With a single member (V1) this is + exercised only by tests, but the rules ship now so member #2 is a + registry entry, not a code change. + """ + tiers.ensure_dirs() + candidates = [] + for path in tiers.hot.iterdir(): + if not path.is_file() or path.name in pinned or path.name.endswith(".staging"): + continue + lower_copy = None + for tier in ("warm", "cold"): + other = tiers.dir_for(tier) / path.name + if other.is_file() and other.stat().st_size == path.stat().st_size: + lower_copy = other + break + if lower_copy is None: + continue # rule 1: never delete the only copy + candidates.append(path) + + # LRU by last access, oldest first. + candidates.sort(key=lambda p: p.stat().st_atime) + + evicted: List[str] = [] + freed = 0 + for path in candidates: + if freed >= needed_bytes: + break + size = path.stat().st_size + path.unlink() + freed += size + evicted.append(path.name) + logger.info("evicted %s from hot tier (freed %d bytes)", path.name, size) + return evicted diff --git a/tests/test_model_manager.py b/tests/test_model_manager.py new file mode 100644 index 0000000..ce29991 --- /dev/null +++ b/tests/test_model_manager.py @@ -0,0 +1,186 @@ +"""Sprint 5 Card 4: model manager v0 — tiering + lifecycle + presence. + +Done-criteria under test: + - cold-start staging: cold -> hot copy with checksum verification and + a measured stage_copy_ms; already-hot weights are a no-op; + - a torn/corrupt copy can never be mistaken for a model; + - eviction respects pins and never deletes the only copy; + - load() flips presence waking -> awake, emits stage_copy_ms/load_ms + metrics; unload() reports asleep; a dead process fails the load + and reports asleep rather than lying about being awake. + +llama-server is stubbed with a real (trivial) subprocess so process +lifecycle is exercised for real; only the HTTP health poll is faked. +""" +from __future__ import annotations + +import json +import subprocess +import sys +from pathlib import Path + +import pytest + +from core.family import load_registry +from model_manager_4090.manager import ModelManager, weights_filename +from model_manager_4090.tiering import TierPaths, ensure_hot, evict_from_hot, sha256_file + +REPO_ROOT = Path(__file__).resolve().parent.parent + + +# --------------------------------------------------------------------------- +# Helpers +# --------------------------------------------------------------------------- + + +def _tiers(tmp_path: Path) -> TierPaths: + t = TierPaths(hot=tmp_path / "hot", warm=tmp_path / "warm", cold=tmp_path / "cold") + t.ensure_dirs() + return t + + +class FakeHub: + def __init__(self): + self.reports = [] + + def report(self, member_id: str, presence: str) -> bool: + self.reports.append((member_id, presence)) + return True + + +def _vera(): + registry = load_registry(REPO_ROOT / "family" / "registry.yaml", REPO_ROOT) + return registry.get("vera") + + +def _manager(tiers: TierPaths, hub: FakeHub, metrics_path: Path) -> ModelManager: + from bench.probes import JsonlSink + + return ModelManager( + tiers=tiers, + hub=hub, + llama_server_bin=sys.executable, # overridden per test via _build_command + load_timeout_seconds=10.0, + health_poll_interval_seconds=0.05, + metrics_sink=JsonlSink(str(metrics_path)), + ) + + +# --------------------------------------------------------------------------- +# Tiering +# --------------------------------------------------------------------------- + + +def test_ensure_hot_stages_from_cold_with_checksum(tmp_path): + tiers = _tiers(tmp_path) + (tiers.cold / "m.gguf").write_bytes(b"weights" * 1000) + + result = ensure_hot("m.gguf", tiers) + assert result.path == tiers.hot / "m.gguf" + assert result.path.is_file() + assert result.staged_from == "cold" + assert result.stage_copy_ms >= 0.0 + assert result.sha256 == sha256_file(tiers.cold / "m.gguf") + # Cold copy is untouched (staging copies, never moves). + assert (tiers.cold / "m.gguf").is_file() + + # Second call is a no-op. + again = ensure_hot("m.gguf", tiers) + assert again.staged_from is None + assert again.stage_copy_ms == 0.0 + + +def test_ensure_hot_prefers_warm_over_cold(tmp_path): + tiers = _tiers(tmp_path) + (tiers.cold / "m.gguf").write_bytes(b"cold") + (tiers.warm / "m.gguf").write_bytes(b"warm") + result = ensure_hot("m.gguf", tiers) + assert result.staged_from == "warm" + assert (tiers.hot / "m.gguf").read_bytes() == b"warm" + + +def test_ensure_hot_missing_everywhere_fails(tmp_path): + with pytest.raises(FileNotFoundError): + ensure_hot("ghost.gguf", _tiers(tmp_path)) + + +def test_eviction_respects_pins_and_sole_copies(tmp_path): + tiers = _tiers(tmp_path) + # a: pinned, backed by warm. b: evictable, backed by cold. + # c: hot-only — the only copy in existence, must never be removed. + for name, backing in (("a.gguf", "warm"), ("b.gguf", "cold")): + (tiers.hot / name).write_bytes(b"x" * 100) + (tiers.dir_for(backing) / name).write_bytes(b"x" * 100) + (tiers.hot / "c.gguf").write_bytes(b"x" * 100) + + evicted = evict_from_hot(tiers, needed_bytes=10_000, pinned={"a.gguf"}) + assert evicted == ["b.gguf"] + assert (tiers.hot / "a.gguf").is_file() # pinned + assert (tiers.hot / "c.gguf").is_file() # sole copy + assert not (tiers.hot / "b.gguf").is_file() + assert (tiers.cold / "b.gguf").is_file() # lower-tier copy intact + + +# --------------------------------------------------------------------------- +# Lifecycle +# --------------------------------------------------------------------------- + + +def test_load_stages_boots_and_reports_presence(tmp_path, monkeypatch): + tiers = _tiers(tmp_path) + hub = FakeHub() + member = _vera() + (tiers.cold / weights_filename(member)).write_bytes(b"gguf" * 256) + + metrics_path = tmp_path / "metrics.jsonl" + mgr = _manager(tiers, hub, metrics_path) + # Real subprocess (sleeps), faked health probe. + monkeypatch.setattr( + mgr, "_build_command", + lambda m, p: [sys.executable, "-c", "import time; time.sleep(60)"], + ) + monkeypatch.setattr(mgr, "_healthy", lambda: True) + + entry = mgr.load(member) + try: + assert entry.process.poll() is None + assert hub.reports == [("vera", "waking"), ("vera", "awake")] + + record = json.loads(metrics_path.read_text().splitlines()[-1]) + assert record["member_id"] == "vera" + assert record["staged_from"] == "cold" + assert record["stage_copy_ms"] >= 0.0 + assert record["load_ms"] >= 0.0 + assert record["ok"] is True + + # Idempotent: loading again returns the same entry. + assert mgr.load(member) is entry + finally: + mgr.unload("vera") + + assert hub.reports[-1] == ("vera", "asleep") + assert entry.process.poll() is not None + assert mgr.unload("vera") is False # already gone + + +def test_dead_process_fails_load_and_reports_asleep(tmp_path, monkeypatch): + tiers = _tiers(tmp_path) + hub = FakeHub() + member = _vera() + (tiers.hot / weights_filename(member)).write_bytes(b"gguf") + + mgr = _manager(tiers, hub, tmp_path / "metrics.jsonl") + monkeypatch.setattr( + mgr, "_build_command", + lambda m, p: [sys.executable, "-c", "raise SystemExit(3)"], + ) + monkeypatch.setattr(mgr, "_healthy", lambda: False) + + with pytest.raises(RuntimeError, match="exited"): + mgr.load(member) + assert hub.reports == [("vera", "waking"), ("vera", "asleep")] + assert "vera" not in mgr.loaded + + record = json.loads((tmp_path / "metrics.jsonl").read_text().splitlines()[-1]) + assert record["ok"] is False + assert "exited" in record["error"] From 60e0938fa016ab26be83121ceb4382d33aa7eb76 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 25 Jul 2026 22:37:08 +0000 Subject: [PATCH 08/20] =?UTF-8?q?feat(metrics):=20Card=207=20=E2=80=94=20q?= =?UTF-8?q?ueue=5Fwait=5Fms,=20roster=20on=20status=20feed,=20bench=20prer?= =?UTF-8?q?eg?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Inbox drains emit a brainstem.inbox_drain record per answered message with member_id, msg_id, queue_wait_ms, and sender attribution — the waiting is measured, not just the turn (generate records already carry member_id since Card 3, and the model manager emits stage_copy_ms/load_ms since Card 4). - /fabric/status now carries the family roster (presence + queue depth per member) for the dashboard. - docs/sprints/SPRINT_5_BENCH_PREREG_2026-07-25.md pre-registers the member #1 baseline before any llama.cpp run: Sprint 3d prompt set through the hub path, quality guard, >=70% of vLLM AWQ median tokens_per_s, cold-start under 5 minutes, and an explicit nothing-tuned-before-baseline rule. The frozen baseline seeds member #1's report card (V2 Section 10). 65 tests pass (1 new). All seven Sprint 5 cards are now implemented. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_011bnTL8M5fY9bMNjvzuyvyp --- .../SPRINT_5_BENCH_PREREG_2026-07-25.md | 49 +++++++++++++++++++ nodes/brainstem_4070/server.py | 32 ++++++++++++ tests/test_inbox_drain.py | 28 +++++++++++ 3 files changed, 109 insertions(+) create mode 100644 docs/sprints/SPRINT_5_BENCH_PREREG_2026-07-25.md diff --git a/docs/sprints/SPRINT_5_BENCH_PREREG_2026-07-25.md b/docs/sprints/SPRINT_5_BENCH_PREREG_2026-07-25.md new file mode 100644 index 0000000..3f5d290 --- /dev/null +++ b/docs/sprints/SPRINT_5_BENCH_PREREG_2026-07-25.md @@ -0,0 +1,49 @@ +# Sprint 5 bench pre-registration — member #1 baseline (Card 7) + +**Date registered:** 2026-07-25 (before any llama.cpp run on the 4090) +**Discipline:** same as Sprint 3d — win conditions declared here, before +the first measured run; bootstrap CIs on latency stats; no retroactive +goalpost moves. This file is the registration; results land in a +separate results doc that links back here. + +## What is being measured + +The V2 runtime swap: **llama.cpp `llama-server`, Qwen3-30B-A3B GGUF +Q4_K_M** (member #1 "vera", `offload_policy: vram_then_ram`) versus the +Sprint 3d baseline (**vLLM, Qwen3-30B-A3B-AWQ**) on the same 4090 host. + +- **Prompt set:** the Sprint 3d bench prompt set, unchanged, same order. +- **Path:** through the hub (`POST /members/vera/chat`), so retrieval, + scope filtering, and write-on-turn costs are included — this is the + number Drew actually experiences, not a bare-runtime number. +- **Metrics:** `tokens_per_s`, `total_ms` p50/p95, `cortex_roundtrip_ms` + p50/p95, plus the new `stage_copy_ms` and `load_ms` for the + cold-start story (no vLLM comparison for those — vLLM never staged + from cold tiers). + +## Win conditions (declared now) + +1. **Quality guard:** the Sprint 3d eval gauntlet regresses by no more + than the guard tolerance already defined there. A faster runtime + that answers worse does not ship. +2. **Throughput:** llama.cpp Q4_K_M reaches ≥ 70% of the vLLM AWQ + `tokens_per_s` median. The swap is motivated by the family + architecture (offload, GGUF portability, one runtime for every + member), not raw speed — but below 70% we stop and investigate + before accepting. +3. **Cold start:** HDD → serving (stage_copy + load) under 5 minutes, + measured by the Card 4 metrics. This is the "member wakes up" + budget the inbox UX is designed around. + +## What may be tuned before the measured run + +Nothing. First measured run is the baseline, as-is from the registry +defaults. `offload_policy` and sampling tuning happen only *after* the +baseline is frozen, each as its own recorded run — that ordering is +the entire point of pre-registering. + +## Report card seed + +The frozen baseline becomes member #1's first report-card entry +(V2 doc Section 10): the reference every future adapter, quant change, +or self-training experiment must beat on the same gauntlet. diff --git a/nodes/brainstem_4070/server.py b/nodes/brainstem_4070/server.py index 047f8b0..2d8103e 100644 --- a/nodes/brainstem_4070/server.py +++ b/nodes/brainstem_4070/server.py @@ -671,6 +671,35 @@ def _drain_inbox(member_id: str) -> int: "memory_written": result.memory_written, }) answered += 1 + + # Card 7: how long the message sat in custody before the member + # answered it. The generate record covers the turn itself; this + # record covers the waiting. + try: + queued_at = datetime.fromisoformat(msg["queued_at"]) + queue_wait_ms = ( + datetime.now(timezone.utc) - queued_at + ).total_seconds() * 1000.0 + except (KeyError, ValueError): + queue_wait_ms = None + drain_record = MetricRecord( + probe_id="brainstem.inbox_drain", + stage="drain", + ingress_ns=now_ns(), + egress_ns=now_ns(), + payload_bytes=len(msg["prompt"].encode("utf-8")), + ok=True, + extra={ + "member_id": member_id, + "msg_id": msg["msg_id"], + "queue_wait_ms": round(queue_wait_ms, 3) if queue_wait_ms is not None else None, + "token_name": msg["person"], + }, + ) + try: + metrics_sink.write(drain_record) + except Exception: + logger.warning("metric sink write failed", exc_info=True) if answered: logger.info("inbox drain: member=%s answered=%d", member_id, answered) return answered @@ -991,6 +1020,9 @@ def fabric_status(): "nas": {**nas_status, "url_configured": settings.nas_url}, "embedder": {**embedder_status, "url_configured": settings.embedder_url}, "metrics": _metrics_summary(), + # Card 7: the family roster on the live status feed — presence + # and queue depth per member, same shape as GET /family. + "family": [_member_summary(m) for m in family_registry.members], "recent_roundtrips": list(recent_roundtrips)[-25:][::-1], } diff --git a/tests/test_inbox_drain.py b/tests/test_inbox_drain.py index a74b0b7..fe04b5c 100644 --- a/tests/test_inbox_drain.py +++ b/tests/test_inbox_drain.py @@ -109,6 +109,34 @@ def test_queued_message_is_answered_on_wake_with_session_continuity(hub): assert writes[-1]["participants"] == ["drew"] +def test_drain_emits_queue_wait_metric(hub, tmp_path): + """Card 7: the waiting is measured, not just the turn.""" + import json + + client, token, _server, _control, _writes = hub + client.post("/members/vera/presence", headers=_auth(token), json={"presence": "asleep"}) + msg = client.post( + "/members/vera/chat", headers=_auth(token), json={"prompt": "measure me"} + ).json() + client.post("/members/vera/presence", headers=_auth(token), json={"presence": "awake"}) + + records = [ + json.loads(line) + for line in (tmp_path / "metrics.jsonl").read_text().splitlines() + ] + drains = [r for r in records if r["probe_id"] == "brainstem.inbox_drain"] + assert len(drains) == 1 + assert drains[0]["member_id"] == "vera" + assert drains[0]["msg_id"] == msg["msg_id"] + assert drains[0]["queue_wait_ms"] >= 0.0 + assert drains[0]["token_name"] == "drew" + + # And the fabric status feed carries the family roster (Card 7). + fabric = client.get("/fabric/status").json() + assert fabric["family"][0]["id"] == "vera" + assert fabric["family"][0]["queue_depth"] == 0 + + def test_queued_messages_survive_restart(hub, tmp_path): client, token, server, _control, _writes = hub From 9357692acdc6110b02b9717b7d88202a89fac994 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 26 Jul 2026 01:11:20 +0000 Subject: [PATCH 09/20] docs: bring memory_system.md and readme up to date with Sprint 5 memory_system.md gains a 'Sprint 5: scoped memory' section covering scopes, provenance fields, the mandatory server-side query filter (and how it amends the Sprint 2 no-filter decision), copy-never-move promotion, and the migration script's dry-run/idempotent/reconcile behavior. readme.md's component list now describes the brainstem's Nexus Hub role with the member API contract and adds the model_manager_4090 node. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_011bnTL8M5fY9bMNjvzuyvyp --- docs/memory_system.md | 58 +++++++++++++++++++++++++++++++++++++++++++ readme.md | 5 ++-- 2 files changed, 61 insertions(+), 2 deletions(-) diff --git a/docs/memory_system.md b/docs/memory_system.md index 5099dce..4be61c5 100644 --- a/docs/memory_system.md +++ b/docs/memory_system.md @@ -130,3 +130,61 @@ Chunk C confirmed the cross-session recall test against this stack. Sprint 3b la ## Auth (Sprint 3b) `/generate`, `/embed`, and `/stm/write` now require `Authorization: Bearer `. Status endpoints (`/health`, `/cortex/health`, `/embedder/health`, `/fabric/status`, `/dashboard`, `/`) stay anonymous. Tokens are minted via `python scripts/create_token.py --name ` inside the brainstem container; the plaintext token is printed once and only the argon2id (or scrypt fallback) hash lives on disk. Per-request token attribution is logged and written to the metric record under `token_name`. See `docs/auth_middleware.md` for the full design and decision log. + +## Sprint 5: scoped memory + +Design of record: `docs/architecture_v2_family_of_models.md` (Sections 4 and 5), implemented per `docs/sprints/SPRINT_5_PLAN_2026-07-25.md` Card 3. Everything above this section describes the single-scope Sprint 2 store; this section describes how it became a multi-member store without a rewrite — the collection, chunker, and BGE model are all unchanged. + +### Scopes + +Every row now lives in exactly one of three scopes: + +- **`private:`** — 1-on-1 conversation turns between a person and that member. This is the default write target for every turn. Only that member's queries can read it. +- **`shared:household`** — the family's common ground: sensor events (Jetson classification, born shared) and conversation memories a person has explicitly promoted. Every member's queries can read it. +- **`experiential:`** — reserved for Project Vector (a member's own sensor platform). No writers in V1; only that member's queries can read it. + +`nodes/embedder_4070/scopes.py` is the single source of truth for these rules (pure functions, no Chroma dependency, so the privacy logic is unit-testable on its own). + +### Provenance metadata + +Every chunk's metadata gained four fields on top of the Sprint 2 schema (`session_id`, `turn_idx`, `ts`, `model_used`, etc. — all unchanged): + +- `scope` — one of the three scopes above. +- `member_id` — the member the row belongs to (`"household"` for `shared:household` rows). +- `origin` — `conversation | sensor | promotion | vector_platform | delegated_task` (the last two are reserved for Project Vector and the V1.5 concierge; no writer produces them yet). +- `participants` — who was in the conversation, from `token_name` attribution. Chroma metadata values must be scalars, so this is stored **comma-joined** (e.g. `"drew"` or `"drew,vera"`), not as a list. + +Promoted rows carry three additional fields: `promoted_from` (the source row id), `promoted_from_member` (which member's private scope it came from), and `promoted_by` (who confirmed the share). + +### `/memory/write` requires scope + member_id + +`POST /memory/write` on the embedder now takes mandatory `scope` and `member_id` fields (`origin` defaults to `"conversation"`, `participants` defaults to empty). The service validates before writing: a member may only write into its own `private:` / `experiential:` scopes or into `shared:household` — never into another member's scopes. A cross-member write attempt is rejected with `400` before anything touches Chroma. There is no longer a way to write an unscoped row. + +### `/memory/query` is server-side scope-filtered — always + +`POST /memory/query` now takes a mandatory `member_id`. The embedder builds the Chroma `where` clause from it unconditionally: + +``` +scope IN (private:, shared:household, experiential:) +``` + +Callers cannot widen this — there is no parameter that requests a different or broader scope set, and the filter is applied inside the embedder service, not trusted to the brainstem or the model. This **amends the Sprint 2 decision** documented above (Chunk B: "no default session filter, because the done-criterion is cross-session recall"). That done-criterion is preserved — a member still recalls every past session it has had — but it no longer means *every session of every member*. Cross-session recall within a member survives; cross-member recall is now structurally impossible through this API. `session_id_filter` and `exclude_parent_turn_id` remain available as optional refinements *inside* the member's visible scopes, not as ways around them. + +### `/memory/promote` — copy, never move + +`POST /memory/promote` (`member_id`, `memory_id`, `promoted_by`) shares a private memory with the household without touching the original: + +- The shared copy gets a **deterministic id** — `{source_id}::promoted` — so promoting the same row twice is a no-op (`already_promoted: true` in the response) rather than a duplicate. +- The copy is written to `shared:household` with the full paper trail: `origin: "promotion"`, `promoted_from`, `promoted_from_member`, `promoted_by`. The private original's metadata and scope are untouched. +- The source row must actually be in `private:` for that member — promoting a row that isn't yours (or isn't private) is rejected with `400`. +- At the hub, `POST /members/{id}/memory/promote` is the person's confirmation step; reaching that endpoint at all establishes consent (only people hold bearer tokens), per the offer-then-confirm rule in the V2 decision record — a member may *offer* to share in conversation, but the write only happens once the person calls this endpoint. + +### Migration: `scripts/migrate_memory_scopes.py` + +Pre-Sprint-5 rows have no `scope` metadata, which makes them invisible to the now-mandatory filter above. The migration script backfills exactly those rows: + +- **Dry-run by default.** `python scripts/migrate_memory_scopes.py` only prints the reconciliation plan (`total` / `already_scoped` / `would update`); nothing is written until you pass `--apply`. +- **Grandfathers into member #1's private scope.** Unscoped rows get `scope=private:` (default: the first entry in `family/registry.yaml`), `member_id=`, `origin=conversation`, and `participants` from the (optional) `--participants` flag — pre-V2 rows never recorded who spoke, so this defaults to empty. +- **Idempotent.** Rows that already carry a `scope` are left untouched, so re-running the script (with or without `--apply`) is always safe. +- **Reconciles to the row.** The script tallies `already_scoped + updated` against the collection's total count and exits nonzero if anything is unaccounted for, rather than silently leaving rows behind. +- Runs inside the embedder container, since that's what owns the Chroma volume: `docker compose exec embedder python scripts/migrate_memory_scopes.py [--apply] [--member ID] [--participants a,b] [--persist-dir ...] [--collection ...]`. diff --git a/readme.md b/readme.md index d5ef1cf..957c4a8 100644 --- a/readme.md +++ b/readme.md @@ -20,10 +20,11 @@ Biological intelligence is distributed. Nerves preprocess. The brainstem filters ### Components 1. **Jetson nodes (peripheral nervous system).** Capture raw video, audio, telemetry. Perform early filtering and compression. Push signals to the brainstem. -2. **4070 node (brainstem).** Validates incoming signals, generates embeddings, applies instinctual rules, buffers short-term memory, and decides what is important enough to consolidate. +2. **4070 node (brainstem).** Validates incoming signals, generates embeddings, applies instinctual rules, buffers short-term memory, and decides what is important enough to consolidate. Since Sprint 5 it also serves as the **Nexus Hub**, the family-of-models orchestration layer: `GET /family` (roster + presence + queue depths), `GET /members/{id}` (spec summary, presence, model info), `POST /members/{id}/chat` (`200` for a live reply, `202` + `msg_id` when the message is queued, `503 member_loading` + `Retry-After` when the member is waking up), `POST /members/{id}/presence`, `GET /members/{id}/inbox/{msg_id}` (poll a queued message's reply), and `POST /members/{id}/memory/promote` (share a private memory to the household). See `docs/architecture_v2_family_of_models.md` for the full design. 3. **NAS (long-term memory).** Vector database for semantic memory, time-ordered episodic logs, knowledge graph structures, decay and deduplication. Synthetic hippocampus. 4. **4090 node (cortex).** Hosts large-scale LLM reasoning, executes high-level planning, integrates episodic and semantic recall, and orchestrates downstream agents. -5. **Consolidation engine (sleep node).** Re-embeds old memories, clusters and abstracts concepts, summarizes logs into narratives, enforces schema consistency, runs "synthetic dreams". +5. **Model manager (4090 node, Sprint 5).** `nodes/model_manager_4090/` — owns weights placement and the `llama-server` lifecycle for whichever family member is resident. Tiers weights hot/warm/cold (NVMe Gen4 / NVMe Gen2 / HDD) via `ensure_hot()` staged, checksum-verified copies, since llama.cpp does not tier for us; loads/unloads `llama-server` per member and reports presence (`waking`/`awake`/etc.) back to the hub. +6. **Consolidation engine (sleep node).** Re-embeds old memories, clusters and abstracts concepts, summarizes logs into narratives, enforces schema consistency, runs "synthetic dreams". ## Research From cb67fcff72dff52028796fc949e1253ce3c1cdda Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 26 Jul 2026 01:11:20 +0000 Subject: [PATCH 10/20] feat(runtime): llama.cpp compose + operator README for the 4090 compose.llamacpp.yaml runs the official CUDA llama-server image with the exact flags the model manager builds for member #1 (Q4_K_M gguf from the hot tier, ctx 32768, all layers on GPU, --no-mmap per vram_then_ram). The trtllm compose.yaml stays as historical reference. README.md documents the two bring-up paths (manager-spawned native process as the V1 default vs standalone compose), the MODELMGR_ env vars, the weights filename convention, and why mmap off the cold tier is the failure mode tiering exists to avoid. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_011bnTL8M5fY9bMNjvzuyvyp --- Nexus-LLM-Runtime-4090/README.md | 68 ++++++++++++++++++++ Nexus-LLM-Runtime-4090/compose.llamacpp.yaml | 40 ++++++++++++ 2 files changed, 108 insertions(+) create mode 100644 Nexus-LLM-Runtime-4090/README.md create mode 100644 Nexus-LLM-Runtime-4090/compose.llamacpp.yaml diff --git a/Nexus-LLM-Runtime-4090/README.md b/Nexus-LLM-Runtime-4090/README.md new file mode 100644 index 0000000..32f0898 --- /dev/null +++ b/Nexus-LLM-Runtime-4090/README.md @@ -0,0 +1,68 @@ +# Nexus-LLM-Runtime-4090 + +Inference runtime for the 4090 host. As of Sprint 5, the runtime direction is +**llama.cpp (`llama-server`)**, not vLLM/TensorRT-LLM — see +`docs/architecture_v2_family_of_models.md` Section 6. `compose.yaml` (the +trtllm setup) stays in this directory as historical reference until Drew +retires it; do not use it for new deployments. + +## Two ways to run the runtime + +**A. Model manager spawns llama-server natively (V1 default).** +`nodes/model_manager_4090` (`manager.py`) owns weights tiering, the +llama-server process lifecycle, and presence reporting to the hub. It calls +`llama-server` as a plain subprocess with the flags below — no Docker +involved. This is the default because the manager needs direct control over +staging weights onto the hot tier (`ensure_hot()`) before each spawn, and +needs to poll `/health` and manage SIGTERM/SIGKILL directly. Use this path +whenever the model manager service is running. + +**B. `compose.llamacpp.yaml` (standalone / manual bring-up).** +Use this compose file when you need to run llama-server by hand — the model +manager isn't running, you're debugging a GGUF outside the manager's +lifecycle, or you want a quick manual smoke test. It runs the official CUDA +server image (`ghcr.io/ggml-org/llama.cpp:server-cuda`), mounts the hot tier +at `/models`, and passes the same flags the manager would pass for family +member #1 ("vera"). It does **not** do tiering or presence reporting — those +only happen when the manager is in the loop (path A). + +## Model manager environment variables + +The manager reads settings via `MODELMGR_`-prefixed env vars +(`nodes/model_manager_4090/config.py`): + +| Variable | Purpose | +|---|---| +| `MODELMGR_HOT_DIR` | Hot tier path (2 TB Gen4 NVMe) — active/loadable weights. | +| `MODELMGR_WARM_DIR` | Warm tier path (1 TB Gen2 NVMe) — occasional members. | +| `MODELMGR_COLD_DIR` | Cold tier path (6 TB HDD/NAS) — archive. | +| `MODELMGR_HUB_URL` | Hub base URL the manager reports presence to. | +| `MODELMGR_HUB_TOKEN` | Bearer token for the hub's presence endpoint. | +| `MODELMGR_LLAMA_SERVER_BIN` | Path/name of the `llama-server` binary the manager spawns. | + +## Weights filename convention + +`weights_filename()` in `nodes/model_manager_4090/manager.py` builds the +on-disk filename as `..`, where the source +tail is the last path segment of `model.source` from `family/registry.yaml`. + +For family member #1 ("vera": `hf:Qwen/Qwen3-30B-A3B-Instruct-2507`, quant +`Q4_K_M`, format `gguf`), the exact expected filename is: + +``` +Qwen3-30B-A3B-Instruct-2507.Q4_K_M.gguf +``` + +Download tooling, the hot/warm/cold tier directories, and both runtime paths +above (A and B) must all agree on this name. + +## Why tiering exists: don't mmap off the cold tier + +`llama.cpp` does not tier weights for us — `mmap` will happily page a GGUF +straight off the HDD/cold tier, and it works, but page-in latency makes +inference miserable. That's the exact failure mode +`nodes/model_manager_4090`'s tiering (`ensure_hot()`) exists to avoid: it +stages weights onto the hot NVMe tier *before* starting llama-server, so the +model only ever loads/mmaps from fast storage. If you bring the runtime up +by hand (path B), make sure the file under `D:/family_weights/hot` is +actually there and not a broken symlink back to cold storage. diff --git a/Nexus-LLM-Runtime-4090/compose.llamacpp.yaml b/Nexus-LLM-Runtime-4090/compose.llamacpp.yaml new file mode 100644 index 0000000..f186438 --- /dev/null +++ b/Nexus-LLM-Runtime-4090/compose.llamacpp.yaml @@ -0,0 +1,40 @@ +services: + llamacpp: + container_name: llamacpp-vera + image: ghcr.io/ggml-org/llama.cpp:server-cuda + + # --- GPU --- + runtime: nvidia + + # --- Networking --- + ports: + - "8000:8000" + + # --- Weights mount (hot tier only — see README on why cold/mmap is avoided) --- + volumes: + - "D:/family_weights/hot:/models" + + # --- Environment --- + environment: + NVIDIA_VISIBLE_DEVICES: all + NVIDIA_DRIVER_CAPABILITIES: compute,utility + + # --- llama-server flags (mirrors what nodes/model_manager_4090/manager.py + # builds for family/registry.yaml member #1, "vera": Qwen3-30B-A3B + # GGUF Q4_K_M, context_length 32768, offload_policy vram_then_ram) --- + command: + - "--model" + - "/models/Qwen3-30B-A3B-Instruct-2507.Q4_K_M.gguf" + - "--host" + - "0.0.0.0" + - "--port" + - "8000" + - "--ctx-size" + - "32768" + - "--n-gpu-layers" + - "999" + - "--no-mmap" + + # --- Keep container alive for interactive or server use --- + tty: true + stdin_open: true From 05bd26e166941e9b2bb06bb646d1573297c5b01e Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 26 Jul 2026 01:11:59 +0000 Subject: [PATCH 11/20] feat(dashboard): family roster panel on the live dashboard Renders the fabric/status family block: per-member presence with the existing status-dot pattern (awake green, busy/waking amber, asleep muted), queue depth highlighted when nonzero, and compact quant/ctx model info. Degrades gracefully against an older server payload with no family field. No new fetches, dependencies, or styles outside the existing tokens. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_011bnTL8M5fY9bMNjvzuyvyp --- nodes/brainstem_4070/dashboard.html | 54 +++++++++++++++++++++++++++++ 1 file changed, 54 insertions(+) diff --git a/nodes/brainstem_4070/dashboard.html b/nodes/brainstem_4070/dashboard.html index 39359c1..5373977 100644 --- a/nodes/brainstem_4070/dashboard.html +++ b/nodes/brainstem_4070/dashboard.html @@ -52,10 +52,16 @@ .dot.up { background: var(--ok); } .dot.down { background: var(--bad); } .dot.unknown { background: var(--warn); } + .dot.presence-awake { background: var(--ok); } + .dot.presence-busy, .dot.presence-waking { background: var(--warn); } + .dot.presence-asleep { background: var(--muted); } .state { font-family: var(--mono); font-size: 12px; } .state.up { color: var(--ok); } .state.down { color: var(--bad); } .state.unknown { color: var(--warn); } + .state.presence-awake { color: var(--ok); } + .state.presence-busy, .state.presence-waking { color: var(--warn); } + .state.presence-asleep { color: var(--muted); } .kv { font-family: var(--mono); font-size: 12px; color: var(--muted); } .kv b { color: var(--text); font-weight: 500; } .link-cell { display: flex; flex-direction: column; align-items: center; @@ -80,6 +86,7 @@ tr:last-child td { border-bottom: none; } td.ok { color: var(--ok); } td.bad { color: var(--bad); } + td.queue-pending { color: var(--warn); font-weight: 600; } .empty { color: var(--muted); padding: 18px 4px; font-size: 13px; } .pill { font-family: var(--mono); font-size: 11px; padding: 1px 7px; border-radius: 999px; border: 1px solid var(--line); color: var(--muted); } @@ -167,6 +174,18 @@

Recent round trips

+

Family

+
+ + + + + + + +
memberpresencequeuemodel
no family roster reported
+
+