Family of Models: Sprint 5 (V1) + Sprint 6 (V1.5 receptionist) implementation - #13
Draft
Ginkobaloba wants to merge 20 commits into
Draft
Family of Models: Sprint 5 (V1) + Sprint 6 (V1.5 receptionist) implementation#13Ginkobaloba wants to merge 20 commits into
Ginkobaloba wants to merge 20 commits into
Conversation
Seven dependency-ordered cards covering the V1 scope from the accepted V2 architecture (Section 11): registry loader, hub member routing, scoped memory + migration, model manager v0, inbox v0, promotion endpoint, and member-aware metrics. Jeffery and delegation remain out of scope per the V1.5/V2.x phasing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011bnTL8M5fY9bMNjvzuyvyp
family/registry.yaml is the single source of truth for the roster, seeded with member #1 (vera, Qwen3-30B-A3B GGUF Q4_K_M) exactly as specified in the V2 doc Section 4.1. core/family.py validates at load: unknown keys, duplicate ids, bad offload policies, and missing or empty spec files are hard startup failures. Adding member #2 is a registry entry + spec file — covered by a test that proves it needs no code. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011bnTL8M5fY9bMNjvzuyvyp
The brainstem grows the family-hub surface from the V2 doc Section 5:
- GET /family and GET /members/{id}: roster, presence, queue depth
(anonymous, like the other status endpoints).
- POST /members/{id}/chat: awake members run the turn live through the
existing /generate path with the member spec injected as the base
system prompt (caller system layers after it, retrieved memory after
that); asleep/busy members return 202 + msg_id into an in-memory
inbox v0 (durable drain lands in Card 5); waking members return the
structured 503 member_loading contract with Retry-After,
generalizing the Sprint 3c cortex-down shape.
- POST /members/{id}/presence: the model manager's reporting hook
(Card 4), doubling as the operator/test switch until it exists.
- GET /members/{id}/inbox/{msg_id}: sender-only status of a queued
message; wrong-person lookups 404 so existence stays private.
- Hub-minted sessions per (person, member), persisted to disk with
turn counters — a restart no longer resets turn_idx (the Sprint 2
wart is fixed and covered by a restart-simulation test).
Registry sampling defaults apply when the caller does not override
temperature. Existing /generate, auth, and cortex-down behavior is
untouched; the full suite (40 tests) passes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011bnTL8M5fY9bMNjvzuyvyp
Every memory row now carries provenance (scope, member_id, origin,
participants) and every query is filtered server-side in the embedder
to the querying member's visible set: private:<member> +
shared:household + experiential:<member>. The filter is built from
member_id inside the service that owns the store — callers cannot
widen it, so cross-member recall is structurally impossible. This
amends Sprint 2's deliberate no-filter design; cross-session recall
within a member is unchanged and still covered by tests.
- embedder: /memory/write requires scope+member_id and rejects writes
into another member's scopes or with unknown origins; /memory/query
requires member_id. Scope rules live in embedder_4070/scopes.py as
pure functions.
- brainstem: /generate refactored into _run_turn(member=...), shared
with /members/{id}/chat. Conversation turns land in the member's
private scope with participants from token attribution; the legacy
/generate path runs as the registry's first member so unscoped
writes no longer exist anywhere. Metric records gain member_id.
- migration: scripts/migrate_memory_scopes.py grandfathers pre-scope
rows into member #1's private scope — dry-run by default, idempotent,
batch update with reconciliation counts, never deletes.
49 tests pass (9 new).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011bnTL8M5fY9bMNjvzuyvyp
POST /members/{id}/memory/promote on the hub confirms a share:
reaching the endpoint is the person's yes (only people hold bearer
tokens), implementing decision 3's offer-then-confirm. The embedder's
/memory/promote copies the private row into shared:household with the
permanent paper trail — origin=promotion, promoted_from,
promoted_from_member, promoted_by — reusing the stored embedding so
the copy is vector-identical. The private original is never modified,
and the shared copy's deterministic id makes promotion idempotent per
source row. A member can only promote out of its own private scope.
55 tests pass (6 new).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011bnTL8M5fY9bMNjvzuyvyp
The inbox v0 from Card 2 becomes durable: queued messages persist to a JSON store with the same atomic-write discipline as the token and session stores, so a hub restart loses nothing. When a member's presence flips to awake, the hub drains its queue oldest-first through the exact same path as live chat — same hub-minted (person, member) session, same memory scoping, same metrics — so a message sent while the member slept is answered as if the sender had waited, and the reply is retrievable by msg_id. Custody is not memory: nothing touches any scope until the member actually processes the turn (covered by a test that counts memory writes across the queue/drain boundary). A cortex failure mid-drain leaves the remaining messages queued for the next wake. 58 tests pass (3 new). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011bnTL8M5fY9bMNjvzuyvyp
New node on the 4090 host owning weights placement and the llama.cpp
server lifecycle:
- tiering.py: ensure_hot() stages weights hot/warm/cold -> hot with a
sha256-verified copy that lands under a .staging name until checked,
so a torn copy can never be mistaken for a model. LRU eviction with
a pin set that never removes a pinned file or the only copy of a
weights file. llama.cpp does not tier for us — mmap off the HDD is
the failure mode this whole module exists to avoid.
- manager.py: load() = ensure_hot -> report waking -> spawn
llama-server (offload_policy from the registry) -> poll health ->
report awake; unload() stops the process and reports asleep; a load
failure reports asleep rather than lying. Emits stage_copy_ms,
staged_from, load_ms, member_id per load into the JSONL harness.
- server.py: /status (loaded members + which tier each member's
weights are on), /members/{id}/load, /members/{id}/unload. Manual
swap only in V1 — presence states and the hub's member_loading
contract are real from day one, and the hub's awake transition
already drains the inbox (Card 5), so load-finished and
queued-messages-answered are the same event.
64 tests pass (6 new); process lifecycle is exercised with real
subprocesses, only the HTTP health poll is faked.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011bnTL8M5fY9bMNjvzuyvyp
…rereg - Inbox drains emit a brainstem.inbox_drain record per answered message with member_id, msg_id, queue_wait_ms, and sender attribution — the waiting is measured, not just the turn (generate records already carry member_id since Card 3, and the model manager emits stage_copy_ms/load_ms since Card 4). - /fabric/status now carries the family roster (presence + queue depth per member) for the dashboard. - docs/sprints/SPRINT_5_BENCH_PREREG_2026-07-25.md pre-registers the member #1 baseline before any llama.cpp run: Sprint 3d prompt set through the hub path, quality guard, >=70% of vLLM AWQ median tokens_per_s, cold-start under 5 minutes, and an explicit nothing-tuned-before-baseline rule. The frozen baseline seeds member #1's report card (V2 Section 10). 65 tests pass (1 new). All seven Sprint 5 cards are now implemented. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011bnTL8M5fY9bMNjvzuyvyp
Ginkobaloba
changed the title
memory_system.md gains a 'Sprint 5: scoped memory' section covering scopes, provenance fields, the mandatory server-side query filter (and how it amends the Sprint 2 no-filter decision), copy-never-move promotion, and the migration script's dry-run/idempotent/reconcile behavior. readme.md's component list now describes the brainstem's Nexus Hub role with the member API contract and adds the model_manager_4090 node. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011bnTL8M5fY9bMNjvzuyvyp
compose.llamacpp.yaml runs the official CUDA llama-server image with the exact flags the model manager builds for member #1 (Q4_K_M gguf from the hot tier, ctx 32768, all layers on GPU, --no-mmap per vram_then_ram). The trtllm compose.yaml stays as historical reference. README.md documents the two bring-up paths (manager-spawned native process as the V1 default vs standalone compose), the MODELMGR_ env vars, the weights filename convention, and why mmap off the cold tier is the failure mode tiering exists to avoid. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011bnTL8M5fY9bMNjvzuyvyp
Renders the fabric/status family block: per-member presence with the existing status-dot pattern (awake green, busy/waking amber, asleep muted), queue depth highlighted when nonzero, and compact quant/ctx model info. Degrades gracefully against an older server payload with no family field. No new fetches, dependencies, or styles outside the existing tokens. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011bnTL8M5fY9bMNjvzuyvyp
The 'dog went outside at 5pm' path (V2 Section 8). Jetson-classified observations land born-shared: - embedder POST /memory/event writes a single shared:household row with origin=sensor, sensor_source, and reported_by (the delivering service's token) — sensor events never pass through any private scope and have no promotion step. - embedder GET /memory/timeline returns recent shared-scope rows newest-first (missing-ts rows sink rather than masquerading as recent); the where clause is fixed to shared:household — no parameter reaches anything private, verified by a test that plants a private row and asserts it never surfaces. - hub POST /household/events (auth; stamps the reporting token into provenance, defaults ts to now) and GET /household/timeline (auth — presence is dashboard material, what happened in the house is not). The embedder service itself is now under test with a real TestClient, stubbing only the model and Chroma seams. 72 tests pass (7 new). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011bnTL8M5fY9bMNjvzuyvyp
…ate volumes brainstem.Dockerfile installs pyyaml and copies family/ into the image at /app/family. The image layout drops the nodes/ prefix, so server.py's repo-root walk (parents[2]) would miss the registry in-container — compose now points BRAINSTEM_FAMILY_REGISTRY_PATH at the absolute path instead, which also anchors spec-file resolution correctly. Compose gains session_data:/data/sessions and inbox_data:/data/inbox named volumes so hub sessions and queued messages survive restarts and rebuilds. No model-manager service here — it runs natively on the 4090 host. Validated with docker compose config. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011bnTL8M5fY9bMNjvzuyvyp
CLI: --family lists the roster; --member [id] chats via
/members/{id}/chat (default_member from config.json, 'vera'); 202
queued replies poll the inbox status URL (--poll-interval/--max-wait,
give-up prints the exact --check-inbox command to resume); 503
member_loading honors Retry-After with bounded retries; REPL gains
/family, /member <id>, /legacy. Member requests deliberately omit
temperature unless the user set one (the member's registry sampling
default applies) and never send X-Session-Id (sessions are hub-minted).
Legacy /generate behavior is untouched.
Web: member picker with presence dots and queue depth, queued-message
waiting state with polling and a resume button, member_loading retry
countdown. serve.py proxy allowlist grows exactly two anchored member
path shapes (chat POST, inbox GET) and now forwards Retry-After —
still not an open relay.
Verified by the full suite (72 passing) plus an end-to-end run against
a mock brainstem covering queue, resume, loading-retry, and REPL flows.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011bnTL8M5fY9bMNjvzuyvyp
Five cards implementing the receptionist subset of V2 Section 9: concierge registry block (staff, not family — no private scope, not on the roster), data-only briefing digest with a test-enforced privacy invariant, Jeffery's dense-8B runtime on the 4070, the spoken briefing that digests only the R2 JSON, and wake-cycle integration attaching the briefing to the first drained turn. Delegation, ledger, scoring, and trust gates stay explicitly out of scope until V2.x per the decision record. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011bnTL8M5fY9bMNjvzuyvyp
…briefing
R1: family/registry.yaml gains a concierge block — staff, not family.
Jeffery (dense 8B Q5 per decision 4; swap the entry, not the code) has
no memory block, no storage tier, never appears on the family roster,
and an id collision with a member is a boot failure. Receptionist spec
at family/jeffery/spec.md: briefs from shared data only, no tools,
declines anything beyond a briefing.
R2: GET /members/{id}/briefing assembles the wake-up digest from
exactly two sources — inbox custody metadata and the shared-scope
timeline — windowed to the member's last sleep edge (24h fallback
before first sleep; sleep/wake timestamps persist in the v2 inbox
store, which still reads Sprint 5 v1 files). Privacy invariant under
test: queued message contents appear nowhere in the payload (a
planted secret phrase is asserted absent), and the household feed
comes only from the embedder's shared-only path. A dead embedder
degrades the briefing (empty events + error note) rather than
breaking wake.
78 tests pass (6 new).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011bnTL8M5fY9bMNjvzuyvyp
Registry model blocks gain an optional gguf_repo (base repos ship safetensors; the quants live in quantizer repos — vera: unsloth/Qwen3-30B-A3B-Instruct-2507-GGUF, jeffery: unsloth/Qwen3-8B-GGUF). Also fixes Jeffery's base source: Qwen3-8B has no separate -Instruct repo. scripts/fetch_weights.py downloads a member's (or the concierge's) GGUF into a storage tier under the canonical weights_filename() name, so ensure_hot() finds it without ceremony. Reads the HF token from HF_TOKEN / HUGGINGFACE_TOKEN / HUGGING_FACE_HUB_TOKEN — per SECURITY.md the token stays in the gitignored docker/.env, never in the repo. Handles split GGUFs (parts kept, note printed), --list for inspection, --tier/--dest for placement. Quant file selection lives in tiering.select_gguf_files (case-insensitive, split-aware, tested). 81 tests pass (3 new). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011bnTL8M5fY9bMNjvzuyvyp
R4: GET /members/{id}/briefing?spoken=true runs the R2 data digest
through Jeffery (llama-server speaks the same OpenAI-compatible API as
the cortex, so the existing client serves). His entire input is spec +
the R2 JSON — the privacy boundary now holds on the prompt side too,
under test. Any failure falls back to data-only: R2 is the contract,
R4 is the voice. Emits briefing_build_ms/briefing_tokens metrics.
concierge_url unset = receptionist not deployed; everything degrades
gracefully, and /fabric/status reports the concierge as
not_deployed / up / down alongside the family roster.
R5: when a member with briefing_on_wake (registry flag, default on)
wakes with mail, the briefing rides as system context on the FIRST
drained turn only — spoken if Jeffery is up, data digest otherwise —
so the member triages oriented without reciting the briefing on every
reply. Drain metrics gain briefing_attached.
84 tests pass (3 new).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011bnTL8M5fY9bMNjvzuyvyp
Always-on concierge service (ghcr.io/ggml-org/llama.cpp:server-cuda, port 8001, loopback-only host bind — the hub reaches him over the compose bridge, nothing outside the box needs him). Flags scale the member-#1 conventions to Jeffery's registry entry: Qwen3-8B.Q5_K_M gguf, ctx 8192, all layers on GPU, --no-mmap. Weights arrive via scripts/fetch_weights.py --member jeffery --dest data/weights/concierge. The brainstem gains BRAINSTEM_CONCIERGE_URL=http://jeffery_4070:8001 but no depends_on — a missing Jeffery degrades spoken briefings to the data digest by design, so the hub must come up without him. wget-based healthcheck (the server-cuda image ships no curl or python). 84 tests pass. Sprint 6 R1-R5 are now all implemented. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011bnTL8M5fY9bMNjvzuyvyp
Ginkobaloba
changed the title
Owner
Author
|
Swept this as part of a repo-wide open-PR triage. Pushed one fix, but I am Fixed and pushed
That one is squarely this branch's bug and is now declared. What I actually measuredFresh venv, Python 3.13.3, Not 84 passing. Worth reconciling before this lands. The remaining failures are NOT this branch's faultAlmost all 20 failures and all 38 errors trace to a single root cause in
This is pre-existing on Suggested order
Left in draft for you. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements the V1 scope of the accepted V2 architecture (
docs/architecture_v2_family_of_models.mdSection 11) and the V1.5 receptionist phase (Section 9.6). One commit per card, PM-reviewed delegated waves for docs/clients/deployment. 84 tests passing (53 new).Sprint 5 — V1 (
docs/sprints/SPRINT_5_PLAN_2026-07-25.md)Card 1 — Family registry + loader (
family/registry.yaml,core/family.py): single source of truth for the roster, seeded with member #1 vera (Qwen3-30B-A3B GGUF Q4_K_M). Unknown keys, duplicate ids, bad offload policies, missing/empty spec files = hard boot failures. Member #2 is a yaml entry + spec file, proven by test.Card 2 — Hub member routing + presence:
GET /family,GET /members/{id},POST /members/{id}/chat(200 live / 202 queued + msg_id / 503member_loading+ Retry-After). Hub-minted persisted sessions per (person, member) — the restart-resets-turn_idx wart is fixed.Card 3 — Scoped memory + provenance: every row carries scope/member_id/origin/participants; queries are server-side filtered to
private:M + shared:household + experiential:M— cross-member recall structurally impossible, cross-session recall within a member preserved. Migration script grandfathers pre-scope rows (dry-run default, idempotent, never deletes).Card 4 — Model manager v0 (
nodes/model_manager_4090):ensure_hot()sha256-verified staged copies across hot/warm/cold tiers; LRU eviction that never removes a pinned file or sole copy; llama-server lifecycle with presence reporting andstage_copy_ms/load_msmetrics.Card 5 — Durable inbox, drained on wake: queued messages survive restarts; wake drains oldest-first through the normal chat path. Custody is not memory — nothing touches a scope until the member processes the turn.
Card 6 — Promotion (copy-never-move): person-confirmed sharing into
shared:householdwith the full paper trail; deterministic copy id = idempotent; the private original is never touched.Card 7 — Metrics + bench prereg:
member_idon every record,queue_wait_msper drained message, roster on/fabric/status, pre-registered member #1 baseline (SPRINT_5_BENCH_PREREG_2026-07-25.md).Sprint 6 — V1.5 receptionist (
docs/sprints/SPRINT_6_PLAN_2026-07-26.md)R1 —
concierge:block in the registry: Jeffery (dense Qwen3-8B Q5_K_M per decision 4) is staff, not family — no private scope, never on the roster, id collision with a member is a boot failure. Receptionist spec: briefs from shared data only, no tools, declines anything beyond a briefing.R2 —
GET /members/{id}/briefing: data-only wake digest from exactly two sources (inbox custody metadata + shared timeline), windowed to the member's persisted sleep edge. Privacy invariant under test: queued message contents appear nowhere in the payload.R3 — Jeffery's llama-server in the 4070 compose stack (loopback-only bind, no
depends_on— the hub must come up without him), reported on/fabric/statusasnot_deployed/up/down.R4 —
?spoken=trueruns the R2 JSON through Jeffery for the morning-report prose; his entire input is spec + digest (privacy boundary holds on the prompt side, tested); any failure falls back to data-only.briefing_build_ms/briefing_tokensmetrics.R5 — wake-cycle integration: members with
briefing_on_wake(registry flag, default on) get the briefing as system context on the first drained turn only. Drain metrics gainbriefing_attached.Delegated waves (PM-reviewed, test-gated per commit)
Docs refresh (
memory_system.md,readme.md); dashboard Family panel; llama.cpp runtime compose + operator README for the 4090; CLI/web member API support (202 polling with resume,member_loadingretries, proxy allowlist +2 anchored shapes); Sprint 5 docker wiring (registry in image via absolute-path override, durable session/inbox volumes); household sensor feed (POST /household/eventsborn-shared,GET /household/timeline— verified unable to surface a planted private row); registry-driven weights fetcher (scripts/fetch_weights.py,gguf_repofields pointing at the real quant repos; also fixed Jeffery's base toQwen/Qwen3-8B— no Instruct variant exists).First bring-up (operator steps)
python scripts/fetch_weights.py --member vera(cold tier) and--member jeffery --dest data/weights/concierge— HF token from the gitignoreddocker/.envper SECURITY.md; both default repos are public apache-2.0.docker compose upon the 4070;scripts/migrate_memory_scopes.pydry-run, then--apply./members/vera/load→ staging + llama-server + presence → inbox drains with briefing.SPRINT_5_BENCH_PREREG_2026-07-25.md.Out of scope per the V2 phasing: delegation ledger, scoring, and trust gates (V2.x), Project Vector, self-training, Sprint 4 callback.
Note:
reqirements.txtgainspyyamlandhuggingface_hub; developed against pydantic v2 +argon2-cffi.