🧠 DeepMemory

English | 中文

Deconstruct language with Peirce's semiotics. Govern group memory with game theory. The first AI memory layer that mines your agent's subconscious — and lets a swarm of agents argue out their disagreements.

Your AI has the memory of a goldfish.

Every time you open a new chat, it forgets what worked yesterday. Every time two agents disagree, the loudest one wins. Every time you fix a bug, the next agent asks the same question again.

DeepMemory fixes this in 5 lines of code.

It's not another vector database. It's a governance layer — built on 100 years of semiotic philosophy (Charles Sanders Peirce's sign / object / interpretant triad) and 80 years of game theory (Vickrey, Nash, Shapley). All of it compiled into pure Python that runs on your laptop. No OpenAI key. No GPU. No cloud.

👋 New here? Start with Getting Started (15 min).

---

⚡ 30-Second Demo

import random
from deepmemory.cognition.bandit import ThompsonRanker
from deepmemory.cognition.bayesian import BayesianConfidenceEngine, FeedbackSignal
from deepmemory.policies.cognition import BayesianBanditCognitionPolicy
from deepmemory.runtime.engine import DeepMemoryRuntime
from deepmemory.runtime.feedback import record_recall_feedback
from deepmemory.storage.in_memory import InMemoryMemoryStore

# Set up a memory that *learns*
bayesian = BayesianConfidenceEngine(cold_start_threshold=2.0)
runtime  = DeepMemoryRuntime(
    store=InMemoryMemoryStore(),
    cognition=BayesianBanditCognitionPolicy(
        bayesian, ThompsonRanker(bayesian, rng=random.Random(42))
    ),
)

# Use it like normal — but feed back what worked
top = runtime.recall_for_object("module:upload")[0]
record_recall_feedback(runtime, top.memory_id, FeedbackSignal.RECALLED_USED)

That's it. Five lines and your agent now learns from feedback.

📈 Verified jump: top-1 hit rate from 20.5% → 66.8%, p < 1e-6 (Phase 1 A/B report).

🤔 Sound familiar?

You've tried...	And hit this wall...	DeepMemory's answer
📋 Stuffing context into every prompt	"I told you yesterday!" Your AI forgets across sessions.	Persistent + learns which notes deserve trust
🔍 Vanilla RAG / vector DB	Retrieves similar, not correct. The wrong answer is the most-similar answer.	Bayesian posterior per memory — the system grades itself
🤖 OpenAI Assistants API	$$, vendor lock-in, no idea why it picks what it picks.	100% local, free, fully introspectable
🛠 Hand-rolled SQL "memory" tables	No conflict detection, no decay, no rep system, no joy.	All built in. Optional. Boring to set up. Dramatic in effect.
⛓️ LangChain memory chains	Heavy deps, opaque ranking, breaks every release.	5-line opt-in. Transparent algorithms. No magic.

🔬 The Big Idea: Two Theories, One System

We didn't invent memory governance. We stole the answer from two century-old academic traditions and compiled them into Python.

1. Single-agent cognition: 🪞 Peirce's Sign / Object / Interpretant

Charles Sanders Peirce, 1900s philosopher, figured out something LLMs still struggle with: a sign isn't the thing it points to. Your AI saying "fix the lock" is a sign; the actual lock contention bug is the object; the agent's interpretation of why it works is the interpretant. Most memory systems collapse all three. DeepMemory keeps them apart, then runs Bayesian inference on the interpretant to learn which interpretations actually pay off.

That's how we mine your agent's subconscious: by tracking, per memory, which interpretations get rewarded by reality.

2. Multi-agent governance: ⚖️ Game theory does the arguing

When 2+ agents disagree about the same object, you need a way to settle it that doesn't reward the loudest voice. We use:

• Vickrey auctions (Nobel-prize algorithm) — memories bid for attention; honest bidding is the dominant strategy.
• Nash equilibrium — when two agents contradict, solve the bimatrix game; output is an adoption probability per memory.
• Shapley values — fairly attribute credit when a recall set collectively succeeds.
• Reputation + VCG — agents who consistently produce useful notes earn admission privileges; spammers get throttled in ~6 rounds.

This is how we stop semantic drift in agent populations: by giving the math, not the loudest agent, the final word.

🧩 What You Get

Each layer is opt-in and stacks on the previous one.

🧠 Cognition (single agent)

Module	One-line description
BayesianConfidenceEngine	Beta distribution per memory. Learns from every "this worked" / "this didn't".
ThompsonRanker	Multi-armed bandit ranking. Balances "use what's proven" with "try what's new".
Bm25Index / TfidfIndex	Search across memories by meaning, not just by ID.
ContradictionDetector	Auto-flags "should X" vs "should not X" pairs for human review.
EwmaRegistry / CUSUM	Useful memories get a longer life; stale ones decay faster.
MctsMemoryComposer	Find the best combination of memories for a complex task.

⚖️ Game Theory (multi-agent)

Module	One-line description
VickreyAuction	Memories silently bid for attention. The winner gets shown; honest bidding wins.
solve_nash	Two memories disagree? Solve the equilibrium, output adoption probabilities.
shapley_*	A team of memories nailed it. Who deserves how much credit?
ReputationRegistry + VcgMechanism	Reputation that learns. Spammers throttled. Top contributors fast-tracked.

🛠 Production-ready

• Restart-safe — append-only JSONL on every state store
• Thread-safe — RLock everywhere; multi-threaded use officially supported
• Observable — runtime.cognition_failed events; cognition_degraded flag on degraded recalls
• Manageable — 17 CLI commands, including reputation-list, state-compact, beta-state-export

🚀 Real Numbers, Not Vibes

Phase 1 — Probabilistic Foundation
  baseline:    20.5% top-1 hit rate
  challenger:  66.8% top-1 hit rate     ←  +46.3pp,  p < 1e-6

Phase 3 — Game-Theoretic Governance (multi-agent)
  baseline:    96.8% top-1 hit rate
  challenger: 100.0% top-1 hit rate     ←  Vickrey is *deterministic*

Phase 4 — Reputation in Action (3 agents, 50 rounds, 1 spammer)
  spammer detected by round   ~6
  expected_task_success:      0.860 → 0.882

Reproduce: PYTHONPATH=. python3 scripts/run_phase1_ab.py --seeds 1,2,3,4,5

📦 Install in 30 Seconds

git clone <repo> && cd deepmemory
python3 -m venv .venv && source .venv/bin/activate   # recommended: use a venv
pip install -e .                                      # core install (pure stdlib, zero external deps)
deepmemory version           # → deepmemory 2.1.0-rc1
deepmemory capabilities      # see what's inside

Optional dependencies

DeepMemory core has zero external dependencies (pure stdlib). Install extras as needed:

pip install -e ".[http]"    # HTTP server (FastAPI + Uvicorn + Pydantic)
pip install -e ".[mcp]"     # MCP server (Model Context Protocol)
pip install -e ".[dev]"     # dev deps (includes http + pytest)
pip install -e ".[http,mcp]"  # both HTTP and MCP

🌐 HTTP Server (v1.1)

Run DeepMemory as a long-lived REST service so any HTTP-capable client (Go, Node, front-end, gateways) can integrate without coupling to the Python SDK. The HTTP surface coexists with the stdio MCP server — both wrap the same DeepMemoryFacade.

pip install -e '.[http]'
DEEPMEMORY_STORE=./artifacts/memory.jsonl deepmemory-http
# → uvicorn on http://127.0.0.1:8765
# → OpenAPI:    http://127.0.0.1:8765/openapi.json
# → Swagger UI: http://127.0.0.1:8765/docs

Endpoints (all under /api/v1/):

Method	Path	Purpose
GET	/health	liveness + memory count + daemon status
POST	/memory/ingest	create one memory unit
GET	/memory/recall?object_id=…	recall by object
POST	/memory/recall_multi	batch recall + dedupe + status-priority sort
POST	/memory/recall_by_prefix	prefix scan over object_id (handles LLM key drift)
POST	/memory/{id}/promote	governance state transition
POST	/memory/{id}/revise	update claim / confidence / context
DELETE	/memory/{id}	delete with reason
POST	/memory/ingest_async	enqueue raw turn text → LLM extraction worker
GET	/memory/ingest_async/{task_id}	poll async task status
POST	/memory/evolve	run one evolution cycle on demand
GET	/memory/events	tail event log

All responses use the envelope {"code": 0, "message": "ok", "data": …}.

Configuration (env vars)

Variable	Default	Effect
DEEPMEMORY_STORE	./artifacts/memory.jsonl	JSONL store path; events go alongside as *_events.jsonl
DEEPMEMORY_DEFAULT_ACTOR	http-server	actor_ref for server-originated writes
DEEPMEMORY_HTTP_HOST / DEEPMEMORY_HTTP_PORT	127.0.0.1 / 8765	bind address
DEEPMEMORY_API_TOKEN	(unset)	when set, all endpoints except /health require Authorization: Bearer …
DEEPMEMORY_EVOLVE_INTERVAL_HOURS	0	when > 0 launches the in-process evolution daemon
DEEPMEMORY_EVOLVE_MIN_CONFIDENCE / DEEPMEMORY_EVOLVE_STALE_HOURS / DEEPMEMORY_EVOLVE_MAX_PER_OBJECT	0.35 / 168 / 20	daemon EvolutionPolicy overrides

/health is intentionally exempt from auth so that K8s liveness / readiness probes work without leaking the token. The response includes extractor_llm ∈ {noop, configured, none} — see the production note below.

⚠️ Production note: ingest_async requires a real LLM client

The default AsyncExtractor (used when deepmemory-http starts without an explicit client wired in) falls back to a _NoopLLM stub that always returns "[]". The /memory/ingest_async endpoint will look healthy at the protocol layer (200 OK, task transitions queued → running → done) but no MemoryUnits will ever be produced — ingested stays [] and recall keeps returning empty.

Two ways to detect this:

1. GET /health returns extractor_llm: "noop" when the stub is in use.
2. The wire-it-in path: build a client implementing async def acomplete(prompt, **kw) -> _LLMResponse (a duck-typed object with a .content JSON string) and pass it to the extractor:

from deepmemory import DeepMemoryFacade
from deepmemory.runtime.async_extractor import AsyncExtractor
from deepmemory.http_server import create_app

facade = DeepMemoryFacade(store_path="./artifacts/memory.jsonl",
                          event_path="./artifacts/events.jsonl")
extractor = AsyncExtractor(facade=facade, llm_client=my_real_llm_client)
app = create_app(facade=facade, extractor=extractor)

Alternatively, do the LLM-side extraction in your gateway and call the synchronous /memory/ingest endpoint with already-structured triples — that path has no LLM dependency.

MCP Server (stdio)

Run DeepMemory as an MCP tool server for AI toolchains (Claude Desktop, Cursor, etc.):

pip install -e ".[mcp]"
deepmemory-mcp
# or: python -m deepmemory.mcp_server

MCP client configuration example:

{
  "mcpServers": {
    "deepmemory": {
      "command": "python",
      "args": ["-m", "deepmemory.mcp_server"],
      "env": {"DEEPMEMORY_STORE": "./artifacts/memory.jsonl"}
    }
  }
}

Startup modes comparison

Mode	Best for	Start command	Dependencies
CLI	Daily ops, quick testing	deepmemory ingest/recall/...	Core install
Python SDK	Embed in agent code	import deepmemory	Core install
HTTP server	Microservice / gateway integration	deepmemory-http	pip install ".[http]"
MCP server	AI toolchain (Claude/Cursor)	deepmemory-mcp	pip install ".[mcp]"

🎯 Daily-Use Cheatsheet

# Who do I trust?
deepmemory reputation-list  --state-dir ./agent_state

# What's contradicting itself?
deepmemory conflict-scan    --events events.jsonl

# Inspect a memory's confidence
deepmemory beta-state-export --state-dir ./agent_state --memory-id mu_xxx

# Punish a spammer manually
deepmemory reputation-set   --state-dir ./agent_state --actor X --delta -0.5

# Reclaim disk
deepmemory state-compact    --state-dir ./agent_state

# v1 stable lifecycle (unchanged)
deepmemory ingest / recall / promote / revise / delete
deepmemory run-harness / run-drift-audit / run-evolution

🛠 Documentation by Audience

You are...	Read this
🆕 Brand new	Getting Started (15 min) (中文)
⬆️ Upgrading from v1	Migration Guide (中文)
🚀 Going to production	Deployment Guide (中文)
📟 On-call SRE	Operations Runbook (中文)
❓ Just curious	FAQ (中文)
📜 Schema-pedantic	core-schema v0.2 (中文)
🤝 Compat-pedantic	compatibility-statement (中文)
🧠 Philosophical	v2 Grand Summary
📚 Everything	docs/README.md

📂 Repo Tour

deepmemory/
├── core/            # Sign / Object / Interpretant — Peirce's triad as code
├── runtime/         # Lifecycle engine, hooks, persistence layout
├── cognition/       # Pure-stdlib algorithm layer
│   ├── bayesian/    # Beta posteriors learn from feedback
│   ├── bandit/      # Thompson sampling — explore vs exploit
│   ├── similarity/  # TF-IDF · BM25 · contradiction detection
│   ├── timeseries/  # EWMA · CUSUM
│   └── mcts/        # Monte Carlo composition search
├── gametheory/      # Pure-stdlib game theory layer
│   ├── auction/     # Vickrey
│   ├── nash/        # Bimatrix support enumeration
│   ├── shapley/     # Exact + Monte Carlo
│   ├── evolutionary/# Replicator dynamics
│   └── mechanism/   # Reputation + VCG
├── policies/        # 4 cognition policies + game policies
├── harness/         # Event-driven metrics + paired A/B comparison
├── packs/coding/    # Coding-domain pack
├── adapters/        # External substrate adapters
└── cli/             # 17 CLI commands

examples/            # Runnable demos for each layer
scripts/             # A/B benchmark scripts
tests/               # 304 tests, zero regression on v1 baseline
docs/                # Bilingual user docs (en + zh)

🧪 Hack on it

python3 -m pytest                                 # 304 tests, ~0.5s
PYTHONPATH=. python3 examples/bayesian_evolution_demo.py
PYTHONPATH=. python3 scripts/run_phase1_ab.py --seeds 1,2,3,4,5

📜 License

MIT.

📌 Status

v2.1.0-rc1 is in closed-R&D phase — no production users yet. Schema and APIs are stabilizing; minor breaking changes still allowed before the first true GA. Old v0.1 jsonl files auto-migrate to v0.2 on read; 51 v1 tests pass byte-level identical.

"Semiotics deconstructs individual cognition. Game theory empowers collective intelligence."

— DeepMemory v2 design philosophy

---

⭐ If this resonates, drop a star — it's how we know the philosophical bet is paying off.