Skip to content

docs(opmem): publication readiness package (13-task pins, no re-run) - #189

Draft
sdntsng wants to merge 1 commit into
devfrom
pr/opmem-publication-readiness-0a7c
Draft

sdntsng wants to merge 1 commit into
devfrom
pr/opmem-publication-readiness-0a7c

Conversation

@sdntsng

@sdntsng sdntsng commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

Summary

OpMem-first publication package built only from existing repo evidence. No benchmarks were run and no provider spend. The goal is to state clearly what is ready for a Paper 1 submission versus what is still missing, and to align normative docs with the live 13-task suite (upd03).

Independent static review confirms the frozen 2026-08-15 pin: Brainy 13/13 and Mem0 Platform 10/13 are supported by markdown artifacts and Brainy per-task JSON; we do not claim re-validation on current bbe55f7 or inflate Platform numbers as OSS-reproducible.

Changes

  • docs/research/opmem/PUBLICATION_READINESS.md — full package:
  • docs/research/opmem/README.md — index and pin summary
  • Stale 12→13 task wording (provable only):
    • docs/research/opmem-spec.md — taxonomy includes upd03, 13 tasks
    • docs/research/public-bench-ladder.md — L0 OpMem row → 13/13 vs 10/13
    • docs/research/mem0-parity-gaps.md — executive OpMem row
    • docs/research/master-plan.md — §1.1 OpMem row
    • docs/research/posts/2026-07-opmem-v0.md — supersession banner (July 12/12 numbers preserved as historical)

Intentionally not rewritten: dated artifacts (opmem-baseline-report.md, opmem-staging-vs-mem0.json, launch-narrative.md, etc.) remain historical 12-task records.

No product code changes.

Test plan

  • find fixtures/opmem -name '*.json' | wc -l → 13
  • opmem-spec.md no longer says v0: 12 tasks
  • go test ./... — not required (docs-only)
  • OpMem re-run on bbe55f7 — explicitly out of scope (no spend)

Linear / tracking

OpMem Paper 1 track (paper-topics.md). Inventory cross-ref: PR #188 (inventory-reproduction-plan-2026-09-18.md).

Risk / follow-ups

  • Not publication-ready: no LaTeX manuscript, no ≥4-system table, no committed 3-system JSON for the 2026-08-15 pin, no bbe55f7 re-measure.
  • CI does not assert 13/13 — run_opmem.py exits 0 on task failures; documented as a threat.
  • PR S5: versioned RRF retrieval for search and /recall #187 (RRF): merge should trigger OpMem re-run before claiming ops non-regression.
  • Do not merge to main without explicit approval (staging = dev only).

Ready vs missing (from package)

Ready Missing
13 fixtures + runner + 3 adapters Zep/Letta/LangMem; v1 ~30 tasks
Protocol + spec + CI harness health OpMem RunManifest / frozen multi-system JSON in git
13/13 vs 10/13 markdown + Brainy integrity JSON Mem0 per-task JSON in git; OSS run; SHA bbe55f7 re-pin
Manuscript outline + threats Camera-ready paper, related work, external fixture audit
Open in Web Open in Cursor 

Add docs/research/opmem/PUBLICATION_READINESS.md with reproducible protocol,
claims/evidence table, independent 13/13 vs 10/13 review, missing work,
threats to validity, artifact instructions, manuscript outline, and PR
#185-#187 overlap map.

Fix provably stale 12-task wording in opmem-spec, public-bench-ladder,
mem0-parity-gaps, and master-plan; banner July opmem post as pre-upd03.

Co-authored-by: aria <aria@engramhq.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants