BEAM 100K
+73.3%
++ Accuracy · 400 probes · 20 conversations +
++ Kimetsu · graph retrieval +
++ Both runs use the same test set. +
+ + BEAM test settings → + +diff --git a/content/docs/memory-benchmark/beam.mdx b/content/docs/memory-benchmark/beam.mdx index 8794ce2..3b6b177 100644 --- a/content/docs/memory-benchmark/beam.mdx +++ b/content/docs/memory-benchmark/beam.mdx @@ -71,8 +71,9 @@ of the 35 conversations (300 probes), uniform 96k budget. | abstention | 9/30 (30%) | | **overall** | **198/300 (66.0%)** | -- **66.0% edges mem0's self-reported 62% at the matched bucket.** Caveats in - [How Kimetsu compares](/docs/memory-benchmark/comparison). +- **Kimetsu scores 66.0%; mem0 reports 64.1% in the 1M bucket.** The readers + and conversation samples differ. Sources and test differences are recorded + in [How Kimetsu compares](/docs/memory-benchmark/comparison). - **Global and temporal abilities degrade with scale, as expected.** When the conversation is ~10x the budget, whole-arc tasks lose ground that local-fact tasks keep. diff --git a/content/docs/memory-benchmark/comparison.mdx b/content/docs/memory-benchmark/comparison.mdx index a0fed0a..7552e30 100644 --- a/content/docs/memory-benchmark/comparison.mdx +++ b/content/docs/memory-benchmark/comparison.mdx @@ -22,14 +22,15 @@ bill, or the cloud.** | LongMemEval (`_s`) | **83.0%** (200-q slice) · ~80.9% weighted | 94.4% (their reader + harness) | not reported | | LoCoMo (1,540 q) | **89.4%** | 92.5% | not reported | | BEAM 100K | **73.3%** (400 probes) | n/a | 79% | -| BEAM **1M** | **66.0%** (300 probes) | 62% | not reported | +| BEAM **1M** | **66.0%** (300 probes) | 64.1% (700 probes) | not reported | | BEAM 10M | future work | 48.6% | 67% | Caveats, because the table is not apples-to-apples: -- **Only the 1M row is a matched bucket.** Our 66.0% edges mem0's 62%, but with - a different reader and 15 of 35 conversations: read it as at least on par, - not a decisive win. +- **The 1M row uses the same token bucket, with different setups.** Kimetsu's + 66.0% covers 15 of 35 conversations; mem0 reports 64.1% over 700 probes. + The readers and samples differ, so the score difference does not establish + a head-to-head win. - **Cognee leads at 100K/10M.** Our 73.3% matches the prior public state of the art on 100K (the 0.735 Cognee cites as the number it beat), model-free. Cognee needs an LLM key on both the write and read paths. @@ -46,6 +47,9 @@ Cognee's [BEAM figures](https://github.com/topoteretes/cognee), the [LongMemEval](https://arxiv.org/abs/2410.10813) and [BEAM](https://github.com/mohammadtavakoli78/BEAM) papers. +Mem0's BEAM 1M figure was checked on September 12, 2026. Its current reported +64.1% replaces the 62% previously quoted here. + ## What we do not yet claim - Multi-hop retrieval of obliquely relevant memories is v2.6 work; the diff --git a/content/docs/memory-benchmark/index.mdx b/content/docs/memory-benchmark/index.mdx index b410b1e..9abfa11 100644 --- a/content/docs/memory-benchmark/index.mdx +++ b/content/docs/memory-benchmark/index.mdx @@ -21,7 +21,7 @@ on v2.8.0. |-----------|--------| | LoCoMo | **89.4%** (1,540 questions, LLM-judged accuracy) | | BEAM 100K | **73.3%**, matching the prior public state of the art, model-free | -| BEAM 1M | **66.0%**, ahead of mem0's self-reported 62% | +| BEAM 1M | **66.0%** (300 probes, 15 of 35 conversations) | | LongMemEval (`_s`) | **83.0%** (200-question stratified slice) | | BrainBench quality index | **80.0%** (142 scenarios, reader-free) | | retrieval quality | recall@4 **0.949**, MRR **0.914** at ~138 ms | diff --git a/src/app/(home)/_components/about-contact.tsx b/src/app/(home)/_components/about-contact.tsx index 0eb19ac..1eec532 100644 --- a/src/app/(home)/_components/about-contact.tsx +++ b/src/app/(home)/_components/about-contact.tsx @@ -12,32 +12,25 @@ export function AboutContact() { >
About
- Kimetsu is an open-source project created by{" "} + Created and maintained by{" "} Rodrigo Córdoba - . It gives coding agents a local place to keep project decisions, - conventions, and fixes, so useful knowledge carries across - sessions. -
-- Built in Rust and backed by SQLite, Kimetsu works alongside the - tools you already use. The source, documentation, and benchmarks - are public, so you can inspect how it works and make it your own. + . Written in Rust and released under the MIT and Apache-2.0 + licenses.