Repository navigation
Commit ff9c5db
authored
Harden the memory search ranking guard so a broken order fails (#2849)
Fixes #2846
Owner: Dev-1
Reviewer: Birfy
Written-by: Dev-1
Bot: Dev-1
Claim-Id: 1afedce05a604ff9a3f7ea8edf0d79fc
Risk tier: T1
Base: 429668e
## The guard
`core/test/memory-search-edge.test.js` is the edge-case file for
`searchIndex`, and its determinism test called the same expression on
both sides of `deepEqual`:
```js
assert.deepEqual(searchIndex(index, 'flight bookings').map(hit => hit.doc.key), searchIndex(index, 'flight bookings').map(hit => hit.doc.key))
```
Two calls on the same index with the same query, compared with each
other — no input, no ordering rule, nothing that can disagree. Every
other assertion in `core/test/**` (253 files, 13,829 assertions scanned)
either pins a literal or constrains a value; this was the only literal
self-comparison left.
The ranking it names is user-visible: `recall` answers with
`searchIndex(buildIndex(docs), query, { k: RECALL_MAX + 1 })`
(`core/src/host/memory.js:1130`), and the index is rebuilt from the
memory files on every read, so ordering is what the user sees when the
list is long.
## What the tip asserts
The test now pins the three things that "deterministic ranking" has to
mean, through one helper that takes the entries and a query:
- Two entries that share every term score the same, so the key breaks
the tie: `['a', 'b']`.
- Re-reading the files in another order does not reorder the result: the
reversed array still ranks `['a', 'b']`.
- An entry matching both query words ranks above one that matches a
single word: `['both', 'only-window']`.
No source file changes. 2 files, +11/−6 each (the test and its generated
mirror).
## Red first
On `429668e2` with only this test changed, I broke the behaviour under
test in `core/src/host/memory-search.js` (twice, each time restored with
`git checkout --`), and ran the file twice per mutation; the old
assertion and the new one saw the same source.
Mutation A — drop the `||
String(a.doc.key).localeCompare(String(b.doc.key))` tie-break at
`memory-search.js:101`:
```
old assertion, run 1: # tests 5 / # pass 5 / # fail 0
old assertion, run 2: # tests 5 / # pass 5 / # fail 0
new assertion, run 1: # tests 5 / # pass 4 / # fail 1
new assertion, run 2: # tests 5 / # pass 4 / # fail 1
not ok 2 - a word stem matches across endings, and the ranking is deterministic
at line 25: actual ['b','a'] expected ['a','b']
```
Mutation B — sort worst-first while keeping the key tie-break (`a.score
- b.score || String(a.doc.key).localeCompare(...)`):
```
old assertion, run 1: # tests 5 / # pass 5 / # fail 0
old assertion, run 2: # tests 5 / # pass 5 / # fail 0
new assertion, run 1: # tests 5 / # pass 4 / # fail 1
new assertion, run 2: # tests 5 / # pass 4 / # fail 1
not ok 2 - a word stem matches across endings, and the ranking is deterministic
at line 29: actual ['only-window','both'] expected ['both','only-window']
```
Source restored, the new test is `5/5` twice, and the three neighbouring
suites (`memory-search`, `memory-search-edge`, `memory`) are `116/116`.
## What I ran at this tip
- `core/test/memory-search-edge.test.js`: `5/5` green.
- Neighbourhood (`memory-search`, `memory-search-edge`, `memory`):
`116/116`.
- `git diff --name-only 429668e..HEAD`:
```
core/test/memory-search-edge.test.js
patches/ds-bot/files/test/memory-search-edge.test.js
```
- `TZ=UTC npm run verify` at the tip: exit 0 — core 3091/3091, social
619, tree 149, timeline 172, workflow 528, project 147, scripts+policy
675, interop 37, live-e2e 34; core lint 88 warnings / 0 errors; `10
patches apply cleanly in order` and canonical no-op. Its internal
`arch-docs: OK` line counts against this clone's `origin/main` ref
(`f42ad016`, the fork's older main), so its file count is not this PR's;
the explicit `ARCH_DOCS_BASE=429668e2` run below is.
- `ARCH_DOCS_BASE=429668e2 node core/scripts/check-arch-docs.mjs` on the
committed tree: `arch-docs: OK (2 changed file(s))`.
Not run: `build-client` (no `core/src/client/` change), the smoke test
(no host boot change), and the merge-queue checks (CI runs them).
## Generated artifacts
`node scripts/ds-bot-patches.mjs generate` rewrites exactly one path
beyond the source test: its mirror
`patches/ds-bot/files/test/memory-search-edge.test.js` (+11/−6, same
bytes as the source file). All ten `patches/ds-bot/0*.patch` files are
byte-identical to `429668e2` — the mirror is not referenced by any
patch. `check` says `ok ds-bot/ = Fei2411/DeepSeek-Bot@2b10c70` / `ok 10
patches apply cleanly in order` / `ok core/ equals ds-bot/ +
patches/ds-bot/`, and `canonical` says `ok patches/ds-bot/ is what
generate writes (generate is a no-op)`.
## Scope
| file | +/− |
| --- | --- |
| `core/test/memory-search-edge.test.js` | +11/−6 |
| `patches/ds-bot/files/test/memory-search-edge.test.js` | +11/−6
(generated) |
Tier basis: `risk-tier: T1; basis:
core/test/memory-search-edge.test.js=T1,
patches/ds-bot/files/test/memory-search-edge.test.js=T1` (18 changed
lines in the 1 non-generated file; the generated mirror is not counted).
## Why no changelog fragment
Test hardening only: no user-visible behaviour changes, and the file the
fragment would describe (`recall` ordering) behaves exactly as it did
before. The repository asks for a fragment when a user-visible fix
ships; this ships a guard, not a fix.
Verified commit: 958d0bc52e8432b358259c9072f45dc22a0d04b2 (tree
ab74309); the pushed head below is that
tree rebuilt on live main.
## Checkpoint
<!-- flybot:checkpoint -->
```json
{
"pr": 2849,
"claim": "Dev-1",
"issue": 2846,
"summary": "The determinism guard in core/test/memory-search-edge.test.js compared the same searchIndex call with itself, so it could never fail. It now pins the tie-break by key, stability across a reordered index rebuild, and best-first ordering — the order recall returns. Two source mutations made the old assertion pass 5/5 and the new one fail 1 of 5; at this tip the file is 5/5 and verify is green.",
"files": [
"core/test/memory-search-edge.test.js",
"patches/ds-bot/files/test/memory-search-edge.test.js"
],
"checks": {
"ran": "TZ=UTC npm run verify",
"result": "green",
"at": "50b7324175b26863a1dc2f77d54d5822f32c575c"
},
"next": "none — test-only T1, ready for the T1 lane"
}
```
<!-- /flybot:checkpoint -->1 parent 429668e commit ff9c5db
2 files changed
Lines changed: 22 additions & 12 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
15 | 15 | | |
16 | 16 | | |
17 | 17 | | |
18 | | - | |
| 18 | + | |
19 | 19 | | |
20 | | - | |
21 | | - | |
22 | | - | |
23 | | - | |
24 | | - | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
25 | 30 | | |
26 | 31 | | |
27 | 32 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
15 | 15 | | |
16 | 16 | | |
17 | 17 | | |
18 | | - | |
| 18 | + | |
19 | 19 | | |
20 | | - | |
21 | | - | |
22 | | - | |
23 | | - | |
24 | | - | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
25 | 30 | | |
26 | 31 | | |
27 | 32 | | |
| |||
0 commit comments