Skip to content

Commit 412ad98

Browse files
committed
fix(host): a stopped state journal asks once more on the next save (#2154)
One failed open — at start-up, or a write that failed later — used to retire the journal for the rest of the run: every later save returned silently and the session kept no record at all, even after the folder or file it refused was repaired. The next save now runs the start-up open() once more. A journal that can record again resumes and appends the change that save carried; one that still refuses stays stopped, keeps the save succeeding, and is not warned about again — a refusal is announced once, as before. The retry is per save and never loops, and a journal file removed to repair the folder starts again with a base line instead of continuing the seq of the file that is gone. Tests: core/test/state-journal.test.js (#2154) Docs: docs/subsystems/core-host.md Changelog: changelog.d/2154-journal-reopen.fixed.md
1 parent c4dc6e4 commit 412ad98

6 files changed

Lines changed: 220 additions & 3 deletions

File tree

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
The state journal no longer stays stopped for the rest of the run after one failed open or write. The next save asks once more whether it can record again: a folder or file repaired meanwhile resumes recording, appending the change that save carried, while a journal that still refuses stays stopped and that save still lands. The asking is silent, so a refusal is warned once rather than once per save ([#2154](https://github.com/ThinkFlowLab/FlyBot/issues/2154)).

‎core/src/host/journal.js‎

Lines changed: 25 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -272,9 +272,33 @@ export function openJournal({ path, warn, now = Date.now, maxBytes = JOURNAL_MAX
272272
}
273273
const opened = open().catch(stop)
274274

275+
// One failed open — at start-up, or from a write that failed later — used to retire the
276+
// journal for the rest of the run, so a single bad moment lost every later change of the
277+
// session. The next save asks once more whether it can record again; a writer that still
278+
// refuses stays stopped and only that save's write is skipped, and the asking is silent so a
279+
// refusal is announced once, as `stop()` always did.
280+
const reopen = async () => {
281+
// A file that was removed to repair the folder must not continue the seq of the file that is
282+
// gone: this restarts from the same empty state a fresh writer has, so the next line is a base.
283+
lastText = undefined
284+
seq = 0
285+
bytes = 0
286+
oldest = undefined
287+
try {
288+
await open()
289+
stopped = false
290+
because = undefined
291+
return true
292+
} catch (error) {
293+
because = `${path} is not recording: ${error}`
294+
return false
295+
}
296+
}
297+
275298
const write = async (snapshot, actor) => {
276299
await opened
277-
if (stopped || snapshot === lastText) return
300+
if (stopped && !(await reopen())) return
301+
if (snapshot === lastText) return
278302
// #2074: the same question `open()` asks, asked again when the write runs. A folder that was
279303
// real at start-up can be a link by now, and appending through it would put this snapshot in
280304
// a folder the team never named.

‎core/test/state-journal.test.js‎

Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -584,3 +584,87 @@ test('a rotation still replaces a plain journal file of this host\'s own (#2146
584584
if (posix) assert.equal(statSync(path).mode & 0o777, 0o600)
585585
rmSync(home, { recursive: true, force: true })
586586
})
587+
588+
// A journal that stopped — a failed open at start-up, or a write that failed later — used to
589+
// stay stopped for the rest of the run, so one bad moment retired the whole session's record. The
590+
// next save asks once more whether it can record again: a journal that can now record appends the
591+
// change that save was about to lose, and one that still refuses stays stopped without failing it.
592+
test('a stopped journal opens once more on the next save and records again (#2154)', async () => {
593+
const home = tempHome()
594+
const outside = mkdtempSync(join(tmpdir(), 'ds-bot-journal-outside-'))
595+
const other = join(outside, 'other-name.jsonl')
596+
mkdirSync(join(home, 'journal'))
597+
const path = join(home, 'journal', 'state.jsonl')
598+
const warnings = []
599+
const journal = openJournal({ path, warn: (...args) => warnings.push(args.join(' ')) })
600+
await journal.record(stateText({ version: 2, revision: 1 }))
601+
assert.equal(parseJournal(readFileSync(path, 'utf8')).length, 1)
602+
// A second name on the journal file makes the append refuse it: recording stops, file intact.
603+
linkSync(path, other)
604+
await journal.record(stateText({ version: 2, revision: 2 }))
605+
assert.equal(journal.status().recording, false)
606+
assert.equal(parseJournal(readFileSync(path, 'utf8')).length, 1, 'the refused save recorded nothing')
607+
// The user removes the second name; the next save asks the journal once more.
608+
rmSync(other)
609+
await journal.record(stateText({ version: 2, revision: 3 }))
610+
assert.deepEqual(journal.status(), { recording: true })
611+
const text = readFileSync(path, 'utf8')
612+
assert.equal(parseJournal(text).length, 2, 'the recovering save appended one entry')
613+
assert.equal(stateText(replayTo(text)), stateText({ version: 2, revision: 3 }))
614+
assert.equal(warnings.length, 1, 'the refusal was announced once; the retry does not announce it again')
615+
rmSync(home, { recursive: true, force: true })
616+
rmSync(outside, { recursive: true, force: true })
617+
})
618+
619+
test('a journal whose open keeps failing retries once per save and never blocks the save (#2154)', async () => {
620+
const home = tempHome()
621+
const outside = mkdtempSync(join(tmpdir(), 'ds-bot-journal-outside-'))
622+
const target = join(outside, 'someone-else.txt')
623+
writeFileSync(target, 'PRIVATE LINE\n')
624+
mkdirSync(join(home, 'journal'))
625+
const path = join(home, 'journal', 'state.jsonl')
626+
symlinkSync(target, path)
627+
const warnings = []
628+
const journal = openJournal({ path, warn: (...args) => warnings.push(args.join(' ')) })
629+
// If a save retried until it worked, a path that keeps refusing would never come back.
630+
const saveWithin = snapshot => Promise.race([
631+
journal.record(snapshot).then(() => 'returned'),
632+
new Promise(resolve => setTimeout(() => resolve('still retrying'), 2000)),
633+
])
634+
for (const revision of [1, 2, 3]) {
635+
assert.equal(await saveWithin(stateText({ version: 2, revision })), 'returned', `save ${revision} returns instead of retrying in a loop`)
636+
}
637+
assert.equal(journal.status().recording, false, 'a journal that still refuses stays stopped')
638+
assert.equal(readFileSync(target, 'utf8'), 'PRIVATE LINE\n', 'nothing is written through the link')
639+
// The link goes away; the next save asks once more and records, so the retry is per save, not once.
640+
rmSync(path)
641+
assert.equal(await saveWithin(stateText({ version: 2, revision: 4 })), 'returned')
642+
assert.deepEqual(journal.status(), { recording: true })
643+
const text = readFileSync(path, 'utf8')
644+
assert.equal(parseJournal(text).length, 1)
645+
assert.equal(stateText(replayTo(text)), stateText({ version: 2, revision: 4 }))
646+
assert.equal(warnings.length, 1, 'the start-up refusal was announced once; a retry does not repeat it')
647+
rmSync(home, { recursive: true, force: true })
648+
rmSync(outside, { recursive: true, force: true })
649+
})
650+
651+
test('a journal reopened after its file was removed starts with a base (#2154)', async () => {
652+
const home = tempHome()
653+
const outside = mkdtempSync(join(tmpdir(), 'ds-bot-journal-outside-'))
654+
const other = join(outside, 'other-name.jsonl')
655+
mkdirSync(join(home, 'journal'))
656+
const path = join(home, 'journal', 'state.jsonl')
657+
const journal = openJournal({ path, warn: () => {} })
658+
await journal.record(stateText({ version: 2, revision: 1 }))
659+
linkSync(path, other)
660+
await journal.record(stateText({ version: 2, revision: 2 }))
661+
assert.equal(journal.status().recording, false)
662+
// The user repairs the folder by deleting the journal file itself; the next save starts a new one.
663+
rmSync(path)
664+
await journal.record(stateText({ version: 2, revision: 3 }))
665+
const text = readFileSync(path, 'utf8')
666+
assert.equal(parseJournal(text)[0].op, 'base', 'a journal that starts over begins with a base')
667+
assert.equal(stateText(replayTo(text)), stateText({ version: 2, revision: 3 }))
668+
rmSync(home, { recursive: true, force: true })
669+
rmSync(outside, { recursive: true, force: true })
670+
})

‎docs/subsystems/core-host.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -557,7 +557,7 @@ A local, replayable record of every change to the saved team, for multi-Bot bugs
557557
- **On.** `stateJournal: true` in the `bot` config. Off (the default) nothing is opened or written, so every other test runs as before.
558558
- **What.** `src/host/journal.js`. After each successful `state.json` write, `store.js` hands the exact snapshot text to the journal, which appends the difference from the last recorded snapshot to `<home>/journal/state.jsonl` (file mode 0600, folder 0700). Lines are `{ seq, t, op, key, prev, next, botId?, actor }`; `op` is `base` (the whole team: the first line, and any change only a key reorder explains), `set`, `delete`, or `append` (`prevLength`, `drop`, the new items: a group log or other list that grew or slid). Paths go at most three keys deep (`bots.<id>.<field>`); `botId` is set on a `bots.<id>` path; `actor` is `host` for a save and `retention` for a folded base (per-call attribution is not recorded). Only the host that holds the state lock writes, so a read-only window records nothing.
559559
- **Retention.** When a write would pass 20 MB, or the oldest change is older than 7 days, the oldest lines are dropped down to 90% of each cap and folded into the base line (atomic rewrite), so the kept lines still replay. A team that alone exceeds the cap stops recording.
560-
- **Fail closed.** Replay (`replayTo(text, seq)`) refuses the whole journal on an unreadable or incomplete line, an unknown op, a gap in `seq`, a first line that is not a base, or a `prev` that does not match what the earlier lines built; a `seq` outside the file is refused, not approximated. On start the writer replays the existing file the same way; a journal that does not replay, a path that is not a plain file, or a failed write stops recording for that run with a warning and leaves the file as it is. Saving the team never depends on the journal. The folder above the journal must be a real directory too (#2056): a link, a plain file or a check that cannot run refuses before the `mkdir`/`chmod` that would follow it, so neither the permissions nor a write reach the folder a link names; a folder that does not exist yet is still this writer's to create, as before. Since [#2074](https://github.com/ThinkFlowLab/FlyBot/issues/2074) the same question is asked again when the write runs: a journal folder swapped for a link is refused with one warning and recording stops, and a `state.jsonl` swapped for a link is never written through. A hard link reads as a plain file to `lstat` while its bytes are also a name outside this home, so the start-up read, the append and — since [#2146](https://github.com/ThinkFlowLab/FlyBot/issues/2146) — the rotation also require the file to carry only this one name (`nlink === 1`): a shared inode is refused exactly like a link — one warning, nothing read, chmod-ed or appended — and a journal with one name records as before. The append path refuses the write, and so does the rotation, before it reads: a link or a second name there stops recording with one warning, so another file's lines are never read as the team's history and the `rename` never drops the other name. A writer that stopped says so off the host too: `rt.journal.status()` answers `{ recording, because }`, and the diagnostics report adds one line — `state journal: recording` or `state journal: stopped (<reason>)` — while a host without the journal adds none ([#2134](https://github.com/ThinkFlowLab/FlyBot/issues/2134)).
560+
- **Fail closed.** Replay (`replayTo(text, seq)`) refuses the whole journal on an unreadable or incomplete line, an unknown op, a gap in `seq`, a first line that is not a base, or a `prev` that does not match what the earlier lines built; a `seq` outside the file is refused, not approximated. On start the writer replays the existing file the same way; a journal that does not replay, a path that is not a plain file, or a failed write stops recording with a warning and leaves the file as it is. A writer that stopped is asked once more on the next save: it runs the start-up `open()` again, so a folder or file repaired meanwhile resumes recording — appending the change that save carried, or starting a new file with a base when the journal was removed — while a journal that still refuses stays stopped and that save still lands; the asking is silent, so a refused start-up or write is warned once, not once per save ([#2154](https://github.com/ThinkFlowLab/FlyBot/issues/2154)). Saving the team never depends on the journal. The folder above the journal must be a real directory too (#2056): a link, a plain file or a check that cannot run refuses before the `mkdir`/`chmod` that would follow it, so neither the permissions nor a write reach the folder a link names; a folder that does not exist yet is still this writer's to create, as before. Since [#2074](https://github.com/ThinkFlowLab/FlyBot/issues/2074) the same question is asked again when the write runs: a journal folder swapped for a link is refused with one warning and recording stops, and a `state.jsonl` swapped for a link is never written through. A hard link reads as a plain file to `lstat` while its bytes are also a name outside this home, so the start-up read, the append and — since [#2146](https://github.com/ThinkFlowLab/FlyBot/issues/2146) — the rotation also require the file to carry only this one name (`nlink === 1`): a shared inode is refused exactly like a link — one warning, nothing read, chmod-ed or appended — and a journal with one name records as before. The append path refuses the write, and so does the rotation, before it reads: a link or a second name there stops recording with one warning, so another file's lines are never read as the team's history and the `rename` never drops the other name. A writer that stopped says so off the host too: `rt.journal.status()` answers `{ recording, because }`, and the diagnostics report adds one line — `state journal: recording` or `state journal: stopped (<reason>)` — while a host without the journal adds none ([#2134](https://github.com/ThinkFlowLab/FlyBot/issues/2134)).
561561
- **Replay.** `core/scripts/replay.mjs` (dev-only, not in the package, see [install](install.md)) prints or writes (`--out`, new file only, 0600) the team after a given `--to <seq>`; it makes no network or process calls. Test: `core/test/state-journal.test.js` (a 3-Bot round replays byte-identical at every step).
562562

563563
### Standing tool approvals (#540)

‎patches/ds-bot/files/src/host/journal.js‎

Lines changed: 25 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -272,9 +272,33 @@ export function openJournal({ path, warn, now = Date.now, maxBytes = JOURNAL_MAX
272272
}
273273
const opened = open().catch(stop)
274274

275+
// One failed open — at start-up, or from a write that failed later — used to retire the
276+
// journal for the rest of the run, so a single bad moment lost every later change of the
277+
// session. The next save asks once more whether it can record again; a writer that still
278+
// refuses stays stopped and only that save's write is skipped, and the asking is silent so a
279+
// refusal is announced once, as `stop()` always did.
280+
const reopen = async () => {
281+
// A file that was removed to repair the folder must not continue the seq of the file that is
282+
// gone: this restarts from the same empty state a fresh writer has, so the next line is a base.
283+
lastText = undefined
284+
seq = 0
285+
bytes = 0
286+
oldest = undefined
287+
try {
288+
await open()
289+
stopped = false
290+
because = undefined
291+
return true
292+
} catch (error) {
293+
because = `${path} is not recording: ${error}`
294+
return false
295+
}
296+
}
297+
275298
const write = async (snapshot, actor) => {
276299
await opened
277-
if (stopped || snapshot === lastText) return
300+
if (stopped && !(await reopen())) return
301+
if (snapshot === lastText) return
278302
// #2074: the same question `open()` asks, asked again when the write runs. A folder that was
279303
// real at start-up can be a link by now, and appending through it would put this snapshot in
280304
// a folder the team never named.

‎patches/ds-bot/files/test/state-journal.test.js‎

Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -584,3 +584,87 @@ test('a rotation still replaces a plain journal file of this host\'s own (#2146
584584
if (posix) assert.equal(statSync(path).mode & 0o777, 0o600)
585585
rmSync(home, { recursive: true, force: true })
586586
})
587+
588+
// A journal that stopped — a failed open at start-up, or a write that failed later — used to
589+
// stay stopped for the rest of the run, so one bad moment retired the whole session's record. The
590+
// next save asks once more whether it can record again: a journal that can now record appends the
591+
// change that save was about to lose, and one that still refuses stays stopped without failing it.
592+
test('a stopped journal opens once more on the next save and records again (#2154)', async () => {
593+
const home = tempHome()
594+
const outside = mkdtempSync(join(tmpdir(), 'ds-bot-journal-outside-'))
595+
const other = join(outside, 'other-name.jsonl')
596+
mkdirSync(join(home, 'journal'))
597+
const path = join(home, 'journal', 'state.jsonl')
598+
const warnings = []
599+
const journal = openJournal({ path, warn: (...args) => warnings.push(args.join(' ')) })
600+
await journal.record(stateText({ version: 2, revision: 1 }))
601+
assert.equal(parseJournal(readFileSync(path, 'utf8')).length, 1)
602+
// A second name on the journal file makes the append refuse it: recording stops, file intact.
603+
linkSync(path, other)
604+
await journal.record(stateText({ version: 2, revision: 2 }))
605+
assert.equal(journal.status().recording, false)
606+
assert.equal(parseJournal(readFileSync(path, 'utf8')).length, 1, 'the refused save recorded nothing')
607+
// The user removes the second name; the next save asks the journal once more.
608+
rmSync(other)
609+
await journal.record(stateText({ version: 2, revision: 3 }))
610+
assert.deepEqual(journal.status(), { recording: true })
611+
const text = readFileSync(path, 'utf8')
612+
assert.equal(parseJournal(text).length, 2, 'the recovering save appended one entry')
613+
assert.equal(stateText(replayTo(text)), stateText({ version: 2, revision: 3 }))
614+
assert.equal(warnings.length, 1, 'the refusal was announced once; the retry does not announce it again')
615+
rmSync(home, { recursive: true, force: true })
616+
rmSync(outside, { recursive: true, force: true })
617+
})
618+
619+
test('a journal whose open keeps failing retries once per save and never blocks the save (#2154)', async () => {
620+
const home = tempHome()
621+
const outside = mkdtempSync(join(tmpdir(), 'ds-bot-journal-outside-'))
622+
const target = join(outside, 'someone-else.txt')
623+
writeFileSync(target, 'PRIVATE LINE\n')
624+
mkdirSync(join(home, 'journal'))
625+
const path = join(home, 'journal', 'state.jsonl')
626+
symlinkSync(target, path)
627+
const warnings = []
628+
const journal = openJournal({ path, warn: (...args) => warnings.push(args.join(' ')) })
629+
// If a save retried until it worked, a path that keeps refusing would never come back.
630+
const saveWithin = snapshot => Promise.race([
631+
journal.record(snapshot).then(() => 'returned'),
632+
new Promise(resolve => setTimeout(() => resolve('still retrying'), 2000)),
633+
])
634+
for (const revision of [1, 2, 3]) {
635+
assert.equal(await saveWithin(stateText({ version: 2, revision })), 'returned', `save ${revision} returns instead of retrying in a loop`)
636+
}
637+
assert.equal(journal.status().recording, false, 'a journal that still refuses stays stopped')
638+
assert.equal(readFileSync(target, 'utf8'), 'PRIVATE LINE\n', 'nothing is written through the link')
639+
// The link goes away; the next save asks once more and records, so the retry is per save, not once.
640+
rmSync(path)
641+
assert.equal(await saveWithin(stateText({ version: 2, revision: 4 })), 'returned')
642+
assert.deepEqual(journal.status(), { recording: true })
643+
const text = readFileSync(path, 'utf8')
644+
assert.equal(parseJournal(text).length, 1)
645+
assert.equal(stateText(replayTo(text)), stateText({ version: 2, revision: 4 }))
646+
assert.equal(warnings.length, 1, 'the start-up refusal was announced once; a retry does not repeat it')
647+
rmSync(home, { recursive: true, force: true })
648+
rmSync(outside, { recursive: true, force: true })
649+
})
650+
651+
test('a journal reopened after its file was removed starts with a base (#2154)', async () => {
652+
const home = tempHome()
653+
const outside = mkdtempSync(join(tmpdir(), 'ds-bot-journal-outside-'))
654+
const other = join(outside, 'other-name.jsonl')
655+
mkdirSync(join(home, 'journal'))
656+
const path = join(home, 'journal', 'state.jsonl')
657+
const journal = openJournal({ path, warn: () => {} })
658+
await journal.record(stateText({ version: 2, revision: 1 }))
659+
linkSync(path, other)
660+
await journal.record(stateText({ version: 2, revision: 2 }))
661+
assert.equal(journal.status().recording, false)
662+
// The user repairs the folder by deleting the journal file itself; the next save starts a new one.
663+
rmSync(path)
664+
await journal.record(stateText({ version: 2, revision: 3 }))
665+
const text = readFileSync(path, 'utf8')
666+
assert.equal(parseJournal(text)[0].op, 'base', 'a journal that starts over begins with a base')
667+
assert.equal(stateText(replayTo(text)), stateText({ version: 2, revision: 3 }))
668+
rmSync(home, { recursive: true, force: true })
669+
rmSync(outside, { recursive: true, force: true })
670+
})

0 commit comments

Comments
 (0)