Skip to content

fix(rivetkit): run onSleep after in-flight work and refuse dispatch once it starts - #5860

Open
eersnington wants to merge 1 commit into
fix/rivetkit-restarting-after-on-sleepfrom
stack/fix-rivetkit-run-onsleep-after-in-flight-work-kttnpoyx
Open

eersnington wants to merge 1 commit into
fix/rivetkit-restarting-after-on-sleepfrom
stack/fix-rivetkit-run-onsleep-after-in-flight-work-kttnpoyx

Conversation

@eersnington

@eersnington eersnington commented Oct 11, 2026 •

Copy link
Copy Markdown
Member

Run onSleep after in-flight work finishes, and refuse new work once onSleep starts. Before this change, an action running when sleep started could fail because TypeScript closed SQLite right after onSleep.

Before

sequenceDiagram
	participant E as Engine
	participant T as ActorTask (core)
	participant TS as TypeScript runtime
	participant A as In-flight action
	E->>T: Stop(Sleep)
	T->>TS: onSleep
	TS->>TS: close SQLite, clear vars
	A--xTS: c.db.execute() fails
	E->>T: new action (still accepted)
	T->>T: sync_alarm fails (SQLite already closed)
Loading

After

sequenceDiagram
	participant E as Engine
	participant T as ActorTask (core)
	participant TS as TypeScript runtime
	participant A as In-flight action
	E->>T: Stop(Sleep)
	T->>T: fire abort signal
	A->>TS: c.db.execute() completes
	Note over T: wait until in-flight work drains, or 3/4 of the grace period
	T->>TS: onSleep + disconnects
	E->>T: new action
	T-->>E: actor.restarting (503), client retries on the next generation
	T->>T: sync_alarm, then close SQLite
Loading

Changes

  • Order: ActorTask runs the sleep hooks when the shutdown counter (actions, keepAwake, waitUntil, registered tasks) and WebSocket callbacks reach zero, or at the hooks deadline. Raw onRequest and run are not waited for. Destroy runs its hooks at once, as before.

  • Reserve: SLEEP_HOOK_RESERVE_DIVISOR = 4 keeps the last quarter of the grace period for onSleep, disconnects, and the final save. With the 15 second default this is 3.75 seconds.

  • Refusal point: fix(rivetkit): refuse actions after onSleep with actor.restarting #5876 refuses dispatch with actor.restarting after the sleep hooks finish. This PR refuses it once the hooks start, because the hooks now run after in-flight work drains. Before the hooks run, dispatch is still accepted. DestroyGrace keeps actor.stopping.

    // rivetkit-core/src/actor/task.rs
    LifecycleState::SleepGrace if self.pending_hooks_deadline().is_some() => None,
  • SQLite close moves to core: On the drained path, cleanupNativeSleepRuntimeState no longer closes SQLite. Core runs sync_alarm and then closes it, so the alarm sync no longer fails on every sleep. If work did not drain by the deadline, TypeScript still closes SQLite early so late work cannot write after shutdown.

  • try_finish_grace requires the hooks to have run, so an idle actor cannot finalize without onSleep.

  • docs-internal/engine/sleep-sequence.md describes the new order.

Behavior change

An action that waits for something onSleep does, such as a flag that onSleep sets, now waits until the hooks deadline. Long actions should watch c.abortSignal, which still fires when sleep starts. No onSleep in the docs or examples uses this pattern (going to follow up on this)


This is part 5 of 6 in a stack:

@railway-app

railway-app Bot commented Oct 11, 2026

Copy link
Copy Markdown

This PR was not deployed automatically as @eersnington does not have access to the Railway project.

In order to get automatic PR deploys, please add @eersnington to your workspace on Railway.

@claude

claude Bot commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor

Review: run onSleep after in-flight work, refuse dispatch once it starts

The design is sound and the ordering is well tested. The paused-time Rust tests assert real ordering. I have a few concerns, mostly about the TypeScript cleanup change.

Potential issues

  1. JS database client may no longer be closed on the drained path (registry/native.ts, cleanupNativeSleepRuntimeState). The PR removes both closeNativeDatabaseClient and closeNativeSqlDatabase from the drained path. Core's ctx.sql().close_and_wait() only covers the native SQL handle. closeNativeDatabaseClient also calls entry.client.close() on the JS-side client (e.g. a drizzle wrapper or user-provided client), and clearNativeRuntimeState does not appear to do that. Please confirm a JS client with a close() hook is still disposed on a normal sleep. If not, keep closeNativeDatabaseClient in the drained path and drop only the native SQL close.
  2. Hooks deadline with a tiny grace period. pending_hooks_deadline is deadline - grace_period / 4. With a very small sleep_grace_period (including 0) hooks run immediately and in-flight work is not waited for. Probably acceptable, but untested.
  3. Behavior change for actions that wait on onSleep. These now stall until the hooks deadline (3.75s by default). A line in the sleep docs about watching c.abortSignal would help. Please also check the docs-sync table.

Smaller notes

  • try_finish_grace now calls try_emit_grace_hooks() as a side effect, which is surprising for its name. A short comment would help.
  • In the activity-wait select! (~L695), the new hooks arm is not followed by a try_finish_grace check, unlike the main loop arm. It looks fine since the loop re-evaluates, but please confirm there is no missed-wakeup window.
  • The driver test is event-ordered on slowStarted and asserts log order, which is robust.

Style and conventions

No CLAUDE.md violations found.

Overall this looks good once item 1 is confirmed.

@eersnington
eersnington force-pushed the stack/fix-rivetkit-run-onsleep-after-in-flight-work-kttnpoyx branch from 6701f1d to 5b6cfaf Compare October 11, 2026 20:26
@eersnington
eersnington force-pushed the stack/fix-rivetkit-run-onsleep-after-in-flight-work-kttnpoyx branch from 5b6cfaf to 69cdf48 Compare October 13, 2026 23:09
@eersnington
eersnington changed the base branch from main to fix/rivetkit-restarting-after-on-sleep October 13, 2026 23:09

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant