[lw-liuwang]

Fixes #220.

A transient connection failure can make Colab CLI lose the output and final result of work it has already submitted. Remote execution can continue independently, which is expected, but an agent then cannot reliably follow the existing task to completion.

Recover the original execution through a request-specific kernel journal. The kernel acknowledges recording before the CLI submits user code, then records output, errors, the reply, and idle status before delivery. The CLI retrieves missing messages through the Contents API, validates their request and sequence, and suppresses duplicate message IDs. It submits the original source once, preserving magics and top-level await.

Recovery refreshes rejected proxy credentials or routes with a bounded GET of assignments, matching the original endpoint. A proxy 404 alone no longer triggers pruning: live probes returned 404 for both an invalid token and a missing kernel. If the WebSocket worker stops, monitoring continues through HTTP. The next execution rebuilds the socket on the same kernel and Jupyter session, with distinct message IDs to avoid matching old replies.

Save the request ID and journal path before submission. After a timeout or CLI exit, colab resume -s NAME retrieves that execution without starting a kernel, submitting code, or allocating a replacement. It replays the transcript from the beginning, so output displayed by the previous process may repeat. For notebooks, it recovers the interrupted cell's transcript without executing later cells or rewriting the local notebook. New submissions cannot overwrite unresolved work. A resumed VM stays allocated; the command prints how to stop it.

An unknown result exits nonzero and preserves state; run also skips automatic teardown. Confirmed completion and script errors retain normal cleanup, subject to --keep. --timeout 0 disables the execution idle limit; unavailable records and missing completion still have bounded waits. Recovery requires accessible records and cannot restore a destroyed kernel's unfinished work, deleted records, or revoked access. Journaling adds a setup round trip and file writes. Confirmed records are removed; uncertain records remain for recovery. Preserved VMs consume resources until stopped or reclaimed by Colab.

Validation

Run the focused regression tests from the repository root:

uv run pytest tests/test_execution_recovery.py tests/test_execution_journal.py \
  tests/test_streaming.py tests/test_runtime.py tests/test_exec.py tests/test_run.py

For live CPU tests, use Bash or WSL with Colab credentials available there:

uv run colab --auth=oauth2 sessions
uv run bash integration/repro_execution_reconnect/test.sh

The script closes TCP connections after output markers, injects invalid proxy credentials/routes, stops the WebSocket worker, and kills a CLI process after work starts. It checks recovered output and results, and uses independent receipts to verify one execution on the original kernel. Injected credential failures test recovery from rejection; they do not reproduce natural token expiry.

Live CPU checks with the development and PyPI kernel clients covered output and completion during outages, exceptions, rich displays, HTTP fallback beyond the reconnect limit, and recovery in a new CLI process. They also checked startup refresh, subsequent execution after socket rebuilding, a recorded kernel-shutdown error without a replacement kernel, runtime preservation, and normal cleanup. Tests isolate session state and release their own assignments. Linux and Windows regression tests cover invalid records, uncertain sends, recovery bounds, persisted identity, and non-OK results.