JP relinking conversation directory

[rxgrant]

fatal: Unable to read current working directory: No such file or directory

[JeanMertz]

Do you have a trace log or a bit more context on what command you ran and when this exactly happened?

[rxgrant]

Context here is that i was in the conversation directory in another tab, and ran a JP command in the session tab, and then couldn't continue the shell in the converation directory.

[JeanMertz]

Was this in two separate projects, or in a single jp workspace, in the same project?

I have a PR in-flight (#743) that will make the storage set-up more robust, and fault-tolerant, but I need more information to know whether that PR solves this problem, or not.

[rxgrant]

I don't recall if the unlinked shell was in the XDG root conversations or in the git workspace conversations, but I was in conversations/ and had a separate active tab and JP session for that JP workspace, and after every JP command conversations/ the shell cwd would be invalidated.

[JeanMertz]

That implies that the entire conversations dir got re-created somehow. I know that logic existed in a (very) old JP version, but I've refactored that since. Let me check what path could still trigger such a case.

Ah yes, I remember now after reading the code again. This is a purposefully atomic directory swap on individual conversation directories. This is documented here:

jp/crates/jp_storage/src/lib.rs

Lines 188 to 215 in b29d37c

	/// Persist a single conversation's metadata and events to disk.
	///
	/// Uses a staging directory and atomic directory swap for write safety:
	///
	/// 1. Write all files into a `.staging-{name}` directory.
	/// 2. Copy non-managed files from the existing conversation directory (e.g.
	/// `QUERY_MESSAGE.md`) into the staging directory.
	/// 3. Rename the existing directory to `.old-{name}`.
	/// 4. Rename the staging directory to the final name.
	/// 5. Remove the `.old-{name}` backup.
	///
	/// If any write in step 1 fails, the staging directory is removed and the
	/// existing conversation is untouched.
	/// The rename in step 4 is a single syscall, so readers never see a
	/// partially-written directory.
	///
	/// Recovery: if the process crashes between steps 3 and 4, the next
	/// startup's validation pass detects the `.old-*` / `.staging-*` pair and
	/// completes or rolls back the swap.
	///
	/// The conversation is stored as three managed files:
	///
	/// - `metadata.json` — lightweight conversation metadata.
	/// - `base_config.json` — the initial `PartialAppConfig` snapshot, written
	/// once at creation time.
	/// Subsequent persists preserve the existing file.
	/// - `events.json` — the event stream (config deltas + conversation
	/// events).

I specifically added this, because I had cases in the past where partial writes when running out of disk space caused conversation corruption. So the question really is one of: do we want to keep the atomic (single syscall) conversation mutation, or do we want to avoid invalid inodes in your shell when you are inside a conversation and some other tab persists new data in that conversation's state?

I prefer the atomic writes (the current design), but I'm only judging it based on my usage patterns and needs, if you have a usage pattern that requires your shell to be inside the conversation itself, then it's worth reconsidering or making the atomicity optional.