Hold one WriteLockedGraph for the whole bulk load (pometry-storage#424)

[fabianmurariu]

Part of Pometry/pometry-storage#423. Closes Pometry/pometry-storage#424.

Summary

Each bulk load now takes one WriteLockedGraph before its chunk loop and holds it until the load finishes. This is the scope the bulk loader lock (#425) will cover. Segments created by node/edge resolution are picked up with lock_new_segments.

 load_edges_from_df / load_nodes_from_df / load_node_props_from_df / load_edges_metadata_from_df
+  write_lock()                 # once, held for the whole load
   for chunk
     resolve properties, layers, nodes
-    write_lock()               # every chunk
+    lock_new_segments()        # segments created by node resolution
     node pass
     lock_new_segments()        # edge segments created by the node pass
     edge pass

Node resolution read the stored type of nodes that already exist through the graph's read path. That takes a segment read lock, which deadlocks against the write lock we now hold. It now reads through the held pages:

 resolve_node_vids_and_types_with_cache
   bulk_load_resolve_node(gid)
     Existing(vid) =>
-      graph.node_metadata(vid, NODE_TYPE_PROP_ID)
+      locked_nodes.node_metadata(vid, STATIC_GRAPH_LAYER_ID, NODE_TYPE_PROP_ID)

An existing vid always lives in a segment that is already held. The other calls the loaders make between chunks (property, layer and node-type resolution, validate, bulk_load_resolve_node, reserve_free_pos, internalise_node, event ids) only touch Meta, the GID resolver, atomics or the segment vector, never segment locks.

Evidence

• Before: with the old graph-based lookup, test_reload_existing_typed_nodes and test_load_typed_nodes_after_edges hit the 60s guard with bulk load did not finish (deadlock?).
  After: both pass, along with test_reload_existing_node_with_conflicting_type_fails, which checks that the conflicting-type error is still raised.
• test_io: 65 passed, 6 ignored. test_lib: 113 passed, 3 ignored. cargo check --workspace --tests in pometry-storage, including the disk storage members, is clean.

Merge Danger

Door: two-way

Blast Radius: bulk loads and readers

Readers are now blocked on existing segments for the whole bulk load rather than one chunk at a time. Regular updates that touch an existing segment wait for the whole load too, until #425 makes that explicit.

🤖 Generated with Claude Code

[github-actions]

⚠️ Performance Alert ⚠️

Possible performance regression was detected for benchmark 'Rust Benchmark'.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 2.

Benchmark suite	Current: 048c334	Previous: b81cffb	Ratio
lotr_graph_subgraph_10pc/num_nodes	11 ns/iter (± 0)	4 ns/iter (± 0)	2.75

This comment was automatically generated by workflow using github-action-benchmark.