Add WriteLockedGraph::lock_new_segments (pometry-storage#427)

[fabianmurariu]

Part of Pometry/pometry-storage#423. Closes Pometry/pometry-storage#427.

Summary

A held WriteLockedGraph can now extend its locks to segments created after it was taken, instead of being dropped and re-taken. This is a prefactor for holding one WriteLockedGraph for a whole bulk load (Pometry/pometry-storage#424). Loading behaves the same as before.

 load_edges_from_df (per chunk)
   write_lock()
   node pass            # next_edge_id may create edge segments
-  drop(write_locked_graph)
-  write_lock()         # re-lock every segment
+  lock_new_segments()  # lock only segments not yet held, keep the rest
   edge pass

WriteLockedGraph::lock_new_segments(&mut self)
  NodeStorageInner::lock_new_segments(&mut nodes)   # for id in held..count: push(head_mut())
  EdgeStorageInner::lock_new_segments(&mut edges)

• WriteLockedNodePages::push / WriteLockedEdgePages::push assert that pages stay indexed by segment id.
• resize_segments_to_vid/eid = grow + lock_new_segments; they no longer release every page.
• The edge-metadata loader's drop and re-take is replaced the same way.

Evidence

• Before: with the new call removed from the edge loader, test_load_edges_creating_many_segments_in_one_chunk (200 new edges in one chunk, page length 2) panics with segment not found while grouping by eid (edges.rs:760).
  After: passes.
• test_lock_new_segments_covers_segments_created_while_locked: after grow on nodes and edges while locked, the held pages match the store's segment counts, each page's id matches its index, and a second call does nothing.
• test_io: 62 passed, 6 ignored. test_lib -- arrow_loader: 18 passed. cargo check --workspace --tests in pometry-storage is clean.

Merge Danger

Door: two-way

The new methods are additive; reverting brings back the drop and re-take.

Blast Radius: bulk loads

Only edge and edge-metadata bulk loads, and resize_segments_to_*, change. Between the node pass and the edge pass they now keep their segment locks instead of briefly releasing them, so readers can't take those segments at that point.

🤖 Generated with Claude Code

[github-actions]

⚠️ Performance Alert ⚠️

Possible performance regression was detected for benchmark 'Rust Benchmark'.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 2.

Benchmark suite	Current: 2cd733d	Previous: b81cffb	Ratio
lotr_graph/num_edges_temporal	317 ns/iter (± 1)	158 ns/iter (± 1)	2.01
lotr_graph_subgraph_10pc/num_nodes	13 ns/iter (± 1)	4 ns/iter (± 0)	3.25
lotr_graph_subgraph_10pc/num_nodes_cached	14 ns/iter (± 0)	5 ns/iter (± 0)	2.80

This comment was automatically generated by workflow using github-action-benchmark.