Skip to content

fix(metal): coalesce adjacent --layers model map spans - #957

Open
paride-cmd wants to merge 1 commit into
antirez:mainfrom
paride-cmd:fix/845-layers-metal-map
Open

fix(metal): coalesce adjacent --layers model map spans#957
paride-cmd wants to merge 1 commit into
antirez:mainfrom
paride-cmd:fix/845-layers-metal-map

Conversation

@paride-cmd

Copy link
Copy Markdown

Summary

  • `ds4_gpu_set_model_map_spans` now coalesces abutting/overlapping tensor spans (with page/64KiB alignment slop) before creating Metal no-copy buffers.
  • A single coalesced range uses the normal overlapping `set_model_map_range` path; remaining ranges use overlapping views without the 128 GiB default span-map cap.
  • This targets the `--layers` / pipeline-parallel shard case from --layers maps the shard as N disjoint Metal buffers even when the range spans the whole file (~77x decode slowdown) #845 where the "restricted" map still covered the whole shard file but as ~156 disjoint buffers (~77x decode slowdown).

Fixes #845

Test plan

  • `make ds4_metal.o` on Apple Silicon
  • On a Metal Flash shard with `--layers`: log should show overlapping/coalesced mapping instead of `N disjoint shared buffers across N tensor spans`
  • Spot-check decode t/s recovers toward the no-`--layers` single-process baseline on the same host
  • Optional: SSD-streaming / TP span maps still start (coalesce only merges abutting ranges)

AI usage disclosure: YES — assisted by Cursor agent; change reviewed against #845 measurements and existing Metal map comments.

Made with Cursor

--layers built one Metal no-copy buffer per isolated tensor span, so a
per-host shard that already covered the whole file mapped as ~156
disjoint buffers and decode fell by ~77x. Merge abutting spans (with
page/64KiB gap slop) and map coalesced ranges with the overlapping-view
path used without --layers.

Fixes antirez#845
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

--layers maps the shard as N disjoint Metal buffers even when the range spans the whole file (~77x decode slowdown)

1 participant