Skip to content

fix: use strided offsets in run_single_once when batch_contiguous is disabled - #612

Merged
TianDi101 merged 1 commit into
ROCm:mainfrom
kudomcho:fix/io-strided-write-offset
Sep 9, 2026
Merged

TianDi101 merged 1 commit into
ROCm:mainfrom
kudomcho:fix/io-strided-write-offset

Conversation

@kudomcho

@kudomcho kudomcho commented Aug 27, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • run_single_once() always used contiguous offsets (buffer_size * i) regardless of batch_contiguous
  • When batch_contiguous=False, buffer is allocated with strided layout (buffer_size + 1) * i, causing each transfer to read from wrong positions (overlapping gap bytes)
  • Target receives corrupted data; validation fails with AssertionError at benchmark.py:705
  • Fix: use strided offsets in run_single_once() when batch_contiguous=False

Test plan

  • Ran MORI-IO internode write benchmark without --batch-contiguous — previously failed with AssertionError, now passes
  • Ran MORI-IO internode write benchmark with --batch-contiguous — still passes (no regression)
  • Tested on 2-node ionic RDMA setup (crsuse2-m2m-v2-017 / crsuse2-m2m-v2-024)

…disabled

run_single_once always used contiguous offsets (buffer_size * i) regardless
of the batch_contiguous flag. When batch_contiguous=False, the buffer is
allocated with strided layout ((buffer_size + 1) * i), so RDMA writes were
reading from wrong positions — each transfer's source overlapped gap bytes
from adjacent segments, producing corrupted data on the target. Validation
at _validate_rdma then correctly detected the mismatch (AssertionError at
benchmark.py:705).

Use strided offsets in run_single_once to match the buffer layout and
validation expectations when batch_contiguous is disabled.

Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com>
@kudomcho

kudomcho commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

@TianDi101 Hi, Please let me know if there are any requests for this PR for review.

@TianDi101
TianDi101 merged commit 07bdace into ROCm:main Sep 9, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants