Skip to content

fix: guard ProxyPostAtomicFetch on emulated path - #24

Closed
itej89 wants to merge 1 commit into
feat/ep-rdma-sharingfrom
feat/ep-rdma-sharing-fix-r2-3-fetch-guard
Closed

fix: guard ProxyPostAtomicFetch on emulated path#24
itej89 wants to merge 1 commit into
feat/ep-rdma-sharingfrom
feat/ep-rdma-sharing-fix-r2-3-fetch-guard

Conversation

@itej89

@itej89 itej89 commented Aug 28, 2026

Copy link
Copy Markdown
Owner

Summary

Addresses PR ROCm#558 round-2 review comment R2-3.

ProxyPostAtomicFetch returns ring->cmds[slot].result which is 0 on the SEND_WITH_IMM emulated path (ionic/bnxt) — silent wrong value. Nobody calls it today, but a future caller would get corrupted data.

Fix: ProxyPostAtomicFetch sets flags=2 ("fetch result required"). CPU DrainCq checks this flag:

  • Native atomics (CX7): writes result from ibuf → correct value
  • Emulated (ionic/bnxt) + fetch required: logs MORI_APP_ERROR and returns PROXY_ERROR instead of silent 0

Native atomics remain Mellanox-only — tested IBV_WR_ATOMIC_FETCH_AND_ADD on DO AINIC ionic and confirmed failure: CQE error status=12 (transport retry counter exceeded).

Test plan

  • EP benchmark on DO AINIC (SEND_WITH_IMM path): Dispatch Pass, Combine Pass
  • Dispatch avg 30.35 GB/s, Combine avg 58.15 GB/s (consistent)
  • Confirmed ionic native atomics fail: CQE transport retry exceeded

🤖 Generated with Claude Code

@itej89
itej89 force-pushed the feat/ep-rdma-sharing-fix-r2-3-fetch-guard branch from c6ca1c2 to be1dcc3 Compare August 28, 2026 22:11
Add ProxyCmdFlags enum (PROXY_FLAGS_DEFAULT, PROXY_FLAGS_FETCH_REQUIRED).
ProxyPostAtomicFetch sets PROXY_FLAGS_FETCH_REQUIRED. CPU DrainCq:
- Any non-default flag on emulated path → MORI_APP_ERROR + PROXY_ERROR
- PROXY_FLAGS_FETCH_REQUIRED on native path → writes ibuf result

Native atomics remain Mellanox-only. Confirmed ionic native atomics
fail (CQE transport retry exceeded on DO AINIC).

Addresses PR ROCm#558 round-2 review comment R2-3.

Co-Authored-By: Claude <noreply@anthropic.com>
@itej89
itej89 force-pushed the feat/ep-rdma-sharing-fix-r2-3-fetch-guard branch from be1dcc3 to 737e866 Compare August 28, 2026 22:17
@itej89 itej89 closed this Aug 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant