[FEA]: Does cutile-python have plans to integrate NCCL as the communication backend?

[wangfakang]

Is this a new feature, an improvement, or a change to existing functionality?

New Feature

How would you describe the priority of this feature request?

High

Please provide a clear description of problem this feature solves

Integrating NCCL with CuTile-Python enables compute-communication operator fusion.

Feature Description

Are there plans to integrate NCCL as the communication backend? Thx.

Describe your ideal solution

def allgather_gemm():
nccl_device.allgather()
gemm()

Describe any alternatives you have considered

No response

Additional context

No response

Contributing Guidelines

• I agree to follow cuTile Python's contributing guidelines
• I have searched the open feature requests and have found no duplicates for this feature request

[haijieg]

@wangfakang can you check if this example meets your need?

[wangfakang]

@wangfakang can you check if this example meets your need?

@haijieg Yeah, but it seems to only support intra-node communication scenarios. Is there a plan to integrate nccl device API IR to support inter-node communication and computing fusion scenarios. Thanks!

[haijieg]

Yes, NCCL support it's on the roadmap.