hip: enable GGML_CUDA_USE_CUB via hipCUB (fixes TOP_K abort for >1024 rows)

[davidcanar]

Problem

On HIP builds GGML_CUDA_USE_CUB is disabled unconditionally (common.cuh), so ggml_cuda_op_top_k always takes the no-CUB bitonic argsort fallback. That kernel launches one block of next_power_of_2(ncols) threads, which is only valid for ncols <= 1024 - the CUDA-side supports_op for TOP_K/ARGSORT even advertises ne[0] <= 1024 when CUB is off, but nothing enforces it at launch time.

GLM-5-Next / GLM-5.3-Flash (PR ggml-org#27754) runs ggml_top_k over n_kv / index_kpool lightning-indexer pools, so once a conversation grows past ~4k tokens of KV the selection exceeds 1024 rows, the block exceeds the device's 1024-thread limit, and the backend aborts:

ggml_cuda_compute_forward: TOP_K failed
ROCm error: invalid configuration argument

With the tensor hosted on a ggml-rpc-server (the natural way to run the 200 GB Q4 GGUF), this aborts the RPC server and takes the whole llama-server with it.

Fix

Enable GGML_CUDA_USE_CUB on HIP and source CUB from hipCUB, which provides every primitive these paths use:

• common.cuh: #include <hipcub/hipcub.hpp>, namespace cub = hipcub (hipCUB has no <cub/cub.cuh>), and a small mapping of cudaStreamCaptureStatus / cudaStreamIsCapturing onto the hip* equivalents used by the capture checks in argsort.cu / mean.cu.
• The CCCL >= 3.2 DeviceTopK::MaxPairs path in top-k.cu and the CCCL >= 3.1 strided-iterator path in argsort.cu remain off on HIP (hipCUB has no cuda::execution / cuda/iterator); HIP gets the offsets-array + radix-sort paths instead.
• argsort.cu: the multi-row non-capturing branch routes through DeviceSegmentedRadixSort on HIP because hipCUB has no DeviceSegmentedSort (same call shape already used for the stream-capturing case).
• cumsum.cu / mean.cu / sum.cu: include <cub/cub.cuh> only when not on HIP.

CUDA builds are untouched - everything new is inside #ifdef GGML_USE_HIP.

Verification

Ryzen AI Max+ 395 (Radeon 8060S, gfx1151) pair, built with the system hipcc (ROCm 7.1), runtime on ROCm 7.14, running GLM-5.3-Flash UD-Q4_K_XL distributed over RPC (--rpc, CUDA graphs enabled):

• before: any prompt growing KV past ~4k cells aborts the RPC server with TOP_K failed / invalid configuration argument
• after: 8.8k-token prompt processed end to end, zero TOP_K errors, CUDA graph capture and replay fine, prompt processing unchanged (~123 t/s)

Fixes the HIP half of the wide-top-k story for glm5next/upstream; happy to rebase onto a different branch if that is preferred.

[Cass67]

Worth noting this isn't only a GLM problem — I hit the same thing running qwen4exp (Qwen3.8-Flash-Next) on gfx1100. The sampler's top-k goes over the whole vocab row, ~151k wide, so it's past the 1024 limit on the first token of any request. No long context needed. It doesn't crash, supports_op declares the limit, so it just silently moves sampling to the CPU. Any HIP build with default sampling hits it as far as i can tell, whatever the model. Seeing it on b10715-mix-86bd2d3 (rocm-gfx110X) on 3x 7900 XT.
Also the two fixes aren't the same. ggml-org#27466 only relaxes TOP_K — ARGSORT still refuses anything over 1024 columns. hipCUB gets both, since they were behind the same switch. Glad to see that stuff I've been poking Claude to fix over the past days is out there and better :)