cuda : accelerate conv2d with implicit GEMM

[leejet]

Overview

Accelerate the existing CUDA CONV_2D operator for F16 weights and F32 activations with a compact implicit-GEMM implementation. It uses a fixed 64x64x64 tile and the existing mma.cuh interface, with F32 accumulation, split-K for small spatial maps, and a cuBLAS path for eligible 1x1 convolutions.

The existing direct kernel remains the fallback, including F32 weights and small convolutions where conversion and launch overhead can outweigh the benefit. Three regression cases cover channel tails, multiple batches, stride/dilation, and the 1x1 path. The change is confined to conv2d.cu and the existing operator test file.

Additional information

Existing work such as #15805 already explores implicit-GEMM convolution. This proposal focuses on keeping the implementation compact and straightforward to review: reuse the existing MMA primitives, use one tile configuration, keep the indexing and dispatch local, and retain the current fallback. The goal is to reduce review and maintenance overhead while providing useful acceleration.

Selected operator timings on an RTX 4090 with CUDA 12.4, Release, and CUDA graphs enabled, compared with 1af554f. Input dimensions are [W,H,IC,N]; kernel dimensions are [KW,KH,IC,OC]. These are microbenchmarks, not end-to-end model results.

F16 convolution	Before (us)	After (us)	Speedup
[58,58,32,1], [3,3,32,64]	79.74	13.81	5.77x
[58,58,32,8], [3,3,32,64]	500.27	41.84	11.96x
[16,16,128,8], [3,3,128,512]	986.87	50.79	19.43x
[19,19,4,16], [2,2,4,4]	4.38	4.42	0.99x

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: YES. Codex assisted with porting the implementation from stable-diffusion.cpp's ggml fork, adapting cuBLAS handle usage.

[ggml-gh-bot]

Hi @leejet, thanks for your contribution!

Per our contribution guidelines, the automated PR checker found the following issue(s) that need your attention:

• Multiple open PRs from a new contributor: We limit new contributors (those without a previously merged PR) to 1 open PR at a time. You currently have 6 open PRs.

---

Please note that maintainers reserve the right to make final decisions on PRs. If you believe there is a mistake, please comment below.

[leejet]

For an example of the actual speedup achieved on real models in sd.cpp, please refer to leejet/stable-diffusion.cpp#1993.

[ggerganov]

I think it is OK to merge since this is battle-tested downstream, but let's give some time in case someone from @ggml-org/ggml-cuda can take a look too.

[am17an]

I tested this out locally and it works. I know there were several attempts at getting this right but this one is particularly well written, so thanks!

[leejet]

Thanks for reviewing and merging the PR! Really appreciate it.