Skip to content

[Engine] sm_52 build: leave the GP100-only weight-stationary down kernels empty below sm_53 #4769

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
roshanraj wants to merge 1 commit into
base: main
Choose a base branch
from
+8 −0
Draft

[Engine] sm_52 build: leave the GP100-only weight-stationary down kernels empty below sm_53 #4769

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions src/prefill/moe_ws_down.cu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,12 @@
// layer's rows: [k/128][rows] block_q8_1_mmq, of which blocks 0-4 are read. (Measured on a GP100 die: ws / per-group
// MMQ 0.24-0.29 at 2.5 rows per expert, 0.38-0.49 at 10, 0.52-0.68 at 35, 0.61-0.74 at 160; bit-identical,
// compute-sanitizer clean with exact-size matrices.)
// These kernels are GP100-only (ws::mode() is 0 on any other card, so they never launch there) but this file is built
// for every architecture of the experimental build. They use half-precision arithmetic (HFMA2), which ptxas accepts
// from sm_53; a Maxwell (sm_52) build stopped here with "Feature 'f16 arithemetic and compare instructions' requires
// .target sm_53 or higher". Below sm_53 the device pass is left empty: the host code is unchanged, and for sm_60 and
// newer the device code is too.
#if !defined(__CUDA_ARCH__) || __CUDA_ARCH__ >= 530
#include "common.cuh"
#include "mmq.cuh"

Expand Down Expand Up @@ -643,3 +649,5 @@ void dn_iq4_nl(const Layer& L, double u, cudaStream_t s, bool fast) {
}

} // namespace strata::prefill::ws::detail

#endif // !defined(__CUDA_ARCH__) || __CUDA_ARCH__ >= 530
Loading