Wanted to flag this properly rather than silently benefit from it: took the hit/miss split idea for MoE prefill weight uploads from FreeToken - device-to-device gather for experts already cache-resident, one batched transfer for the rest, instead of always paying full PCIe cost. Re-derived it against our own LFRU pool rather than porting the CUDA kernels directly (different engine shape - single unified cache vs. ours), but the design is yours. Thank you.
Context: https://github.com/sangharshadhyeta/solid.cpp
Wanted to flag this properly rather than silently benefit from it: took the hit/miss split idea for MoE prefill weight uploads from FreeToken - device-to-device gather for experts already cache-resident, one batched transfer for the rest, instead of always paying full PCIe cost. Re-derived it against our own LFRU pool rather than porting the CUDA kernels directly (different engine shape - single unified cache vs. ours), but the design is yours. Thank you.
Context: https://github.com/sangharshadhyeta/solid.cpp