Production-grade speculative decoding toolkit for LLMs — multi-algorithm, FP8 quantization, online training, distributed serving, and comprehensive evaluation suite.
python nlp benchmarking machine-learning deep-learning eagle pytorch medusa gpu-acceleration distributed-training model-acceleration inference-optimization ai-inference llm-serving llm-inference speculative-decoding fp8-quantization draft-models
-
Updated
Jul 1, 2026 - Python