nvidia-l4

Here are 5 public repositories matching this topic...

hanxiao / Qwen3.6-35B-A3B-MTP-L4

Deploy Qwen3.6-35B-A3B (Q4_K_XL) + MTP speculative decoding on a single NVIDIA L4 24GB — GCP g2-standard-8 — via the official llama.cpp Docker image. Decode-optimized to ~91–99 tok/s (min ~91 chat, max ~99 math), lossless (full GPU residency + ECC-off).

gcp moe quantization inference-optimization mixture-of-experts large-language-models llm llama-cpp gpu-inference local-llm qwen gguf speculative-decoding qwen3 multi-token-prediction nvidia-l4

Updated Aug 24, 2026
Shell

hanxiao / Qwen3.8-27B-UD-Q4_K_XL-L4

Star

Qwen3.8-27B (Unsloth UD-Q4_K_XL GGUF) served at 37.4 tok/s decode over a 226,048-token context on a single NVIDIA L4 24 GB: llama.cpp with DFlash 2 block-diffusion speculative decoding, a one-line CUDA kernel-routing patch worth 16%, one-command GCP provisioning, quality verification, and a measured account of where the ceiling is.

benchmark cuda gcp quantization mtp long-context llama-cpp local-llm llm-inference qwen gguf speculative-decoding qwen3 block-diffusion nvidia-l4 dflash

Updated Aug 20, 2026
Shell

LuSrodri / canva-copy-api

Sponsor

Star

Veja como usar umas das melhores IA para remoção de fundo de imagem!

python api docker machine-learning computer-vision image-processing transformers background-removal fastapi huggingface nvidia-l4

Updated Aug 26, 2025
Python

hjsblogger / tensorrt-l4-demo

Star

Minimal TensorRT demo on NVIDIA L4 GPUs: Export PyTorch MLP to ONNX, optimize with FP16, and achieve 40%+ latency reduction via batched inference.

python benchmarking tensor-rt gpu-cloud gpu-models llm nvidia-l4

Updated Nov 14, 2025
Python

amitgambhir / llm-inference-bench

Star

Benchmark study: NVIDIA L4 + Llama 3.1 8B FP8 on Red Hat OpenShift AI. Concurrent load benchmarks, quality evaluation, and deployment advisor grounded in real measured data.

benchmark quality-evaluation ttft fp8 llm-serving vllm llm-inference rhoai openshift-ai nvidia-l4

Updated Jun 15, 2026
Python

Improve this page

Add a description, image, and links to the nvidia-l4 topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the nvidia-l4 topic, visit your repo's landing page and select "manage topics."

Learn more