Summary
We are looking for volunteers to test the new DeepSeek V4 Flash Vision-Exp AProjQ4 GGUF on:
- Apple Silicon / Metal
- NVIDIA / CUDA
- AMD Strix Halo / ROCm
- Fully resident and SSD-streaming configurations
The runtime implementation is tracked in PR #621.
This request complements #838, which covers the non-Vision AProjQ4 model, and follows the original Vision support request in #851.
You only need to test one hardware/backend combination. Reports of crashes, incorrect image understanding, quality drift, unexpectedly low performance, or successful runs are all useful.
Model artifact
The experimental model is available from Hugging Face PR #22:
DeepSeek-V4-Flash-Vision-Exp-IQ2XXS-w2Q2K-AProjQ4-SExpQ8-OutQ8-imatrix.gguf
Size: 84,420,584,608 bytes (78.62 GiB)
SHA-256: a1d0fde967c910e37e4bea00c4db1b2e8a88978770828a95772bc66837386cb8
The model was produced by requantizing 215 dense-attention projections—five projections across 43 layers—from Q8_0 to Q4_K, using the 220k routed-and-dense DS4 imatrix.
All tensors outside the selected attention projections retain the source model’s quantization layout.
The matching Vision-Exp encoder sidecar is also required:
DeepSeek-V4-Flash-Vision-Encoder.gguf
Download
Install the current Hugging Face CLI if needed, then run:
mkdir -p gguf
hf download antirez/deepseek-v4-gguf \
DeepSeek-V4-Flash-Vision-Exp-IQ2XXS-w2Q2K-AProjQ4-SExpQ8-OutQ8-imatrix.gguf \
--revision refs/pr/22 \
--local-dir gguf
hf download antirez/deepseek-v4-gguf \
DeepSeek-V4-Flash-Vision-Encoder.gguf \
--revision main \
--local-dir gguf
Verify the language-model GGUF:
shasum -a 256 \
gguf/DeepSeek-V4-Flash-Vision-Exp-IQ2XXS-w2Q2K-AProjQ4-SExpQ8-OutQ8-imatrix.gguf
On Linux, use sha256sum instead.
Build the current PR head
Please fetch the latest PR head immediately before testing and report its exact commit:
git clone https://github.com/antirez/ds4.git ds4-vision-q4
cd ds4-vision-q4
git fetch origin pull/621/head:pr-621
git checkout pr-621
git rev-parse HEAD
Build for your backend:
# Apple Silicon / Metal
make clean
make
# DGX Spark / GB10
make clean
make cuda-spark
# Other NVIDIA CUDA systems
make clean
make cuda-generic
# AMD Strix Halo / ROCm
make clean
make strix-halo -j"$(nproc)"
Inspect the model first
MODEL=gguf/DeepSeek-V4-Flash-Vision-Exp-IQ2XXS-w2Q2K-AProjQ4-SExpQ8-OutQ8-imatrix.gguf
VISION=gguf/DeepSeek-V4-Flash-Vision-Encoder.gguf
./ds4 --cpu --inspect -m "$MODEL"
Expected high-level metadata:
GGUF v3
1328 tensors
43 layers
q4_k: 215 tensors
q8_0: 130 tensors
file size: 78.62 GiB
Run the model
Metal with SSD streaming
./ds4 --metal --ssd-streaming \
--ctx 4096 --temp 0 \
-m "$MODEL" \
--vision "$VISION"
CUDA with SSD streaming
./ds4 --cuda --ssd-streaming \
--ctx 4096 --temp 0 \
-m "$MODEL" \
--vision "$VISION"
ROCm with SSD streaming
./ds4 --rocm --ssd-streaming \
--ctx 4096 --temp 0 \
-m "$MODEL" \
--vision "$VISION"
For fully resident testing, remove --ssd-streaming. The model, vision encoder, KV cache, and runtime buffers must all fit in available memory.
Inside the interactive CLI, submit a PNG or JPEG with:
/read /path/to/test-image.png
Please use public or non-sensitive images.
Minimum useful test
Please try at least:
- One ordinary photograph with several recognizable objects.
- One screenshot or image containing readable text.
- One diagram, chart, or spatial-reasoning image.
- A follow-up question referring to details from the same image.
- A text-only prompt in the same session, to detect general language-quality regressions.
Check for:
- crashes or backend errors;
- NaNs or non-finite logits;
- repeated BOS tokens or looping output;
- incorrect image-token handling;
- severe hallucinations or loss of image grounding;
- unexpected quality differences from the AProjQ8 model;
- unusually slow vision encoding, prefill, or generation;
- excessive memory usage or SSD-cache churn.
Optional AProjQ8 comparison
If storage and memory permit, compare against the matching Vision-Exp AProjQ8 language model:
DeepSeek-V4-Flash-Vision-Exp-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8.gguf
Please use the same:
- DS4 commit;
- backend and execution mode;
- context size;
- vision encoder;
- image;
- prompt;
- temperature.
A matched AProjQ8/AProjQ4 comparison is especially valuable for detecting quantization-related quality drift.
Reporting template
Please post results using this template:
Hardware:
RAM / VRAM:
OS:
Backend:
Build command:
DS4 commit:
Resident or SSD streaming:
Context size:
Q4 SHA-256:
Vision encoder filename:
Exact command:
Image format and dimensions:
Image or public source link:
Result:
Response excerpt:
Vision grounding issues:
Crash/error output:
Vision/prefill timing:
Generation speed:
Peak or reported memory:
AProjQ8 comparison, if available:
Additional environment variables:
Other notes:
Please include complete error messages where possible, but redact private paths, tokens, credentials, and sensitive image content.
Current validation status
The GGUF has already passed:
- structural GGUF inspection;
- tensor-count and tensor-type validation;
- strict imatrix compatibility checks;
- a text-only Metal SSD-streaming smoke test covering all 43 layers.
Full multimodal validation across different GPUs and operating systems is still needed. No broad Vision quality-equivalence claim is being made yet.
Summary
We are looking for volunteers to test the new DeepSeek V4 Flash Vision-Exp AProjQ4 GGUF on:
The runtime implementation is tracked in PR #621.
This request complements #838, which covers the non-Vision AProjQ4 model, and follows the original Vision support request in #851.
You only need to test one hardware/backend combination. Reports of crashes, incorrect image understanding, quality drift, unexpectedly low performance, or successful runs are all useful.
Model artifact
The experimental model is available from Hugging Face PR #22:
DeepSeek-V4-Flash-Vision-Exp-IQ2XXS-w2Q2K-AProjQ4-SExpQ8-OutQ8-imatrix.ggufThe model was produced by requantizing 215 dense-attention projections—five projections across 43 layers—from
Q8_0toQ4_K, using the 220k routed-and-dense DS4 imatrix.All tensors outside the selected attention projections retain the source model’s quantization layout.
The matching Vision-Exp encoder sidecar is also required:
Download
Install the current Hugging Face CLI if needed, then run:
Verify the language-model GGUF:
On Linux, use
sha256suminstead.Build the current PR head
Please fetch the latest PR head immediately before testing and report its exact commit:
git clone https://github.com/antirez/ds4.git ds4-vision-q4 cd ds4-vision-q4 git fetch origin pull/621/head:pr-621 git checkout pr-621 git rev-parse HEADBuild for your backend:
Inspect the model first
MODEL=gguf/DeepSeek-V4-Flash-Vision-Exp-IQ2XXS-w2Q2K-AProjQ4-SExpQ8-OutQ8-imatrix.gguf VISION=gguf/DeepSeek-V4-Flash-Vision-Encoder.gguf ./ds4 --cpu --inspect -m "$MODEL"Expected high-level metadata:
Run the model
Metal with SSD streaming
CUDA with SSD streaming
ROCm with SSD streaming
For fully resident testing, remove
--ssd-streaming. The model, vision encoder, KV cache, and runtime buffers must all fit in available memory.Inside the interactive CLI, submit a PNG or JPEG with:
Please use public or non-sensitive images.
Minimum useful test
Please try at least:
Check for:
Optional AProjQ8 comparison
If storage and memory permit, compare against the matching Vision-Exp AProjQ8 language model:
Please use the same:
A matched AProjQ8/AProjQ4 comparison is especially valuable for detecting quantization-related quality drift.
Reporting template
Please post results using this template:
Please include complete error messages where possible, but redact private paths, tokens, credentials, and sensitive image content.
Current validation status
The GGUF has already passed:
Full multimodal validation across different GPUs and operating systems is still needed. No broad Vision quality-equivalence claim is being made yet.