Skip to content

About reproducing the results in the blog #166

Description

@sglucas

Thank you very much for your great work.

May I ask how to reproduce the results in the blog video of your README and which GPU you are using. The main concern is that I use the following command and cannot get the results.

dflash benchmark transformers \
    --model Qwen/Qwen3-8B --draft z-lab/Qwen3-8B-DFlash-b16 \
    --dataset gsm8k --max-samples 200 \
    --reasoning off --max-new-tokens 512

Thr result is:

Baseline throughput: 45.48 tok/s
DFlash throughput:  215.20 tok/s
Decoding speedup: 4.73
Average Acceptance length: 6.39
Acceptance length histogram: ['0.0%', '11.2%', '16.4%', '12.0%', '8.7%', '6.9%',
'6.2%', '5.5%', '4.3%', '4.8%', '2.5%', '3.7%', '2.3%', '2.5%', '2.5%', '2.0%', '8.4%']

But the DFlash throught in the video is 400+ tok/s

Image

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions