llama.cpp Adds MTP and DSpark Speculative Decoding for DeepSeek V4 Flash, Delivering ~50% Inference Speedup
Summary
- • llama.cpp PR #25784 merges MTP and DSpark speculative decoding support for DeepSeek V4 Flash local inference
- • DGX Spark benchmarks show wall time drops from 102s to 67s with MTP (~50% faster) and 56s with DSpark mode enabled
- • MTP draft acceptance rate reaches 65%; peak throughput rises from ~16 tok/s baseline to 39 tok/s on translation tasks with DSpark
- • Standalone MTP-only GGUF files published on HuggingFace, enabling users to add speedup without re-downloading the full model
Details
MTP Speculative Decoding Added to llama.cpp
PR #25784 adds Multi-Token Prediction head support for DeepSeek V4 Flash in llama.cpp, enabling speculative draft generation; enabled via the --spec-draft-n-max flag.
~50% Wall-Time Reduction with MTP
Standard 9-task benchmark wall time drops from 102.15s to 66.9s with MTP enabled; tok/s rises from ~16.3–16.5 baseline to 23.2–29.7 across task types.
65.09% Draft Token Acceptance Rate (MTP)
MTP achieves a 65.09% aggregate draft acceptance rate across 9 benchmark tasks, indicating high MTP head quality — roughly two-thirds of speculatively generated tokens are accepted by the verifier.
DSpark Mode Pushes Peak to 39.3 tok/s
Using --spec-type draft-dspark with n-max 5, translation and stepwise math tasks hit 39.3 tok/s; overall wall time falls to 55.95s. DSpark accept rate is 46.42% — lower but still beneficial at higher draft counts.
Standalone MTP GGUF Files on HuggingFace
ddh0 published MTP-only GGUF files at huggingface.co/ddh0/DeepSeek-V4-Flash-GGUF, letting users add MTP to existing quantized models without re-downloading the full checkpoint.
PR Currently WIP; Community Already Testing
Author notes the implementation is a work-in-progress and will be cleaned up over the following days. Early testers confirmed it working on Ubuntu with 4× Radeon 9700 GPUs and mixed CPU/GPU offload via DDR4 system RAM.
Source: llama.cpp GitHub PR #25784 (via Reddit AI, ingested 2026-08-02). Benchmark data from DGX Spark hardware runs. PR developed with Claude and Codex assistance per author disclosure.
What This Means
llama.cpp's new MTP and DSpark speculative decoding support cuts inference time for DeepSeek V4 Flash by roughly half on standard benchmarks, with peak throughput reaching 39 tok/s on the DGX Spark — a significant practical improvement for users running frontier models locally. The key mechanism is speculative decoding: the model drafts multiple tokens ahead and accepts them at a ~65% rate (MTP) or ~46% rate (DSpark), reducing the sequential compute bottleneck. The availability of standalone MTP GGUF files on HuggingFace ensures most existing users can benefit without redownloading the full model. For the open-source AI community, this narrows the performance gap with cloud inference APIs and makes local DeepSeek V4 Flash deployment considerably more practical.