llama.cpp v0.6.0 Released with Extended Batch API, MTP Speculative Decoding, and Major Performance Gains
Summary
- • llama.cpp v0.6.0 ships new extended batch API for mixed token and embedding inputs
- • MTP speculative decoding for Qwen4Exp delivers ~1.5x decode speedup on DGX Spark
- • New model support: GLM-5.3-Flash 320B hybrid and Clef decision model added
- • Apple GPU mat-mul up to ~3x faster; Web UI overhauled with HuggingFace Hub layer
Details
New llama_batch_ext extended batch API with llama_process()
The new API supports mixed token/embedding batches and per-token 'state' embeddings for MTP and deepstack models, replacing the previous batch system across examples, speculative decoding, and the server.
MTP speculative decoding for Qwen4Exp: ~1.5x speedup
Qwen4Exp now has high-quality MTP speculative decoding support with probabilistic sampling, delivering approximately 1.5x decode speedup on DGX Spark hardware with various correctness fixes.
GLM-5.3-Flash 320B hybrid model and Clef decision model added
GLM-5.3-Flash (GLM5-Next) is a 320B KDA/DSA hybrid text+vision model with MoE. Clef is a decision model with full text and vision support. Nimble and Ling 3.0 VL also added.
New /v1/systemone server API for five decision models
A new server endpoint supports the laya, julia-1, lev, openjev (with vision), and kev decision models. The /v1/embeddings endpoint also now accepts typed vision/audio/video content.
Apple GPU MMA kernels deliver up to ~3x faster mat-mul
New few-row MMA mat-mul kernels for speculative and batched decoding on Metal backend achieve up to ~3x faster mat-mul on Apple GPUs. A new tensor API flash attention kernel for F16 KV also added.
Vulkan: sparse flash attention for quantized K/V added
The Vulkan backend gains sparse flash attention support for quantized K/V caches, expanding GPU-accelerated inference options beyond Apple Metal.
Web UI overhauled with HuggingFace Hub data and model download layer
The built-in Web UI receives a significant overhaul featuring a Hugging Face Hub data layer and integrated model download pipeline, improving the local inference user experience.
ggml updated to v0.26.0; session format versions bumped
ggml dependency updated to v0.26.0. LLAMA_SESSION_VERSION bumped to 11 and LLAMA_STATE_SEQ_VERSION to 4, meaning existing saved sessions may need to be regenerated.
llama.cpp v0.6.0 release notes — key features, performance improvements, and API changes.
What This Means
llama.cpp v0.6.0 is a substantial infrastructure release that meaningfully advances local LLM inference capability. The new extended batch API lays groundwork for more flexible model pipelines, while MTP speculative decoding and improved Metal kernels deliver real-world throughput gains for users running Qwen4Exp and Apple Silicon hardware. The addition of 320B hybrid model support and a dedicated decision-model API signals llama.cpp's expanding role beyond basic chat inference toward agentic and multimodal workloads. The HuggingFace Hub integration further lowers the barrier to local model deployment for non-expert users.