llama.cpp Releases

This release fixes stale tensor-split parameters for draft models, specifically correcting metadata for GQA attention. It also includes a revert of a recent code tidying commit and a backend meta asset hack.

Jul 5, 2026

This release introduces a safety check that aborts the program execution if a multi-buffer configuration is detected, likely to prevent undefined behavior or crashes in specific backend scenarios.

Jul 5, 2026

Fixes a crash when using tensor parallelism (-sm tensor) combined with CPU-offloaded MoE experts (-ncmoe). The fix moves the split-state lookup before the contiguity assert.

Jul 5, 2026

Implements a CUDA version of the concat operation for quantized tensors. This mirrors the CPU fix from b9871 and improves GPU backend performance.

Jul 5, 2026

Adds a guard for K/V rotation inputs when buffers are unallocated. This prevents a crash (GGML_ASSERT) in DFlash speculative decoding when buffers are NULL.

Jul 4, 2026

Fixes a broken CPU implementation for concatenating quantized tensors. The previous implementation likely failed or produced incorrect results for quantized types.

Jul 4, 2026

Fixes long reasoning loops in chat by trimming messages sent to the StepFun parser. The trimming is applied before rendering to content parts.

Jul 3, 2026

Extends support for speculative decoding in DFlash by adding the spec-draft-p-min parameter. Includes guards for n_min to ensure valid configurations.

Jul 3, 2026

Enables topk-moe fusion for 288 experts in CUDA, previously only supported for power-of-2 counts or 576. This improves decode performance for large MoE models.

Jul 3, 2026

Enhances server and UI stability for SSE streams by adding silent ping mechanisms. It introduces a configurable ping interval to prevent disconnects during slow prefill operations.

Jul 3, 2026

This release optimizes CUDA kernels by removing redundant copies after gated_delta_net operations. It writes recurrent state snapshots directly into the recurrent cache, reducing kernel calls for MTP draft lengths.

Jul 3, 2026

Updates the vendor library cpp-httplib to version 0.49.0. This is a dependency update with no functional changes to the core inference engine.

Jul 3, 2026

Adds a new public C API function to retrieve the name of the model quantization file type (e.g., 'Q8_0'), improving introspection capabilities for applications.

Jul 2, 2026

Enhances OpenCL backend performance by allowing the loading of precompiled binary kernels from a shared library for GEMM operations, including support for MOE and various quantization types.

Jul 1, 2026

Fixes a bug where the Hugging Face primary split wasn't being used as the model path, ensuring correct model loading behavior.

Jul 1, 2026

Performs a major rework of the Hexagon Flash Attention implementation, focusing on performance optimizations, accuracy improvements, and code refactoring including fp16 math usage and mask processing.

Jul 1, 2026

Refactors CUDA Flash Attention code to consistently use pointer restrictions and PDL (Pointer Distance Logic) for better code optimization and consistency.

Jul 1, 2026

Enhances CPU performance by adding AVX2 optimizations for NVFP4 dot product operations and utilizing the UE4M3 lookup table for improved efficiency.

Jul 1, 2026

Introduces support for the q1_0 quantization format in the OpenCL backend, including specific optimizations for Adreno GPUs to accelerate GEMM and GEMV operations.

Jul 1, 2026

Addresses critical bug fixes in the CUDA Flash Attention implementation by preventing integer overflow and truncation errors when calculating KQ mask strides.

Jun 30, 2026