b9862
ggml-org/llama.cppb9862Jul 3, 2026by github-actions[bot]same day as b9861
What changed since b9861
CUDA gated_delta_net kernel now writes recurrent state directly to cache, eliminating redundant copy kernels for MTP workloads.
Improvements
- CUDA GDN kernel writes recurrent state snapshots straight into the recurrent cache, bypassing intermediate tail writes.
- Detects the gated_delta_net -> view -> cpy pattern and skips the extra ggml_cuda_cpy calls when safe.
- Removes 4 redundant copy kernels during target decode with MTP draft length 3 (K=4).
- Speeds up inference for models using Multi-Token Prediction on CUDA backends.
Upgrade notes
This is a CUDA-only performance optimization with no config or API changes; non-CUDA backends are unaffected. Rebuild or pull the new binaries to benefit from the reduced kernel launches in MTP decoding.
Analyzed Oct 30, 2026
Upgrading?
Release notes summary
This release optimizes CUDA kernels by removing redundant copies after gated_delta_net operations. It writes recurrent state snapshots directly into the recurrent cache, reducing kernel calls for MTP draft lengths.
Key Highlights
- Optimized CUDA GDN kernel to skip intermediate tail writes
- Reduces 4 extra copy kernels for MTP draft length 3
- Improves performance for models using Multi-Token Prediction (MTP)
New Features
- Optimized CUDA recurrent state handling
Release Notes (excerpt)
<details open> Remove redundant CUDA copies after gated_delta_net. (#23940) * Remove redundant CUDA copies after gated_delta_net. Currently, GDN writes recurrent state snapshots into its output tail, then the graph immediately copies those snapshots into ssm_states_all. With MTP draft length 3, target decode uses K=4, so that becomes 4 extra ggml_cuda_cpy calls.…
Published by ggml-org on GitHub.
Read the full release notesRelated Repositories
:electron: Build cross-platform desktop apps with JavaScript, HTML, and CSS
Godot Engine – Multi-platform 2D and 3D game engine
ClickHouse® is a real-time analytics database management system
Self-hosted game stream host for Moonlight.
DuckDB is a high-performance analytical database system