b9591
ggml-org/llama.cppb9591Jun 10, 2026by github-actions[bot]same day as b9590
What changed since b9590
b9591 replaces the MTP padding/copy hack with a single strided copy and refactors the gated-delta-net op signature.
Breaking changes
- ggml_gated_delta_net now accepts only the initial recurrent state (D, 1, n_seqs) and takes the snapshot count K as an op parameter, so any out-of-tree backend or custom graph builder using the old signature must be updated.
Improvements
- MTP recurrent state handling no longer pads state tensors, cutting memory overhead.
- Emitted snapshots are written into the recurrent cache with one strided ggml_cpy instead of several device-to-device copies.
- The gated-delta-net op signature change was propagated consistently across all supported backends.
Bug fixes
- Resolved CI build errors introduced while rolling the gated-delta-net changes through the backends.
Upgrade notes
This build only touches internal MTP/gated-delta-net plumbing, so no config or CLI changes are needed for normal users. If you maintain a custom ggml backend or build graphs by hand, update your ggml_gated_delta_net call to the new initial-state-only signature with K passed as an op parameter. The LFM2 json_schema fix from b9590 is unchanged and carries over.
Analyzed Nov 1, 2026
Upgrading?
Release notes summary
Optimization of MTP (Multi-Token Prediction) recurrent state handling to reduce memory overhead.
Key Highlights
- Removed padding and multiple D2D copies for MTP
- Refactored ggml_gated_delta_net to take initial recurrent state as a parameter
- Implemented single strided copy for emitting snapshots
New Features
- MTP performance optimization
- Memory efficiency improvement
Release Notes (excerpt)
<details open> Remove padding and multiple D2D copies for MTP (#24086) * Make ggml_gated_delta_net take only the initial recurrent state (D, 1, n_seqs) and passes the snapshot count K as an op parameter instead of inferring it from state->ne[1]. Remove the padding hack and copy all emitted snapshots into the recurrent cache with a single strided ggml_cpy * Make GDN changes in all backends. Address review comments. * Fix CI build errors </details> **macOS/iOS:**…
Published by ggml-org on GitHub.
Read the full release notesRelated Repositories
:electron: Build cross-platform desktop apps with JavaScript, HTML, and CSS
Godot Engine – Multi-platform 2D and 3D game engine
ClickHouse® is a real-time analytics database management system
Self-hosted game stream host for Moonlight.
DuckDB is a high-performance analytical database system