b9591

ggml-org/llama.cppb9591Jun 10, 2026by github-actions[bot]same day as b9590

What changed since b9590

medium risk

b9591 replaces the MTP padding/copy hack with a single strided copy and refactors the gated-delta-net op signature.

Breaking changes

  • ggml_gated_delta_net now accepts only the initial recurrent state (D, 1, n_seqs) and takes the snapshot count K as an op parameter, so any out-of-tree backend or custom graph builder using the old signature must be updated.

Improvements

  • MTP recurrent state handling no longer pads state tensors, cutting memory overhead.
  • Emitted snapshots are written into the recurrent cache with one strided ggml_cpy instead of several device-to-device copies.
  • The gated-delta-net op signature change was propagated consistently across all supported backends.

Bug fixes

  • Resolved CI build errors introduced while rolling the gated-delta-net changes through the backends.

Upgrade notes

This build only touches internal MTP/gated-delta-net plumbing, so no config or CLI changes are needed for normal users. If you maintain a custom ggml backend or build graphs by hand, update your ggml_gated_delta_net call to the new initial-state-only signature with K passed as an op parameter. The LFM2 json_schema fix from b9590 is unchanged and carries over.

Analyzed Nov 1, 2026

Upgrading?

Release notes summary

Optimization of MTP (Multi-Token Prediction) recurrent state handling to reduce memory overhead.

Key Highlights

  • Removed padding and multiple D2D copies for MTP
  • Refactored ggml_gated_delta_net to take initial recurrent state as a parameter
  • Implemented single strided copy for emitting snapshots

New Features

  • MTP performance optimization
  • Memory efficiency improvement

Release Notes (excerpt)

<details open> Remove padding and multiple D2D copies for MTP (#24086) * Make ggml_gated_delta_net take only the initial recurrent state (D, 1, n_seqs) and passes the snapshot count K as an op parameter instead of inferring it from state->ne[1]. Remove the padding hack and copy all emitted snapshots into the recurrent cache with a single strided ggml_cpy * Make GDN changes in all backends. Address review comments. * Fix CI build errors </details> **macOS/iOS:**…

Published by ggml-org on GitHub.

Read the full release notes

Related Repositories