b9606

ggml-org/llama.cppb9606Jun 12, 2026by github-actions[bot]same day as b9605

What changed since b9605

medium risk

b9606 adds EAGLE3 speculative decoding with layer-input extraction, plus internal speculative-decoding API cleanup.

Breaking changes

  • The internal helper common_speculative_setup_draft_model() was removed, so downstream code calling it must be updated.
  • Speculative cparams field output_layer_inp was renamed to embeddings_layer_inp, and the target_hidden_size hyperparameter was dropped.
  • Several unused llama APIs were cleaned up, which can break out-of-tree builds that referenced them.

New features

  • EAGLE3 speculative decoding is now supported, including layer input extraction from the target model.
  • Gemma4 EAGLE3 draft models from RedHatAI are recognized and usable.
  • Draft-model output.weight is now optional and inherited from the target model when absent.
  • A generic norm-before-residual hyperparameter was introduced for architecture definitions.

Improvements

  • Feature extraction from the target model now sets synchronization correctly.
  • Micro-batch handling in embedding layer-input extraction and the encoder was corrected.
  • d2t vocabulary mapping is now applied inside the decode graph rather than at setup time.
  • Naming and style cleanups across speculative decoding and llama-ext code.

Bug fixes

  • Fixed a parameter-handling bug in the EAGLE3 implementation.
  • Fixed a multi-sequence issue in the d2t vocabulary mapping.
  • Fixed rebase/upstream compatibility issues and excluded the eagle3 arch from test-llama-archs.
  • Corrected editorconfig check failures introduced by the new code.

Upgrade notes

Binary users can upgrade directly; the new speculative decoding path is opt-in via EAGLE3 draft models. If you build against llama.cpp or llama-ext sources, update references to the removed setup helper and the renamed embeddings_layer_inp parameter, and drop any use of target_hidden_size. Verify multi-sequence speculative workloads, since d2t mapping and ubatch handling changed.

Analyzed Nov 1, 2026

Upgrading?

Release notes summary

Implements support for EAGLE3 speculative decoding, enabling layer input extraction for improved generation efficiency. This includes compatibility with Gemma4 eagle3 and fixes for parameter bugs and multi-sequence issues.

Key Highlights

  • Add EAGLE3 speculative decoding support
  • Enable layer input extraction
  • Support Gemma4 eagle3 from RedHatAI
  • Fix params bug
  • Fix multi-seq issue in d2t vocab mapping

Release Notes (excerpt)

<details open> spec: add EAGLE3 speculative decoding support (#18039) * llama : enable layer input extraction * spec: support eagle3 * eagle3: fix params bug * eagle3: support Gemma4 eagle3 from RedHatAI * eagle3: set sync when get features from target Co-authored-by: tnhnyzc <115956684+tnhnyzc@users.noreply.github.com> * eagle3 : fix ubatch handling in embd_layer_inp extraction and encoder Co-authored-by: Doğaç Eldenk <dogacel@gmail.com> * eagle3: adapt to upstream changes…

Published by ggml-org on GitHub.

Read the full release notes

Related Repositories