b9910

ggml-org/llama.cppb9910Jul 8, 2026by github-actions[bot]same day as b9909

What changed since b9909

low risk

Fixes draft-model fit/load mismatch and refactors speculative decoding init in the server.

Improvements

  • Speculative initialization logic moved into speculative.cpp for cleaner separation of concerns.
  • Draft and MTP parameter initialization, model loading, and context loading are now unified.
  • Progress callbacks are no longer throttled while the draft model loads, giving smoother feedback.
  • Draft model/context load method renamed for clarity.

Bug fixes

  • Resolved an inconsistency between how the draft model was fitted versus loaded, which could destabilize speculative decoding.

Upgrade notes

This is a small patch-level delta with no config or CLI changes expected; a rebuild/redeploy of the server binary is sufficient. If you rely on speculative decoding, re-test draft model loading and progress reporting after upgrading, since internal ownership of the draft model/context changed to raw pointers.

Analyzed Oct 31, 2026

Upgrading?

Release notes summary

Resolves inconsistencies between how draft models are fitted and loaded in the server, improving the stability of speculative decoding implementations.

Key Highlights

  • Fix draft model fit vs load inconsistency
  • Refactor speculative initialization logic
  • Unify draft/mtp parameter initialization
  • Don't throttle progress callback when loading draft model

New Features

  • Speculative decoding stability improvements

Release Notes (excerpt)

<details open> server : fix draft model fit vs load inconsistency (#25056) * fix: draft model fit vs load inconsistency * refactor(server): unify draft/mtp parameter initialization, model, and context load - moves speculative init to speculative.cpp - changes server_context_impl model_dft and ctx_dft to use raw pointers - fix: don't throttle progress callback when loading draft model - refactor: rename draft model/ctx load method * fix: valign </details> **macOS/iOS:**…

Published by ggml-org on GitHub.

Read the full release notes

Related Repositories