b10210
ggml-org/llama.cppb10210Jul 31, 2026by github-actions[bot]same day as b10209
What changed since b10209
Server-side fix corrects accepted-token accounting when speculative draft tokens must be replayed, replacing the prior CUDA Q2_0 kernel tweak as the only change.
Improvements
- Speculative decoding token accounting in the server was reworked and renamed for clarity as part of the fix.
- The previous release's CUDA Q2_0 __byte_perm element-extraction optimization remains in place.
Bug fixes
- The server API now reports the correct number of accepted tokens when draft token replay is required during generation.
- Speculative decoding paths that trigger replay no longer produce skewed acceptance counts in server responses.
Upgrade notes
This is a single-commit increment over b10209, so no config or model changes are needed. Upgrade if you rely on server-reported accepted-token statistics with speculative decoding enabled; otherwise the delta is negligible.
Analyzed Oct 31, 2026
Upgrading?
Release notes summary
This release fixes a bug in the server API where accepted tokens were not correctly counted when draft token replay was needed during generation.
Key Highlights
- Corrects accepted tokens when draft token replay is needed
- Fixes server API correctness
New Features
- Server token replay fix
Release Notes (excerpt)
<details open> server: correct accepted tokens when need draft token replay (#26320) * spec: correct accepted tokens when need draft token replay * cont : naming --------- Co-authored-by: Georgi Gerganov <ggerganov@gmail.com> </details> **Website:** - <https://llama.app> **macOS/iOS:** - macOS Apple Silicon (arm64) - macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED - macOS Intel (x64) - iOS XCFramework **Linux:** - Ubuntu x64 (CPU) - Ubuntu arm64 (CPU) - Ubuntu s390x (CPU)…
Published by ggml-org on GitHub.
Read the full release notesRelated Repositories
:electron: Build cross-platform desktop apps with JavaScript, HTML, and CSS
Godot Engine – Multi-platform 2D and 3D game engine
ClickHouse® is a real-time analytics database management system
Self-hosted game stream host for Moonlight.
DuckDB is a high-performance analytical database system