b10011
ggml-org/llama.cppb10011Jul 14, 2026by github-actions[bot]same day as b10010
What changed since b10010
Server prompt cache state ownership refactored, moving prompt state into server_prompt_cache with batched slot clearing support.
Improvements
- Prompt state data is now owned by server_prompt_cache rather than scattered across server structures.
- Checkpoints are cleared when a prompt is cleared, keeping cache state consistent.
- Batched slots being cleared are now handled explicitly, improving multi-slot cache correctness.
Bug fixes
- Stale checkpoints could linger after a prompt clear; they are now discarded.
Upgrade notes
This is an internal server refactor with no CLI or HTTP API changes, so no configuration updates are required. Behavior around prompt caching and slot clearing may differ subtly, so validate multi-slot and cache-reuse workloads after upgrading.
Analyzed Oct 30, 2026
Upgrading?
Release notes summary
Refactored prompt cache state ownership in the server by moving prompt state data to `server_prompt_cache` and adding handling for batched slot clearing.
Key Highlights
- Refactored prompt cache state ownership in the server.
- Moved prompt state data to `server_prompt_cache`.
- Added handling for batched slot being cleared.
Release Notes (excerpt)
<details open> server : refactor prompt cache state ownership (#25649) * server : clear checkpoints upon prompt clear * server : move the prompt state data to the server_prompt_cache Assisted-by: pi:llama.cpp/Qwen3.6-27B * server : handle batched slot being cleared </details> **macOS/iOS:** - macOS Apple Silicon (arm64) - macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED - macOS Intel (x64) - iOS XCFramework **Linux:** - Ubuntu x64 (CPU) - Ubuntu arm64 (CPU) - Ubuntu s390x (CPU)…
Published by ggml-org on GitHub.
Read the full release notesRelated Repositories
:electron: Build cross-platform desktop apps with JavaScript, HTML, and CSS
Godot Engine – Multi-platform 2D and 3D game engine
ClickHouse® is a real-time analytics database management system
Self-hosted game stream host for Moonlight.
DuckDB is a high-performance analytical database system