b9752
ggml-org/llama.cppb9752Jun 21, 2026by github-actions[bot]same day as b9751
What changed since b9751
Server batch construction refactored with new abort_all_slots capability and safer batch-full handling.
New features
- Added abort_all_slots functionality to the server for cancelling all active slots at once.
Improvements
- Refactored the server's batch construction logic for clearer and more maintainable code.
- Batch overflow is now handled more carefully instead of being pushed past capacity.
- Added optional debug timing instrumentation (disabled by default) with forced synchronization for accurate measurements.
Bug fixes
- Corrected internal assertion failures exposed by the batch construction rework.
Upgrade notes
This is an internal server refactor with no documented config or API changes, so a drop-in binary swap should suffice. If you rely on slot management behavior, review the new abort_all_slots path and verify batch-full edge cases under load. No migration steps are required.
Analyzed Nov 1, 2026
Upgrading?
Release notes summary
Refactored the server's batch construction logic to handle batch overflow more carefully and added an abort mechanism for slots.
Key Highlights
- Refactor server batch construction
- Add abort_all_slots functionality
- Handle batch full more carefully
- Fix internal asserts
Release Notes (excerpt)
<details open> server: refactor batch construction (#24843) * server: refactor batch construction * wip * wip 2 * wip 3 * wip 4 * add abort_all_slots * handle batch full more carefully * fix assert * rm debug log * small nits * (debug) add timings * debug: force llama_synchronize for accurate timings * address comments * disable DEBUG_TIMINGS </details> **macOS/iOS:** - macOS Apple Silicon (arm64) - macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED - macOS Intel (x64)…
Published by ggml-org on GitHub.
Read the full release notesRelated Repositories
:electron: Build cross-platform desktop apps with JavaScript, HTML, and CSS
Godot Engine – Multi-platform 2D and 3D game engine
ClickHouse® is a real-time analytics database management system
Self-hosted game stream host for Moonlight.
DuckDB is a high-performance analytical database system