b9752

ggml-org/llama.cppb9752Jun 21, 2026by github-actions[bot]same day as b9751

What changed since b9751

medium risk

Server batch construction refactored with new abort_all_slots capability and safer batch-full handling.

New features

  • Added abort_all_slots functionality to the server for cancelling all active slots at once.

Improvements

  • Refactored the server's batch construction logic for clearer and more maintainable code.
  • Batch overflow is now handled more carefully instead of being pushed past capacity.
  • Added optional debug timing instrumentation (disabled by default) with forced synchronization for accurate measurements.

Bug fixes

  • Corrected internal assertion failures exposed by the batch construction rework.

Upgrade notes

This is an internal server refactor with no documented config or API changes, so a drop-in binary swap should suffice. If you rely on slot management behavior, review the new abort_all_slots path and verify batch-full edge cases under load. No migration steps are required.

Analyzed Nov 1, 2026

Upgrading?

Release notes summary

Refactored the server's batch construction logic to handle batch overflow more carefully and added an abort mechanism for slots.

Key Highlights

  • Refactor server batch construction
  • Add abort_all_slots functionality
  • Handle batch full more carefully
  • Fix internal asserts

Release Notes (excerpt)

<details open> server: refactor batch construction (#24843) * server: refactor batch construction * wip * wip 2 * wip 3 * wip 4 * add abort_all_slots * handle batch full more carefully * fix assert * rm debug log * small nits * (debug) add timings * debug: force llama_synchronize for accurate timings * address comments * disable DEBUG_TIMINGS </details> **macOS/iOS:** - macOS Apple Silicon (arm64) - macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED - macOS Intel (x64)…

Published by ggml-org on GitHub.

Read the full release notes

Related Repositories