b10312
ggml-org/llama.cppb10312Aug 7, 2026by github-actions[bot]same day as b10311
What changed since b10311
Server router now skips eviction of models that are actively in use, preventing busy models from being unloaded.
New features
- Router eviction policy now excludes models with in-flight or active requests, so busy models stay resident in memory.
Improvements
- Router memory management is more robust under concurrent load, reducing the chance that an in-use model is swapped out mid-request.
Upgrade notes
This is a small behavioral change confined to the server router; no config or CLI flags are introduced. Operators running multi-model router setups should expect fewer evictions under load, which may increase peak memory residency. The Qwen3-TTS MTMD fix from b10311 is unchanged and carries over.
Analyzed Oct 30, 2026
Upgrading?
Release notes summary
This release modifies the server router's eviction policy to prevent evicting busy models. This ensures that models currently in use are not unloaded from memory unnecessarily.
Key Highlights
- Server router now skips eviction for busy models (#26567)
Release Notes (excerpt)
<details open> server: (router) do not evict busy models (#26567) </details> **Website:** - <https://llama.app> **macOS/iOS:** - macOS Apple Silicon (arm64) - macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED - macOS Intel (x64) - iOS XCFramework **Linux:** - Ubuntu x64 (CPU) - Ubuntu arm64 (CPU) - Ubuntu s390x (CPU) - Ubuntu x64 (Vulkan) - Ubuntu arm64 (Vulkan) - Ubuntu x64 (ROCm 7.2) - Ubuntu x64 (OpenVINO) - Ubuntu x64 (SYCL FP32) - Ubuntu x64 (SYCL FP16) **Android:**…
Published by ggml-org on GitHub.
Read the full release notesRelated Repositories
:electron: Build cross-platform desktop apps with JavaScript, HTML, and CSS
Godot Engine – Multi-platform 2D and 3D game engine
ClickHouse® is a real-time analytics database management system
Self-hosted game stream host for Moonlight.
DuckDB is a high-performance analytical database system