b10312

ggml-org/llama.cppb10312Aug 7, 2026by github-actions[bot]same day as b10311

What changed since b10311

low risk

Server router now skips eviction of models that are actively in use, preventing busy models from being unloaded.

New features

  • Router eviction policy now excludes models with in-flight or active requests, so busy models stay resident in memory.

Improvements

  • Router memory management is more robust under concurrent load, reducing the chance that an in-use model is swapped out mid-request.

Upgrade notes

This is a small behavioral change confined to the server router; no config or CLI flags are introduced. Operators running multi-model router setups should expect fewer evictions under load, which may increase peak memory residency. The Qwen3-TTS MTMD fix from b10311 is unchanged and carries over.

Analyzed Oct 30, 2026

Upgrading?

Release notes summary

This release modifies the server router's eviction policy to prevent evicting busy models. This ensures that models currently in use are not unloaded from memory unnecessarily.

Key Highlights

  • Server router now skips eviction for busy models (#26567)

Release Notes (excerpt)

<details open> server: (router) do not evict busy models (#26567) </details> **Website:** - <https://llama.app> **macOS/iOS:** - macOS Apple Silicon (arm64) - macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED - macOS Intel (x64) - iOS XCFramework **Linux:** - Ubuntu x64 (CPU) - Ubuntu arm64 (CPU) - Ubuntu s390x (CPU) - Ubuntu x64 (Vulkan) - Ubuntu arm64 (Vulkan) - Ubuntu x64 (ROCm 7.2) - Ubuntu x64 (OpenVINO) - Ubuntu x64 (SYCL FP32) - Ubuntu x64 (SYCL FP16) **Android:**…

Published by ggml-org on GitHub.

Read the full release notes

Related Repositories