b10313
ggml-org/llama.cppb10313Aug 7, 2026by github-actions[bot]same day as b10312
What changed since b10312
Server router gains an LRU scheduler for model eviction and request queue management.
New features
- LRU-based scheduler added to the server router to manage model eviction and waiting requests.
- Request coalescing is handled when a request leaves the waiting queue.
- Streaming request handling added for the new scheduler path.
Improvements
- Router scheduling logic refactored around a dedicated LRU scheduler component.
- Test coverage added for the router scheduler behavior.
Bug fixes
- Fixed stream case handling in the router scheduler.
Upgrade notes
This is a single-commit step over b10312, so the only behavioral change is in the server router's eviction/scheduling path. No config or CLI changes are mentioned; if you rely on router model-loading order or eviction timing, re-validate under load after upgrading.
Analyzed Oct 30, 2026
Upgrading?
Release notes summary
This release adds an LRU (Least Recently Used) scheduler to the server's router component. This allows for better management of model evictions and waiting queues.
Key Highlights
- Added LRU scheduler to server router (#26572)
- Handles coalescing of requests leaving the waiting queue
- Added tests and fixed stream case handling
Release Notes (excerpt)
<details open> server: (router) add LRU scheduler (#26572) * add lru_sched * handle coalescing (req leaves waiting queue) * add tests * fix stream case * address review comments </details> **Website:** - <https://llama.app> **macOS/iOS:** - macOS Apple Silicon (arm64) - macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED - macOS Intel (x64) - iOS XCFramework **Linux:** - Ubuntu x64 (CPU) - Ubuntu arm64 (CPU) - Ubuntu s390x (CPU) - Ubuntu x64 (Vulkan) - Ubuntu arm64 (Vulkan)…
Published by ggml-org on GitHub.
Read the full release notesRelated Repositories
:electron: Build cross-platform desktop apps with JavaScript, HTML, and CSS
Godot Engine – Multi-platform 2D and 3D game engine
ClickHouse® is a real-time analytics database management system
Self-hosted game stream host for Moonlight.
DuckDB is a high-performance analytical database system