b10519
ggml-org/llama.cppb10519Aug 20, 2026by github-actions[bot]same day as b10517
What changed since b10517
Server sleep handling refactored so /metrics stays reachable while sleeping, plus race-condition fixes; Vulkan FA work from b10517 is unchanged.
New features
- The /metrics endpoint can now be queried while the server is in a sleeping state, improving observability of idle instances.
- Cached responses are served for requests made during sleep, avoiding redundant recomputation.
Improvements
- Sleep-state handling was refactored around on_sleeping_state for clearer control flow.
- Metrics and slots background tasks/results were split into separate paths for cleaner concurrency.
- Server documentation was updated to reflect the new sleep and metrics behavior.
- Vulkan Flash Attention retains the KV dequantization and scratch-buffer fallback work introduced in the previous build.
Bug fixes
- Fixed a race condition in the server's metrics task.
- Corrected get_res_model_info behavior in the refactored server code.
- The metrics task no longer resets the timer, so bucket reset decisions are handled explicitly.
Upgrade notes
No config or migration steps are required; the change is internal to the server's sleep/metrics plumbing. If you scrape /metrics, expect samples to be available during sleep where they previously were not, and re-check any alerting that assumed a gap. Verify custom monitoring or wrappers that depended on the old sleeping-state response behavior.
Analyzed Oct 24, 2026
Upgrading?
Release notes summary
Refactors the server's sleep handling logic to allow access to the `/metrics` endpoint even when the server is in a sleeping state, improving observability and fixing race conditions.
Key Highlights
- Refactored sleep handling logic
- Allowed /metrics access during sleep
- Fixed race conditions in metrics tasks
Release Notes (excerpt)
<details open> server: refactor sleep handling, allow access /metrics during sleep (#27376) * add cached responses * refactor on_sleeping_state * allow accessing metrics during sleep * metrics task should not reset timer * updated docs * fix * fix get_res_model_info * add test * fix a race condition * split metrics and slots tasks / results * should_reset_buckets </details> **Website:** - <https://llama.app> **Attestations:**…
Published by ggml-org on GitHub.
Read the full release notesRelated Repositories
:electron: Build cross-platform desktop apps with JavaScript, HTML, and CSS
Godot Engine – Multi-platform 2D and 3D game engine
ClickHouse® is a real-time analytics database management system
Self-hosted game stream host for Moonlight.
DuckDB is a high-performance analytical database system