b10519

ggml-org/llama.cppb10519Aug 20, 2026by github-actions[bot]same day as b10517

What changed since b10517

medium risk

Server sleep handling refactored so /metrics stays reachable while sleeping, plus race-condition fixes; Vulkan FA work from b10517 is unchanged.

New features

  • The /metrics endpoint can now be queried while the server is in a sleeping state, improving observability of idle instances.
  • Cached responses are served for requests made during sleep, avoiding redundant recomputation.

Improvements

  • Sleep-state handling was refactored around on_sleeping_state for clearer control flow.
  • Metrics and slots background tasks/results were split into separate paths for cleaner concurrency.
  • Server documentation was updated to reflect the new sleep and metrics behavior.
  • Vulkan Flash Attention retains the KV dequantization and scratch-buffer fallback work introduced in the previous build.

Bug fixes

  • Fixed a race condition in the server's metrics task.
  • Corrected get_res_model_info behavior in the refactored server code.
  • The metrics task no longer resets the timer, so bucket reset decisions are handled explicitly.

Upgrade notes

No config or migration steps are required; the change is internal to the server's sleep/metrics plumbing. If you scrape /metrics, expect samples to be available during sleep where they previously were not, and re-check any alerting that assumed a gap. Verify custom monitoring or wrappers that depended on the old sleeping-state response behavior.

Analyzed Oct 24, 2026

Upgrading?

Release notes summary

Refactors the server's sleep handling logic to allow access to the `/metrics` endpoint even when the server is in a sleeping state, improving observability and fixing race conditions.

Key Highlights

  • Refactored sleep handling logic
  • Allowed /metrics access during sleep
  • Fixed race conditions in metrics tasks

Release Notes (excerpt)

<details open> server: refactor sleep handling, allow access /metrics during sleep (#27376) * add cached responses * refactor on_sleeping_state * allow accessing metrics during sleep * metrics task should not reset timer * updated docs * fix * fix get_res_model_info * add test * fix a race condition * split metrics and slots tasks / results * should_reset_buckets </details> **Website:** - <https://llama.app> **Attestations:**…

Published by ggml-org on GitHub.

Read the full release notes

Related Repositories