b10282

ggml-org/llama.cppb10282Aug 5, 2026by github-actions[bot]same day as b10280

What changed since b10280

low risk

b10282 adds vLLM-aligned speculative-decoding counters to the server's /metrics endpoint.

New features

  • The /metrics Prometheus endpoint now exposes speculative-decoding counters for monitoring draft-model acceptance and throughput.
  • Metric parameter names follow vLLM conventions exactly, easing reuse of existing vLLM dashboards and alerts.

Improvements

  • Improved observability of speculative decoding via standard server metrics rather than logs alone.
  • Vendor subprocess.h patches from b10280 remain in place; no regressions noted.

Upgrade notes

This is an additive observability change with no config or CLI changes required. If you scrape /metrics, expect new series to appear; update dashboards or recording rules if you filter metric names strictly. The vLLM-style naming means existing vLLM-based queries may work unchanged.

Analyzed Oct 31, 2026

Upgrading?

Release notes summary

Adds spec-decode counters to the /metrics endpoint to improve observability and align with vLLM standards.

Key Highlights

  • Adds spec-decode counters to /metrics endpoint
  • Aligns parameter names exactly with vLLM

Release Notes (excerpt)

<details open> server: Adding spec-decode counters to /metrics endpoint (#26389) * * server: add spec-decode counters to /metrics endpoint * server: fixed review comments and now aligned param names exactly with vLLM. </details> **Website:** - <https://llama.app> **macOS/iOS:** - macOS Apple Silicon (arm64) - macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED - macOS Intel (x64) - iOS XCFramework **Linux:** - Ubuntu x64 (CPU) - Ubuntu arm64 (CPU) - Ubuntu s390x (CPU)…

Published by ggml-org on GitHub.

Read the full release notes

Related Repositories