b10282
ggml-org/llama.cppb10282Aug 5, 2026by github-actions[bot]same day as b10280
What changed since b10280
b10282 adds vLLM-aligned speculative-decoding counters to the server's /metrics endpoint.
New features
- The /metrics Prometheus endpoint now exposes speculative-decoding counters for monitoring draft-model acceptance and throughput.
- Metric parameter names follow vLLM conventions exactly, easing reuse of existing vLLM dashboards and alerts.
Improvements
- Improved observability of speculative decoding via standard server metrics rather than logs alone.
- Vendor subprocess.h patches from b10280 remain in place; no regressions noted.
Upgrade notes
This is an additive observability change with no config or CLI changes required. If you scrape /metrics, expect new series to appear; update dashboards or recording rules if you filter metric names strictly. The vLLM-style naming means existing vLLM-based queries may work unchanged.
Analyzed Oct 31, 2026
Upgrading?
Release notes summary
Adds spec-decode counters to the /metrics endpoint to improve observability and align with vLLM standards.
Key Highlights
- Adds spec-decode counters to /metrics endpoint
- Aligns parameter names exactly with vLLM
Release Notes (excerpt)
<details open> server: Adding spec-decode counters to /metrics endpoint (#26389) * * server: add spec-decode counters to /metrics endpoint * server: fixed review comments and now aligned param names exactly with vLLM. </details> **Website:** - <https://llama.app> **macOS/iOS:** - macOS Apple Silicon (arm64) - macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED - macOS Intel (x64) - iOS XCFramework **Linux:** - Ubuntu x64 (CPU) - Ubuntu arm64 (CPU) - Ubuntu s390x (CPU)…
Published by ggml-org on GitHub.
Read the full release notesRelated Repositories
:electron: Build cross-platform desktop apps with JavaScript, HTML, and CSS
Godot Engine – Multi-platform 2D and 3D game engine
ClickHouse® is a real-time analytics database management system
Self-hosted game stream host for Moonlight.
DuckDB is a high-performance analytical database system