b10538
ggml-org/llama.cppb10538Aug 21, 2026by github-actions[bot]same day as b10537
What changed since b10537
Metal backend now dequantizes the KV cache only for large batches, cutting overhead in macOS inference.
Improvements
- Metal KV cache dequantization is gated to large batches, avoiding per-step dequant work on small batches.
- Reduces backend overhead in memory-intensive Metal inference paths, improving throughput/efficiency on Apple Silicon.
Upgrade notes
This is a single-commit Metal performance tweak with no API, config, or CLI changes; a drop-in replacement for b10537. Only macOS/iOS Metal users are affected, and behavior for small batches should be unchanged apart from reduced overhead. The Windows OpenMP/LLVM packaging change landed in b10537 and is not part of this delta.
Analyzed Oct 27, 2026
Upgrading?
Release notes summary
Optimizes the macOS Metal backend by dequantizing the Key-Value (KV) cache only for large batches to reduce overhead and improve efficiency.
Key Highlights
- Metal dequantization optimization for KV cache
- Applied only for large batches to minimize overhead
- Improves performance for memory-intensive operations
New Features
- Metal backend performance optimization
Release Notes (excerpt)
<details open> metal : dequant kv cache only for large batches (#27438) </details> **Website:** - <https://llama.app> **Attestations:** - <https://github.com/ggml-org/llama.cpp/attestations/42031082> **macOS/iOS:** - macOS Apple Silicon (arm64) - macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED - macOS Intel (x64) - iOS XCFramework **Linux:** - Ubuntu x64 (CPU) - Ubuntu arm64 (CPU) - Ubuntu s390x (CPU) - Ubuntu x64 (Vulkan) - Ubuntu arm64 (Vulkan) - Ubuntu x64 (ROCm 7.14)DISABLED…
Published by ggml-org on GitHub.
Read the full release notesRelated Repositories
:electron: Build cross-platform desktop apps with JavaScript, HTML, and CSS
Godot Engine – Multi-platform 2D and 3D game engine
ClickHouse® is a real-time analytics database management system
Self-hosted game stream host for Moonlight.
DuckDB is a high-performance analytical database system