b10538

ggml-org/llama.cppb10538Aug 21, 2026by github-actions[bot]same day as b10537

What changed since b10537

low risk

Metal backend now dequantizes the KV cache only for large batches, cutting overhead in macOS inference.

Improvements

  • Metal KV cache dequantization is gated to large batches, avoiding per-step dequant work on small batches.
  • Reduces backend overhead in memory-intensive Metal inference paths, improving throughput/efficiency on Apple Silicon.

Upgrade notes

This is a single-commit Metal performance tweak with no API, config, or CLI changes; a drop-in replacement for b10537. Only macOS/iOS Metal users are affected, and behavior for small batches should be unchanged apart from reduced overhead. The Windows OpenMP/LLVM packaging change landed in b10537 and is not part of this delta.

Analyzed Oct 27, 2026

Upgrading?

Release notes summary

Optimizes the macOS Metal backend by dequantizing the Key-Value (KV) cache only for large batches to reduce overhead and improve efficiency.

Key Highlights

  • Metal dequantization optimization for KV cache
  • Applied only for large batches to minimize overhead
  • Improves performance for memory-intensive operations

New Features

  • Metal backend performance optimization

Release Notes (excerpt)

<details open> metal : dequant kv cache only for large batches (#27438) </details> **Website:** - <https://llama.app> **Attestations:** - <https://github.com/ggml-org/llama.cpp/attestations/42031082> **macOS/iOS:** - macOS Apple Silicon (arm64) - macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED - macOS Intel (x64) - iOS XCFramework **Linux:** - Ubuntu x64 (CPU) - Ubuntu arm64 (CPU) - Ubuntu s390x (CPU) - Ubuntu x64 (Vulkan) - Ubuntu arm64 (Vulkan) - Ubuntu x64 (ROCm 7.14)DISABLED…

Published by ggml-org on GitHub.

Read the full release notes

Related Repositories