b9767
ggml-org/llama.cppb9767Jun 23, 2026by github-actions[bot]same day as b9763
What changed since b9763
WebGPU inference for small batches now uses a mat-vec path, speeding up MTP decoding; no API changes since b9763.
Improvements
- WebGPU backend now routes small-batch decoding through a matrix-vector path, improving multi-token prediction (MTP) throughput.
- Added a barrier inside the NUM_COLS loop of the mul-mat-vec WebGPU kernel to keep the new path correct.
- Prebuilt binaries refreshed across macOS, Linux, and Windows backends to include the WebGPU change.
Upgrade notes
This is a drop-in update from b9763 with no config, API, or model-format changes. Only WebGPU users doing small-batch or speculative/MTP decoding will see a measurable difference; other backends are effectively unchanged. Swap in the new binaries if you rely on WebGPU.
Analyzed Nov 1, 2026
Upgrading?
Release notes summary
This release optimizes WebGPU inference for small batches by utilizing a matrix-vector path. It includes updated binaries for supported platforms and backends.
Key Highlights
- Improve MTP inference for small batches in WebGPU
- macOS/iOS binaries (Apple Silicon, Intel, XCFramework)
- Linux binaries (CPU, Vulkan, ROCm, OpenVINO, SYCL)
- Windows binaries (CPU, CUDA 12/13, Vulkan, OpenVINO, SYCL, HIP)
New Features
- Improve MTP inference for small batches in WebGPU
Release Notes (excerpt)
<details open> ggml-webgpu: improve MTP inference by using mat-vec path for small batches (#24811) * ggml-webgpu: improve small batches decoding * Add barrier to the NUM_COLS loop in mul-mat-vec </details> **macOS/iOS:** - macOS Apple Silicon (arm64) - macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED - macOS Intel (x64) - iOS XCFramework **Linux:** - Ubuntu x64 (CPU) - Ubuntu arm64 (CPU) - Ubuntu s390x (CPU) - Ubuntu x64 (Vulkan) - Ubuntu arm64 (Vulkan) - Ubuntu x64 (ROCm 7.2)…
Published by ggml-org on GitHub.
Read the full release notesRelated Repositories
:electron: Build cross-platform desktop apps with JavaScript, HTML, and CSS
Godot Engine – Multi-platform 2D and 3D game engine
ClickHouse® is a real-time analytics database management system
Self-hosted game stream host for Moonlight.
DuckDB is a high-performance analytical database system