b10517
ggml-org/llama.cppb10517Aug 20, 2026by github-actions[bot]same day as b10516
What changed since b10516
Vulkan Flash Attention now dequantizes q8_0 KV cache once in cooperative matrix paths, with safer fallbacks.
New features
- Vulkan Flash Attention dequantizes q8_0 KV cache a single time in the coopmat1 path instead of repeatedly.
- New contiguous KV-cache allocation tests cover quantized K/V Flash Attention scenarios.
Improvements
- Flash Attention scratch buffer requests exceeding maxStorageBufferRange now fall back gracefully rather than aborting.
- The FA dequant path is skipped on coopmat2 and excluded on Intel Xe1, keeping those devices on the established code path.
- Permutation and KV-cache layout checks were tightened so the optimized path only runs when preconditions hold.
- prealloc_x_need_sync is now set after the FA dispatch, correcting synchronization ordering.
Bug fixes
- Prevents hard aborts on Vulkan devices with limited storage buffer range when FA scratch memory is too large.
Upgrade notes
No configuration or API changes are required; this is a drop-in build bump. Vulkan users running quantized (q8_0) KV cache with Flash Attention should see reduced dequantization overhead, while coopmat2 and Intel Xe1 devices intentionally stay on the prior path. Validate output quality on Vulkan if you rely on quantized KV cache, since the optimized path is gated by new layout checks.
Analyzed Oct 24, 2026
Upgrading?
Release notes summary
Optimizes Flash Attention in the Vulkan backend by dequantizing KV cache only once in cooperative matrix operations and handling edge cases like FA scratch buffer limits.
Key Highlights
- Optimized KV dequantization in Vulkan Flash Attention
- Added fallback for FA scratch buffer exceeding maxStorageBufferRange
- Added tests for contiguous KV allocation
Release Notes (excerpt)
<details open> vulkan : dequant q8_0 KV once in coopmat1 (#25494) * vulkan : dequant q8_0 KV once in coopmat1 Assisted-by: Claude (Opus 4.8) * vulkan : fall back instead of aborting when FA scratch exceeds maxStorageBufferRange * vulkan : require KV-cache layout in FA dequant path Assisted-by: Claude (Opus 4.8) * vulkan : skip FA dequant path on coopmat2 Assisted-by: Claude (Opus 4.8) * tests : add contiguously-allocated quant K/V FA tests Assisted-by: Claude (Opus 4.8)…
Published by ggml-org on GitHub.
Read the full release notesRelated Repositories
:electron: Build cross-platform desktop apps with JavaScript, HTML, and CSS
Godot Engine – Multi-platform 2D and 3D game engine
ClickHouse® is a real-time analytics database management system
Self-hosted game stream host for Moonlight.
DuckDB is a high-performance analytical database system