b10517

ggml-org/llama.cppb10517Aug 20, 2026by github-actions[bot]same day as b10516

What changed since b10516

low risk

Vulkan Flash Attention now dequantizes q8_0 KV cache once in cooperative matrix paths, with safer fallbacks.

New features

  • Vulkan Flash Attention dequantizes q8_0 KV cache a single time in the coopmat1 path instead of repeatedly.
  • New contiguous KV-cache allocation tests cover quantized K/V Flash Attention scenarios.

Improvements

  • Flash Attention scratch buffer requests exceeding maxStorageBufferRange now fall back gracefully rather than aborting.
  • The FA dequant path is skipped on coopmat2 and excluded on Intel Xe1, keeping those devices on the established code path.
  • Permutation and KV-cache layout checks were tightened so the optimized path only runs when preconditions hold.
  • prealloc_x_need_sync is now set after the FA dispatch, correcting synchronization ordering.

Bug fixes

  • Prevents hard aborts on Vulkan devices with limited storage buffer range when FA scratch memory is too large.

Upgrade notes

No configuration or API changes are required; this is a drop-in build bump. Vulkan users running quantized (q8_0) KV cache with Flash Attention should see reduced dequantization overhead, while coopmat2 and Intel Xe1 devices intentionally stay on the prior path. Validate output quality on Vulkan if you rely on quantized KV cache, since the optimized path is gated by new layout checks.

Analyzed Oct 24, 2026

Upgrading?

Release notes summary

Optimizes Flash Attention in the Vulkan backend by dequantizing KV cache only once in cooperative matrix operations and handling edge cases like FA scratch buffer limits.

Key Highlights

  • Optimized KV dequantization in Vulkan Flash Attention
  • Added fallback for FA scratch buffer exceeding maxStorageBufferRange
  • Added tests for contiguous KV allocation

Release Notes (excerpt)

<details open> vulkan : dequant q8_0 KV once in coopmat1 (#25494) * vulkan : dequant q8_0 KV once in coopmat1 Assisted-by: Claude (Opus 4.8) * vulkan : fall back instead of aborting when FA scratch exceeds maxStorageBufferRange * vulkan : require KV-cache layout in FA dequant path Assisted-by: Claude (Opus 4.8) * vulkan : skip FA dequant path on coopmat2 Assisted-by: Claude (Opus 4.8) * tests : add contiguously-allocated quant K/V FA tests Assisted-by: Claude (Opus 4.8)…

Published by ggml-org on GitHub.

Read the full release notes

Related Repositories