b9828
ggml-org/llama.cppb9828Jun 27, 2026by github-actions[bot]same day as b9827
What changed since b9827
OpenCL backend gets a reworked Flash Attention with quantized (q4_0/q8_0) support and prefill prepass kernels; no changes to other backends.
New features
- OpenCL Flash Attention kernels reworked for f16 and f32 with a per-tile tuning table that can be overridden.
- OpenCL Flash Attention now supports q4_0 and q8_0 quantized KV data, plus matching dequant kernels.
- Prefill prepass kernels pad KV/mask tiles and classify tiles as masked, mixed, or unmasked so the main kernel can skip work.
- Added OpenCL set_rows conversion from f32 to q8_0/q4_0.
Improvements
- OpenCL attention performance improved through tile classification that skips fully-masked tiles and avoids mask lookups for unmasked ones.
- q4_0 MoE tensors on OpenCL are now stored SOA for better layout efficiency.
- Host-side OpenCL code was refactored and wired up for the new Flash Attention path with clearer comments.
Bug fixes
- Fixed infinity handling when the OpenCL compiler flag -cl-finite-math-only is used.
Upgrade notes
This release only touches the OpenCL backend, so CPU, CUDA, Metal, and other users see no behavioral change and can upgrade without config edits. OpenCL users running quantized KV caches or Flash Attention should see speedups and may optionally tune the new FA tile table override for their hardware. No migrations or deprecations are involved.
Analyzed Nov 1, 2026
Upgrading?
Release notes summary
Significant performance and feature updates to the OpenCL backend including a major rework of Flash Attention kernels supporting various quantizations and precisions.
Key Highlights
- Reworked Flash Attention kernels for f16/f32
- Added prefill prepass kernels
- Added support for q4_0 and q8_0 quantizations
- Added set_rows functionality for f32 to q8_0/q4_0
Release Notes (excerpt)
<details open> opencl: flash attention improvement (#25069) * opencl: rework FA kernel for f16 and f32 * opencl: flash-attention prefill prepass kernels - flash_attn_kv_pad_f16 pads the tail KV tile to a BLOCK_N multiple - flash_attn_mask_pad_f16 pads the matching mask tile - flash_attn_blk_f16 classifies each KV tile per query block as fully masked / mixed / fully unmasked, so the main kernel can skip fully-masked tiles…
Published by ggml-org on GitHub.
Read the full release notesRelated Repositories
:electron: Build cross-platform desktop apps with JavaScript, HTML, and CSS
Godot Engine – Multi-platform 2D and 3D game engine
ClickHouse® is a real-time analytics database management system
Self-hosted game stream host for Moonlight.
DuckDB is a high-performance analytical database system