b9828

ggml-org/llama.cppb9828Jun 27, 2026by github-actions[bot]same day as b9827

What changed since b9827

medium risk

OpenCL backend gets a reworked Flash Attention with quantized (q4_0/q8_0) support and prefill prepass kernels; no changes to other backends.

New features

  • OpenCL Flash Attention kernels reworked for f16 and f32 with a per-tile tuning table that can be overridden.
  • OpenCL Flash Attention now supports q4_0 and q8_0 quantized KV data, plus matching dequant kernels.
  • Prefill prepass kernels pad KV/mask tiles and classify tiles as masked, mixed, or unmasked so the main kernel can skip work.
  • Added OpenCL set_rows conversion from f32 to q8_0/q4_0.

Improvements

  • OpenCL attention performance improved through tile classification that skips fully-masked tiles and avoids mask lookups for unmasked ones.
  • q4_0 MoE tensors on OpenCL are now stored SOA for better layout efficiency.
  • Host-side OpenCL code was refactored and wired up for the new Flash Attention path with clearer comments.

Bug fixes

  • Fixed infinity handling when the OpenCL compiler flag -cl-finite-math-only is used.

Upgrade notes

This release only touches the OpenCL backend, so CPU, CUDA, Metal, and other users see no behavioral change and can upgrade without config edits. OpenCL users running quantized KV caches or Flash Attention should see speedups and may optionally tune the new FA tile table override for their hardware. No migrations or deprecations are involved.

Analyzed Nov 1, 2026

Upgrading?

Release notes summary

Significant performance and feature updates to the OpenCL backend including a major rework of Flash Attention kernels supporting various quantizations and precisions.

Key Highlights

  • Reworked Flash Attention kernels for f16/f32
  • Added prefill prepass kernels
  • Added support for q4_0 and q8_0 quantizations
  • Added set_rows functionality for f32 to q8_0/q4_0

Release Notes (excerpt)

<details open> opencl: flash attention improvement (#25069) * opencl: rework FA kernel for f16 and f32 * opencl: flash-attention prefill prepass kernels - flash_attn_kv_pad_f16 pads the tail KV tile to a BLOCK_N multiple - flash_attn_mask_pad_f16 pads the matching mask tile - flash_attn_blk_f16 classifies each KV tile per query block as fully masked / mixed / fully unmasked, so the main kernel can skip fully-masked tiles…

Published by ggml-org on GitHub.

Read the full release notes

Related Repositories