b9893

ggml-org/llama.cppb9893Jul 7, 2026by github-actions[bot]same day as b9892

What changed since b9892

low risk

OpenCL flash-attention decode performance improves and a DK=512 Gemma-4 decode crash is fixed.

Improvements

  • OpenCL flash-attention decode kernels now use vectorized paths for f16, q8_0, and q4_0 KV caches.
  • Non-flash-attention KQ matrix-vector kernels were tuned for OpenCL.
  • Multiquery flash-attention kernels received targeted tuning.
  • f16-KV decode work is split into smaller chunks to improve GPU occupancy.
  • MQ_GQA=8 FA kernels compile in a minimal program to avoid Adreno compiler memory exhaustion.

Bug fixes

  • Fixed a crash in DK=512 decode for Gemma-4 models.
  • Fixed inconsistent supports_op results for DK=512 decode-only programs.
  • Guarded unset block_n to prevent out-of-range reads during dispatch.
  • Removed a stray token that broke the f32_f16 flash-attention program build.

Upgrade notes

No config or migration changes are expected. OpenCL users should see faster decode and fewer Gemma-4 DK=512 crashes; note that DK=512 decode is intentionally declined on GPU and runs on CPU because it is bandwidth-bound. Rebuild or pull the new binaries to pick up the OpenCL kernel changes.

Analyzed Oct 31, 2026

Upgrading?

Release notes summary

Focuses on OpenCL flash attention decode performance optimizations with multiple kernel improvements and bug fixes for various quantization formats and model configurations.

Key Highlights

  • OpenCL flash attention decode performance optimizations
  • Fix for DK=512 decode crash in Gemma-4 models
  • Multiquery FA support improvements
  • Split f16-KV decode into finer chunks for better GPU utilization

New Features

  • OpenCL flash attention decode performance improvements
  • DK=512 decode-only program fixes
  • Multiquery FA kernel optimizations

Release Notes (excerpt)

<details open> opencl: general flash attention decode performance optimizations (#25366) * opencl: vec flash-attention decode kernels for f16/q8_0/q4_0 KV * opencl: improve non FA KQ mv kernels * opencl: tweaks for multiquery FA * opencl: some tweaks for FA q1 kernels * opencl: FA with DK=DV=512 for gemma-4 * opencl: various fixes * opencl: cleanup * opencl: fix FA decode crash for DK=512 (gemma-4) The DK=512 decode-only program does not create the f32_f16 prefill…

Published by ggml-org on GitHub.

Read the full release notes

Related Repositories