b10208
ggml-org/llama.cppb10208Jul 31, 2026by github-actions[bot]same day as b10207
What changed since b10207
Adds oneMKL GEMM-based Flash Attention for SYCL, roughly doubling XMX prompt-processing throughput while fixing F16-cache multi-turn corruption.
New features
- SYCL backend gains a oneMKL GEMM Flash Attention path for XMX-accelerated prompt processing on Intel Arc/Battlemage hardware.
- New environment variables to control and debug the MKL Flash Attention path, including a disable switch for A/B testing and diagnostic/debug output modes.
- Documentation for the new SYCL Flash Attention options added to the backend docs.
Improvements
- Prompt processing reported up to ~2x faster on tested Gemma-4 and Qwen models with q8_0 KV cache at 32K context.
- Removed redundant SYCL stream synchronization calls, keeping only the barriers required for the oneMKL GEMM handshake.
- Cleaned out unused stride fields, diagnostic clutter, and dead hex-dump code in the SYCL Flash Attention implementation.
Bug fixes
- Corrected the normalize kernel to write head outputs in interleaved layout, fixing corrupted attention for models other than the one that masked the bug via sparse GQA heads.
- Fixed multi-turn conversation corruption observed with F16 KV caches.
- Fixed stride handling in the SYCL Flash Attention path.
Upgrade notes
No config or API changes are required; the new SYCL Flash Attention path is used automatically on supported Intel hardware. If you see numerical differences or regressions on SYCL, set the provided disable environment variable to fall back to the previous TILE implementation, and use the debug/diagnostic variables when reporting issues. Non-SYCL backends are unaffected by this release.
Analyzed Nov 1, 2026
Upgrading?
Release notes summary
This major release introduces oneMKL GEMM-based Flash Attention for SYCL, delivering up to 2x performance improvements on XMX-accelerated hardware (Intel Arc/Battlemage). It includes fixes for multi-turn conversation corruption and stride handling.
Key Highlights
- Adds oneMKL GEMM Flash Attention for XMX-accelerated prompt processing
- Performance improvements up to 2x faster for models like Gemma-4 and Qwen
- Fixes multi-turn corruption issues with F16 caches
- Adds environment variables for toggling and debugging MKL FA
Release Notes (excerpt)
<details open> SYCL: add oneMKL GEMM flash attention for XMX-accelerated prompt proc… (#25025) * SYCL: add oneMKL GEMM flash attention for XMX-accelerated prompt processing * fattn-mkl: fix interleaved dst layout in normalize kernel - Fix mkl_fa_normalize_head: use interleaved dst layout ((query * n_q_heads + head) * DV) matching TILE's flash_attn_combine_results. Previously used dense head-major layout which wrote head outputs to wrong addresses, corrupting…
Published by ggml-org on GitHub.
Read the full release notesRelated Repositories
:electron: Build cross-platform desktop apps with JavaScript, HTML, and CSS
Godot Engine – Multi-platform 2D and 3D game engine
ClickHouse® is a real-time analytics database management system
Self-hosted game stream host for Moonlight.
DuckDB is a high-performance analytical database system