b9901
ggml-org/llama.cppb9901Jul 7, 2026by github-actions[bot]same day as b9899
What changed since b9899
SYCL DMMV path tuned for better GPU utilization, plus fixes to the reorder gate and buffer tensor type handling.
New features
- SYCL dequantize-mul-mat-vec path now runs with K_QUANTS_PER_ITERATION=1, enabling the reordered DMMV kernel on supported quantization types.
Improvements
- GPU utilization and token-generation throughput improved on Intel B70 hardware for dequantize-heavy mat-vec workloads.
- The reorder-DMMV support check now correctly advertises the quantization types actually implemented by the kernel.
- SYCL buffer initialization now allocates the extra per-tensor field for the missing types, so the reorder path actually engages.
- QK_5 kernel code now uses the generic WARP_SIZE constant instead of the QK-specific one.
Bug fixes
- Fixed a gating bug where supported types were not reported by ggml_sycl_supports_reorder_dmmv, silently disabling the reordered path.
- Fixed missing type entries in ggml_backend_sycl_buffer_init_tensor that prevented the reorder optimization from taking effect.
Upgrade notes
This is a SYCL-backend-only performance change; no CLI, API, or model-format changes are involved. Rebuild the SYCL backend to pick up the new kernel constants and buffer allocation behavior. Users on non-SYCL backends should see no functional difference.
Analyzed Oct 30, 2026
Upgrading?
Release notes summary
Optimizes SYCL DMMV (Dequantize-Mul-Mat-Vec) path by setting K_QUANTS_PER_ITERATION to 1, significantly improving GPU utilization and performance on B70 hardware.
Key Highlights
- SYCL DMMV optimization with K_QUANTS_PER_ITERATION=1
- Improved GPU utilization on B70 hardware
- Better performance for dequantize operations
Release Notes (excerpt)
<details open> sycl : set K_QUANTS_PER_ITERATION to 1 on DMMV path (#25063) * sycl: add supported types to ggml_sycl_supports_reorder_dmmv The reordered feature is implemented in ggml_sycl_op_dequantize_mul_mat_vec, but gated by ggml_sycl_supports_reorder_dmmv. This commit fixes the gate. Signed-off-by: Todd Malsbary <todd.malsbary@intel.com> * sycl: set K_QUANTS_PER_ITERATION=1 to improve utilization When combined with opening the reorder gate, this improves GPU…
Published by ggml-org on GitHub.
Read the full release notesRelated Repositories
:electron: Build cross-platform desktop apps with JavaScript, HTML, and CSS
Godot Engine – Multi-platform 2D and 3D game engine
ClickHouse® is a real-time analytics database management system
Self-hosted game stream host for Moonlight.
DuckDB is a high-performance analytical database system