b9901

ggml-org/llama.cppb9901Jul 7, 2026by github-actions[bot]same day as b9899

What changed since b9899

low risk

SYCL DMMV path tuned for better GPU utilization, plus fixes to the reorder gate and buffer tensor type handling.

New features

  • SYCL dequantize-mul-mat-vec path now runs with K_QUANTS_PER_ITERATION=1, enabling the reordered DMMV kernel on supported quantization types.

Improvements

  • GPU utilization and token-generation throughput improved on Intel B70 hardware for dequantize-heavy mat-vec workloads.
  • The reorder-DMMV support check now correctly advertises the quantization types actually implemented by the kernel.
  • SYCL buffer initialization now allocates the extra per-tensor field for the missing types, so the reorder path actually engages.
  • QK_5 kernel code now uses the generic WARP_SIZE constant instead of the QK-specific one.

Bug fixes

  • Fixed a gating bug where supported types were not reported by ggml_sycl_supports_reorder_dmmv, silently disabling the reordered path.
  • Fixed missing type entries in ggml_backend_sycl_buffer_init_tensor that prevented the reorder optimization from taking effect.

Upgrade notes

This is a SYCL-backend-only performance change; no CLI, API, or model-format changes are involved. Rebuild the SYCL backend to pick up the new kernel constants and buffer allocation behavior. Users on non-SYCL backends should see no functional difference.

Analyzed Oct 30, 2026

Upgrading?

Release notes summary

Optimizes SYCL DMMV (Dequantize-Mul-Mat-Vec) path by setting K_QUANTS_PER_ITERATION to 1, significantly improving GPU utilization and performance on B70 hardware.

Key Highlights

  • SYCL DMMV optimization with K_QUANTS_PER_ITERATION=1
  • Improved GPU utilization on B70 hardware
  • Better performance for dequantize operations

Release Notes (excerpt)

<details open> sycl : set K_QUANTS_PER_ITERATION to 1 on DMMV path (#25063) * sycl: add supported types to ggml_sycl_supports_reorder_dmmv The reordered feature is implemented in ggml_sycl_op_dequantize_mul_mat_vec, but gated by ggml_sycl_supports_reorder_dmmv. This commit fixes the gate. Signed-off-by: Todd Malsbary <todd.malsbary@intel.com> * sycl: set K_QUANTS_PER_ITERATION=1 to improve utilization When combined with opening the reorder gate, this improves GPU…

Published by ggml-org on GitHub.

Read the full release notes

Related Repositories