b9664

ggml-org/llama.cppb9664Jun 16, 2026by github-actions[bot]same day as b9663

What changed since b9663

low risk

SYCL backend gains reordered-weight MoE matmul support for Q4_K, Q5_K, and Q6_K expert tensors.

New features

  • SYCL fused MoE MUL_MAT_ID now handles reordered Q4_K, Q5_K, and Q6_K expert weights.
  • Added reordered DMMV coverage for Q5_K quantized tensors on SYCL.

Improvements

  • Unsupported 3D reorder cases on SYCL now fall back to a safe path instead of aborting the process.
  • Broader K-quant coverage for SYCL MoE inference, reducing the chance of hitting an unimplemented path.

Bug fixes

  • Eliminated hard aborts on SYCL when a 3D reorder layout is not supported, replacing them with graceful fallback.

Upgrade notes

This is a single-commit increment affecting only the SYCL backend; CPU, CUDA, Metal, and Vulkan behavior is unchanged. No configuration or model-format changes are required — just rebuild or pull the new SYCL binaries to benefit from K-quant MoE acceleration. Users running MoE models with Q4_K/Q5_K/Q6_K experts on Intel GPUs are the primary beneficiaries.

Analyzed Nov 1, 2026

Upgrading?

Release notes summary

Enhances SYCL support for Mixture of Experts (MoE) models by extending reordered-weight handling to Q4_K, Q5_K, and Q6_K quantization formats during matrix multiplication (MUL_MAT_ID) and DMMV operations.

Key Highlights

  • SYCL support for reordered Q4_K, Q5_K, and Q6_K MoE MUL_MAT_ID
  • Added Q5_K reordered DMMV coverage
  • Improved error handling: unsupported 3D reorder cases now fall back instead of aborting

Release Notes (excerpt)

<details open> sycl: support reordered Q4_K/Q5_K/Q6_K MoE MUL_MAT_ID (#24452) * sycl: support reordered Q4_K and Q5_K MoE MUL_MAT_ID Extend reordered-weight handling to fused MoE MUL_MAT_ID for Q4_K and Q5_K expert tensors and add Q5_K reordered DMMV coverage. Unsupported 3D reorder cases now fall back instead of aborting. * sycl: extend MoE reorder to Q6_K mul_mat_id </details> **macOS/iOS:** - macOS Apple Silicon (arm64) - macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED…

Published by ggml-org on GitHub.

Read the full release notes

Related Repositories