b9664
ggml-org/llama.cppb9664Jun 16, 2026by github-actions[bot]same day as b9663
What changed since b9663
SYCL backend gains reordered-weight MoE matmul support for Q4_K, Q5_K, and Q6_K expert tensors.
New features
- SYCL fused MoE MUL_MAT_ID now handles reordered Q4_K, Q5_K, and Q6_K expert weights.
- Added reordered DMMV coverage for Q5_K quantized tensors on SYCL.
Improvements
- Unsupported 3D reorder cases on SYCL now fall back to a safe path instead of aborting the process.
- Broader K-quant coverage for SYCL MoE inference, reducing the chance of hitting an unimplemented path.
Bug fixes
- Eliminated hard aborts on SYCL when a 3D reorder layout is not supported, replacing them with graceful fallback.
Upgrade notes
This is a single-commit increment affecting only the SYCL backend; CPU, CUDA, Metal, and Vulkan behavior is unchanged. No configuration or model-format changes are required — just rebuild or pull the new SYCL binaries to benefit from K-quant MoE acceleration. Users running MoE models with Q4_K/Q5_K/Q6_K experts on Intel GPUs are the primary beneficiaries.
Analyzed Nov 1, 2026
Upgrading?
Release notes summary
Enhances SYCL support for Mixture of Experts (MoE) models by extending reordered-weight handling to Q4_K, Q5_K, and Q6_K quantization formats during matrix multiplication (MUL_MAT_ID) and DMMV operations.
Key Highlights
- SYCL support for reordered Q4_K, Q5_K, and Q6_K MoE MUL_MAT_ID
- Added Q5_K reordered DMMV coverage
- Improved error handling: unsupported 3D reorder cases now fall back instead of aborting
Release Notes (excerpt)
<details open> sycl: support reordered Q4_K/Q5_K/Q6_K MoE MUL_MAT_ID (#24452) * sycl: support reordered Q4_K and Q5_K MoE MUL_MAT_ID Extend reordered-weight handling to fused MoE MUL_MAT_ID for Q4_K and Q5_K expert tensors and add Q5_K reordered DMMV coverage. Unsupported 3D reorder cases now fall back instead of aborting. * sycl: extend MoE reorder to Q6_K mul_mat_id </details> **macOS/iOS:** - macOS Apple Silicon (arm64) - macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED…
Published by ggml-org on GitHub.
Read the full release notesRelated Repositories
:electron: Build cross-platform desktop apps with JavaScript, HTML, and CSS
Godot Engine – Multi-platform 2D and 3D game engine
ClickHouse® is a real-time analytics database management system
Self-hosted game stream host for Moonlight.
DuckDB is a high-performance analytical database system