b10181
ggml-org/llama.cppb10181Jul 29, 2026by github-actions[bot]same day as b10180
What changed since b10180
CUDA backend now disables MMQ on GPUs with under 48 KiB shared memory, preventing prefill crashes by falling back to BLAS.
New features
- Automatic BLAS fallback path for CUDA devices that cannot fit any supported MMQ tile, including some MUSA QY1 hardware reporting 28 KiB shared memory.
Improvements
- MMQ selection now checks per-block shared memory budget rather than quantization type alone, aligning with the 48 KiB minimum assumed by current MMQ configurations.
Bug fixes
- Fixed a GGML_ABORT crash during batched prefill (batch > 1) on devices with less than 48 KiB shared memory, reproduced on a Moore Threads MTT S70 with an RWKV-7 Q8_0 model.
Upgrade notes
No configuration or API changes are required. Users on low-shared-memory CUDA/MUSA GPUs will silently switch to the BLAS path for quantized matmul, which restores prefill functionality at roughly the same throughput as a forced-cuBLAS build. Token generation and FP16/Q4_K_M paths are unaffected.
Analyzed Nov 1, 2026
Upgrading?
Release notes summary
Fixes a crash on GPUs with less than 48 KiB of shared memory by disabling Multi-Matrix Quantization (MMQ) and falling back to BLAS.
Key Highlights
- Fix for MMQ crash on low shared memory devices
- Fallback to BLAS path for compatible hardware
- Support for MUSA QY1 devices
Release Notes (excerpt)
<details open> ggml-cuda : disable MMQ on devices with less than 48 KiB shared memory (#26141) ggml_cuda_should_use_mmq() selects MMQ purely from the quantization type. The current MMQ configurations are designed and maintained against a minimum of 48 KiB per-block shared memory, the limit provided by NVIDIA Pascal GPUs and later. On devices that report less, no supported MMQ tile fits and mul_mat_q_switch_J() aborts when every tile size exceeds the device's per-block shared memory budget.…
Published by ggml-org on GitHub.
Read the full release notesRelated Repositories
:electron: Build cross-platform desktop apps with JavaScript, HTML, and CSS
Godot Engine – Multi-platform 2D and 3D game engine
ClickHouse® is a real-time analytics database management system
Self-hosted game stream host for Moonlight.
DuckDB is a high-performance analytical database system