b10181

ggml-org/llama.cppb10181Jul 29, 2026by github-actions[bot]same day as b10180

What changed since b10180

low risk

CUDA backend now disables MMQ on GPUs with under 48 KiB shared memory, preventing prefill crashes by falling back to BLAS.

New features

  • Automatic BLAS fallback path for CUDA devices that cannot fit any supported MMQ tile, including some MUSA QY1 hardware reporting 28 KiB shared memory.

Improvements

  • MMQ selection now checks per-block shared memory budget rather than quantization type alone, aligning with the 48 KiB minimum assumed by current MMQ configurations.

Bug fixes

  • Fixed a GGML_ABORT crash during batched prefill (batch > 1) on devices with less than 48 KiB shared memory, reproduced on a Moore Threads MTT S70 with an RWKV-7 Q8_0 model.

Upgrade notes

No configuration or API changes are required. Users on low-shared-memory CUDA/MUSA GPUs will silently switch to the BLAS path for quantized matmul, which restores prefill functionality at roughly the same throughput as a forced-cuBLAS build. Token generation and FP16/Q4_K_M paths are unaffected.

Analyzed Nov 1, 2026

Upgrading?

Release notes summary

Fixes a crash on GPUs with less than 48 KiB of shared memory by disabling Multi-Matrix Quantization (MMQ) and falling back to BLAS.

Key Highlights

  • Fix for MMQ crash on low shared memory devices
  • Fallback to BLAS path for compatible hardware
  • Support for MUSA QY1 devices

Release Notes (excerpt)

<details open> ggml-cuda : disable MMQ on devices with less than 48 KiB shared memory (#26141) ggml_cuda_should_use_mmq() selects MMQ purely from the quantization type. The current MMQ configurations are designed and maintained against a minimum of 48 KiB per-block shared memory, the limit provided by NVIDIA Pascal GPUs and later. On devices that report less, no supported MMQ tile fits and mul_mat_q_switch_J() aborts when every tile size exceeds the device's per-block shared memory budget.…

Published by ggml-org on GitHub.

Read the full release notes

Related Repositories