b10164
ggml-org/llama.cppb10164Jul 28, 2026by github-actions[bot]same day as b10159
What changed since b10159
Adds a CUDA chunked SSD matmul kernel to speed up Mamba-2 prefill, on top of the prior Metal FWHT kernel.
New features
- New chunked SSD (state-space duality) matrix multiplication path in the CUDA backend for Mamba-2 prefill acceleration.
- M-matrix materialization is fused into the pre_matmul kernel, with an accompanying test enabled.
Improvements
- SSD prepare_dt kernel gains better memory coalescing and merges the B_weighted and C_scaled steps for efficiency.
- s0_stride_seq promoted to int64_t to keep correctness on large sequence strides.
- CUDA SSD code path made portable across CUDA, HIP, MUSA and MSVC toolchains.
- Test data limits de-hardcoded so SSD tests are less brittle.
Bug fixes
- Fixed a shared-memory read/write race in the prepare_dt fallback scan loop.
- Corrected stride handling that could produce wrong results in the SSD kernel.
Upgrade notes
This is a source/build-level bump: rebuild to pick up the new CUDA kernel, and CUDA/HIP/MUSA users running Mamba-2 models should see faster prefill with no config changes. The Metal FWHT kernel from the previous build remains unchanged, so Apple Silicon behavior is unaffected. No API or model-format changes are implied by these notes.
Analyzed Oct 31, 2026
Upgrading?
Release notes summary
Accelerates Mamba-2 prefilling operations using a chunked SSD (Sliding Window) matrix multiplication implementation in CUDA.
Key Highlights
- Mamba-2 prefill acceleration via CUDA SSD matmul
- Performance optimizations for SSM (State Space Model) kernels
Release Notes (excerpt)
<details open> ggml-cuda: add chunked SSD matmul for Mamba-2 prefill acceleration (#22675) * ggml-cuda: add chunked SSD matmul for Mamba-2 prefill acceleration * cuda: added SSD CICD fixes for CUDA / HIP / MUSA / MSVC. * ggml-cuda: review comments fixed. * ggml-cuda: Fuse M matrix materialization into pre_matmul kernel and enabled test. * ggml-cuda: test updates and fixes * ggml-cuda: test updates to remove hardcoding of tensor initialise data limits.…
Published by ggml-org on GitHub.
Read the full release notesRelated Repositories
:electron: Build cross-platform desktop apps with JavaScript, HTML, and CSS
Godot Engine – Multi-platform 2D and 3D game engine
ClickHouse® is a real-time analytics database management system
Self-hosted game stream host for Moonlight.
DuckDB is a high-performance analytical database system