b10164

ggml-org/llama.cppb10164Jul 28, 2026by github-actions[bot]same day as b10159

What changed since b10159

medium risk

Adds a CUDA chunked SSD matmul kernel to speed up Mamba-2 prefill, on top of the prior Metal FWHT kernel.

New features

  • New chunked SSD (state-space duality) matrix multiplication path in the CUDA backend for Mamba-2 prefill acceleration.
  • M-matrix materialization is fused into the pre_matmul kernel, with an accompanying test enabled.

Improvements

  • SSD prepare_dt kernel gains better memory coalescing and merges the B_weighted and C_scaled steps for efficiency.
  • s0_stride_seq promoted to int64_t to keep correctness on large sequence strides.
  • CUDA SSD code path made portable across CUDA, HIP, MUSA and MSVC toolchains.
  • Test data limits de-hardcoded so SSD tests are less brittle.

Bug fixes

  • Fixed a shared-memory read/write race in the prepare_dt fallback scan loop.
  • Corrected stride handling that could produce wrong results in the SSD kernel.

Upgrade notes

This is a source/build-level bump: rebuild to pick up the new CUDA kernel, and CUDA/HIP/MUSA users running Mamba-2 models should see faster prefill with no config changes. The Metal FWHT kernel from the previous build remains unchanged, so Apple Silicon behavior is unaffected. No API or model-format changes are implied by these notes.

Analyzed Oct 31, 2026

Upgrading?

Release notes summary

Accelerates Mamba-2 prefilling operations using a chunked SSD (Sliding Window) matrix multiplication implementation in CUDA.

Key Highlights

  • Mamba-2 prefill acceleration via CUDA SSD matmul
  • Performance optimizations for SSM (State Space Model) kernels

Release Notes (excerpt)

<details open> ggml-cuda: add chunked SSD matmul for Mamba-2 prefill acceleration (#22675) * ggml-cuda: add chunked SSD matmul for Mamba-2 prefill acceleration * cuda: added SSD CICD fixes for CUDA / HIP / MUSA / MSVC. * ggml-cuda: review comments fixed. * ggml-cuda: Fuse M matrix materialization into pre_matmul kernel and enabled test. * ggml-cuda: test updates and fixes * ggml-cuda: test updates to remove hardcoding of tensor initialise data limits.…

Published by ggml-org on GitHub.

Read the full release notes

Related Repositories