b10194

ggml-org/llama.cppb10194Jul 30, 2026by github-actions[bot]same day as b10192

What changed since b10192

low risk

CUDA backend gains a transpose-free gemv path for 1xK weight matrices, on top of the prior ggml sync.

New features

  • CUDA backend can now skip the transpose step for gemv when weight matrices are shaped 1xK, routing through mat_mul_vec_f.

Improvements

  • Faster matrix-vector multiplication on CUDA for the narrow 1xK weight shape.
  • Builds on the previous release's library-wide ggml synchronization, which carried no user-facing changes of its own.

Upgrade notes

No configuration or API changes are required; this is a drop-in rebuild. The benefit only applies to CUDA users whose weights hit the 1xK shape, so other backends and shapes should see identical behavior. Validate CUDA output correctness if you rely on unusual weight shapes, since the code path selection changed.

Analyzed Oct 31, 2026

Upgrading?

Release notes summary

Optimizes CUDA performance by allowing a transpose-free computation for matrices shaped 1xK using the mat_mul_vec_f function.

Key Highlights

  • CUDA transpose-free gemmv optimization
  • Leverages mat_mul_vec_f for 1xK weights
  • Performance improvement for specific matrix shapes

Release Notes (excerpt)

<details open> ggml-cuda: Allow transpose-free gemmv computation (#26171) When matrix's weights are shaped 1xK is leverage a transpose-free computation to use mat_mul_vec_f. </details> **Website:** - <https://llama.app> **macOS/iOS:** - macOS Apple Silicon (arm64) - macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED - macOS Intel (x64) - iOS XCFramework **Linux:** - Ubuntu x64 (CPU) - Ubuntu arm64 (CPU) - Ubuntu s390x (CPU) - Ubuntu x64 (Vulkan) - Ubuntu arm64 (Vulkan)…

Published by ggml-org on GitHub.

Read the full release notes

Related Repositories