b10194
ggml-org/llama.cppb10194Jul 30, 2026by github-actions[bot]same day as b10192
What changed since b10192
CUDA backend gains a transpose-free gemv path for 1xK weight matrices, on top of the prior ggml sync.
New features
- CUDA backend can now skip the transpose step for gemv when weight matrices are shaped 1xK, routing through mat_mul_vec_f.
Improvements
- Faster matrix-vector multiplication on CUDA for the narrow 1xK weight shape.
- Builds on the previous release's library-wide ggml synchronization, which carried no user-facing changes of its own.
Upgrade notes
No configuration or API changes are required; this is a drop-in rebuild. The benefit only applies to CUDA users whose weights hit the 1xK shape, so other backends and shapes should see identical behavior. Validate CUDA output correctness if you rely on unusual weight shapes, since the code path selection changed.
Analyzed Oct 31, 2026
Upgrading?
Release notes summary
Optimizes CUDA performance by allowing a transpose-free computation for matrices shaped 1xK using the mat_mul_vec_f function.
Key Highlights
- CUDA transpose-free gemmv optimization
- Leverages mat_mul_vec_f for 1xK weights
- Performance improvement for specific matrix shapes
Release Notes (excerpt)
<details open> ggml-cuda: Allow transpose-free gemmv computation (#26171) When matrix's weights are shaped 1xK is leverage a transpose-free computation to use mat_mul_vec_f. </details> **Website:** - <https://llama.app> **macOS/iOS:** - macOS Apple Silicon (arm64) - macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED - macOS Intel (x64) - iOS XCFramework **Linux:** - Ubuntu x64 (CPU) - Ubuntu arm64 (CPU) - Ubuntu s390x (CPU) - Ubuntu x64 (Vulkan) - Ubuntu arm64 (Vulkan)…
Published by ggml-org on GitHub.
Read the full release notesRelated Repositories
:electron: Build cross-platform desktop apps with JavaScript, HTML, and CSS
Godot Engine – Multi-platform 2D and 3D game engine
ClickHouse® is a real-time analytics database management system
Self-hosted game stream host for Moonlight.
DuckDB is a high-performance analytical database system