b10330
ggml-org/llama.cppb10330Aug 8, 2026by github-actions[bot]same day as b10329
What changed since b10329
b10330 adds a CUDA kernel fusion for rms_norm + mul + RoPE, cutting memory traffic; no server/UI changes beyond b10329.
New features
- CUDA backend now fuses rms_norm, mul, and RoPE (optionally with view and set_rows) into a single kernel launch.
- Added test coverage for broadcast weight cases in the fused rms_norm_mul_rope path.
Improvements
- Reduced memory traffic and kernel launch overhead on CUDA by combining normalization, scaling, and rotary embedding steps.
- Memory range checks were added before applying the rms_norm/RoPE and rope/set_rows fusions to guard correctness.
Upgrade notes
This is a CUDA-only performance change with no API, config, or CLI changes; a drop-in binary swap is sufficient. Users on non-CUDA backends will see no behavioral difference. Validate output quality on models using fused norm+RoPE patterns, since the fusion is guarded by new memory-range checks.
Analyzed Nov 1, 2026
Upgrading?
Release notes summary
Optimizes CUDA performance by fusing RMS normalization, multiplication, and RoPE operations into a single kernel launch.
Key Highlights
- Implements kernel fusion for `rms_norm + mul + rope (+ view + set_rows)` to reduce memory traffic.
- Includes memory range checks to ensure correctness of the fused operations.
- Adds tests for broadcast weight cases in the rms_norm_mul_rope fusion.
Release Notes (excerpt)
<details open> CUDA: fuse rms_norm + mul + rope (+ view + set_rows) (#26767) * CUDA: fuse rms_norm + mul + rope (+ view + set_rows) * tests: add broadcast weight case to rms_norm_mul_rope * CUDA: check memory ranges before rms_norm rope fusion * CUDA: check memory ranges in rope set_rows fusion </details> **Website:** - <https://llama.app> **macOS/iOS:** - macOS Apple Silicon (arm64) - macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED - macOS Intel (x64) - iOS XCFramework…
Published by ggml-org on GitHub.
Read the full release notesRelated Repositories
:electron: Build cross-platform desktop apps with JavaScript, HTML, and CSS
Godot Engine – Multi-platform 2D and 3D game engine
ClickHouse® is a real-time analytics database management system
Self-hosted game stream host for Moonlight.
DuckDB is a high-performance analytical database system