b10321
ggml-org/llama.cppb10321Aug 7, 2026by github-actions[bot]same day as b10319
What changed since b10319
b10321 adds a Metal backend correctness fix for NORM/RMS_NORM on rows with partial SIMD groups, on top of b10319's mtmd fix.
Improvements
- Metal NORM/RMS_NORM threadgroup size now rounds row length up to a whole number of simdgroups instead of clamping, avoiding dropped partial sums while keeping threadgroups small.
- Added regression test coverage for row lengths 33, 132, and 260 to guard the Metal norm path.
Bug fixes
- Fixed incorrect mean and variance in Metal NORM and RMS_NORM when a row leaves a partial simdgroup, which previously produced too-small row sums.
- Both GGML_OP_NORM and GGML_OP_RMS_NORM are corrected since they share the same Metal dispatch routine.
Upgrade notes
This is a drop-in patch release with no config or API changes. Metal/Apple Silicon users should upgrade, since the older build silently returns wrong normalization results for affected row lengths; mainstream LLM hidden sizes were not impacted. Note that b10319's mtmd longest_edge fix is already included in this build.
Analyzed Oct 30, 2026
Upgrading?
Release notes summary
Fixes a critical correctness bug in the Metal backend where NORM and RMS_NORM operations produced incorrect results for rows with partial SIMD groups.
Key Highlights
- Fixed `ggml_metal_op_norm` threadgroup sizing to handle rows with partial SIMD groups correctly.
- Resolves incorrect mean and variance calculations for specific row lengths (e.g., 33).
- Added test cases for row lengths 33, 132, and 260 to prevent regression.
Release Notes (excerpt)
<details open> metal : fix NORM/RMS_NORM for row lengths that leave a partial simdgroup (#26708) ggml_metal_op_norm sized the threadgroup with `nth = std::min(nth, args.ne00_t)`, which can leave nth not a multiple of the simdgroup size. The kernels finish their row reduction with a cross-simdgroup step where each lane of the last simdgroup reads one per-simdgroup partial sum out of shmem_f32: if (tiisg == 0) { shmem_f32[sgitg] = sumf; }…
Published by ggml-org on GitHub.
Read the full release notesRelated Repositories
:electron: Build cross-platform desktop apps with JavaScript, HTML, and CSS
Godot Engine – Multi-platform 2D and 3D game engine
ClickHouse® is a real-time analytics database management system
Self-hosted game stream host for Moonlight.
DuckDB is a high-performance analytical database system