b10321

ggml-org/llama.cppb10321Aug 7, 2026by github-actions[bot]same day as b10319

What changed since b10319

low risk

b10321 adds a Metal backend correctness fix for NORM/RMS_NORM on rows with partial SIMD groups, on top of b10319's mtmd fix.

Improvements

  • Metal NORM/RMS_NORM threadgroup size now rounds row length up to a whole number of simdgroups instead of clamping, avoiding dropped partial sums while keeping threadgroups small.
  • Added regression test coverage for row lengths 33, 132, and 260 to guard the Metal norm path.

Bug fixes

  • Fixed incorrect mean and variance in Metal NORM and RMS_NORM when a row leaves a partial simdgroup, which previously produced too-small row sums.
  • Both GGML_OP_NORM and GGML_OP_RMS_NORM are corrected since they share the same Metal dispatch routine.

Upgrade notes

This is a drop-in patch release with no config or API changes. Metal/Apple Silicon users should upgrade, since the older build silently returns wrong normalization results for affected row lengths; mainstream LLM hidden sizes were not impacted. Note that b10319's mtmd longest_edge fix is already included in this build.

Analyzed Oct 30, 2026

Upgrading?

Release notes summary

Fixes a critical correctness bug in the Metal backend where NORM and RMS_NORM operations produced incorrect results for rows with partial SIMD groups.

Key Highlights

  • Fixed `ggml_metal_op_norm` threadgroup sizing to handle rows with partial SIMD groups correctly.
  • Resolves incorrect mean and variance calculations for specific row lengths (e.g., 33).
  • Added test cases for row lengths 33, 132, and 260 to prevent regression.

Release Notes (excerpt)

<details open> metal : fix NORM/RMS_NORM for row lengths that leave a partial simdgroup (#26708) ggml_metal_op_norm sized the threadgroup with `nth = std::min(nth, args.ne00_t)`, which can leave nth not a multiple of the simdgroup size. The kernels finish their row reduction with a cross-simdgroup step where each lane of the last simdgroup reads one per-simdgroup partial sum out of shmem_f32: if (tiisg == 0) { shmem_f32[sgitg] = sumf; }…

Published by ggml-org on GitHub.

Read the full release notes

Related Repositories