b9931

ggml-org/llama.cppb9931Jul 8, 2026by github-actions[bot]same day as b9930

What changed since b9930

low risk

OpenCL gains ragged-tile MoE prefill FP16 GEMM optimization that skips padded expert tiles, with new env-var controls.

New features

  • Ragged-tile MoE prefill FP16 GEMM optimization for OpenCL that skips padded expert tiles.
  • Quarter-granularity tile-skip using 8-column skip-groups in the MoE GEMM kernels.
  • New GGML_OPENCL_MOE_RAGGED_FP16 toggle (default on) to opt out of ragged FP16 MoE handling.
  • New GGML_OPENCL_MOE_RAGGED_GRAN setting ({8,16,32}) to select quarter, half, or no skip granularity.

Improvements

  • OpenCL MoE prefill throughput improves at low tokens-per-expert by avoiding mostly-padding tiles.
  • Skipping is numerically identical to the prior path, so results are unchanged.
  • Ragged MoE env var handling was moved into cl_init for cleaner initialization.

Upgrade notes

No API or config changes are required; the optimization is enabled by default on OpenCL. Set GGML_OPENCL_MOE_RAGGED_FP16=0 to disable it, or adjust GGML_OPENCL_MOE_RAGGED_GRAN to 16 or 32 if you want coarser or no tile-skipping. Non-OpenCL backends are unaffected.

Analyzed Oct 30, 2026

Upgrading?

Release notes summary

OpenCL optimizations for MoE (Mixture of Experts) prefill operations.

Key Highlights

  • Optimized ragged-tile MoE prefill FP16 GEMM by skipping padded expert tiles
  • Implemented quarter-granularity ragged MoE tile-skip (8-col skip-groups)
  • Added environment variables for configuring ragged MoE behavior

Release Notes (excerpt)

<details open> opencl: ragged-tile MoE prefill FP16 GEMM optimization (skip padded expert tiles) (#25433) * opencl: ragged-tile MoE prefill GEMM (skip padded expert tiles) The MoE prefill GEMM groups tokens into TILESIZE_N=32 per-expert tiles; at low tokens-per-expert most tiles are mostly padding. When a tile's upper 16 slots are all padding (router index 0xFFFFFFFF), skip the second dotx16_reduce8 half. Numerically identical (skipped lanes are padding). Applied to all eight *_f32_ns…

Published by ggml-org on GitHub.

Read the full release notes

Related Repositories