b9931
ggml-org/llama.cppb9931Jul 8, 2026by github-actions[bot]same day as b9930
What changed since b9930
OpenCL gains ragged-tile MoE prefill FP16 GEMM optimization that skips padded expert tiles, with new env-var controls.
New features
- Ragged-tile MoE prefill FP16 GEMM optimization for OpenCL that skips padded expert tiles.
- Quarter-granularity tile-skip using 8-column skip-groups in the MoE GEMM kernels.
- New GGML_OPENCL_MOE_RAGGED_FP16 toggle (default on) to opt out of ragged FP16 MoE handling.
- New GGML_OPENCL_MOE_RAGGED_GRAN setting ({8,16,32}) to select quarter, half, or no skip granularity.
Improvements
- OpenCL MoE prefill throughput improves at low tokens-per-expert by avoiding mostly-padding tiles.
- Skipping is numerically identical to the prior path, so results are unchanged.
- Ragged MoE env var handling was moved into cl_init for cleaner initialization.
Upgrade notes
No API or config changes are required; the optimization is enabled by default on OpenCL. Set GGML_OPENCL_MOE_RAGGED_FP16=0 to disable it, or adjust GGML_OPENCL_MOE_RAGGED_GRAN to 16 or 32 if you want coarser or no tile-skipping. Non-OpenCL backends are unaffected.
Analyzed Oct 30, 2026
Upgrading?
Release notes summary
OpenCL optimizations for MoE (Mixture of Experts) prefill operations.
Key Highlights
- Optimized ragged-tile MoE prefill FP16 GEMM by skipping padded expert tiles
- Implemented quarter-granularity ragged MoE tile-skip (8-col skip-groups)
- Added environment variables for configuring ragged MoE behavior
Release Notes (excerpt)
<details open> opencl: ragged-tile MoE prefill FP16 GEMM optimization (skip padded expert tiles) (#25433) * opencl: ragged-tile MoE prefill GEMM (skip padded expert tiles) The MoE prefill GEMM groups tokens into TILESIZE_N=32 per-expert tiles; at low tokens-per-expert most tiles are mostly padding. When a tile's upper 16 slots are all padding (router index 0xFFFFFFFF), skip the second dotx16_reduce8 half. Numerically identical (skipped lanes are padding). Applied to all eight *_f32_ns…
Published by ggml-org on GitHub.
Read the full release notesRelated Repositories
:electron: Build cross-platform desktop apps with JavaScript, HTML, and CSS
Godot Engine – Multi-platform 2D and 3D game engine
ClickHouse® is a real-time analytics database management system
Self-hosted game stream host for Moonlight.
DuckDB is a high-performance analytical database system