b9859

ggml-org/llama.cppb9859Jul 1, 2026by github-actions[bot]same day as b9858

What changed since b9858

medium risk

b9859 adds OpenCL support for loading precompiled binary kernels, speeding up MOE GEMM across several quantization types.

New features

  • OpenCL backend can now load precompiled binary kernels from a shared library instead of relying solely on runtime compilation.
  • MOE GEMM operations on OpenCL are optimized through library-provided kernels.
  • Library-based MOE GEMM kernels are available for q8_0, q4_0, q4_1, and q4_k quantization types.
  • A dedicated gemm_moe_mxfp4_f32_ns kernel is loaded from the kernel library when present.

Improvements

  • Added a libdl.h shim to avoid a cyclic dependency between ggml-opencl and ggml-backend-dl.
  • OpenCL kernel loading falls back gracefully when the binary kernel library is unavailable.
  • Documentation updated to describe the precompiled kernel library mechanism.

Upgrade notes

This release is additive for the OpenCL backend; existing CPU, CUDA, Metal, and Vulkan paths are unaffected. Users on Adreno/OpenCL hardware should see faster MOE inference if the precompiled kernel library ships with their build, otherwise behavior is unchanged. No config or model migration is required.

Analyzed Oct 31, 2026

Upgrading?

Release notes summary

Enhances OpenCL backend performance by allowing the loading of precompiled binary kernels from a shared library for GEMM operations, including support for MOE and various quantization types.

Key Highlights

  • Load precompiled binary kernels from library
  • Optimized MOE GEMM operations
  • Support for q8_0, q4_0, q4_1, q4_k moe gemm

Release Notes (excerpt)

<details open> opencl: allow loading precompiled binary kernels from library (#23042) * opencl: allow loading binary kernel * opencl: add libdl.h * ggml-backend-dl is in ggml, which depends backend libs, thus ggml-opencl cannot depend on ggml-backend-dl * add libdl.h to break cyclic dep * opencl: allow loading bin kernel lib * opencl: load `gemm_moe_mxfp4_f32_ns` from kernel lib if available * opencl: load q8_0 gemm from kernel lib * opencl: load q4_0 moe gemm from kernel lib…

Published by ggml-org on GitHub.

Read the full release notes

Related Repositories