b10246

ggml-org/llama.cppb10246Aug 3, 2026by github-actions[bot]same day as b10245

What changed since b10245

low risk

OpenCL backend now routes large Q6_K lm_head weights to flat GEMV, fixing slowdowns on large models like Gemma-4 E2B.

New features

  • OpenCL: large Q6_K lm_head tensors are now dispatched to the flat GEMV path instead of the shuffle variant.

Improvements

  • Added a direct weight-size condition alongside the existing dimension check so oversized tensors that fail the ne0 >= 2048 test still take the faster kernel.
  • Removes the performance regression seen with large Q6_K language-model heads on OpenCL devices.

Bug fixes

  • Corrected kernel selection for large Q6_K lm_head weights (e.g. [1536, 262144] in Gemma-4 E2B) that previously fell through to a slow GEMV path.

Upgrade notes

This is a single-commit OpenCL-only change; no config, API, or model-format changes are involved. Users on OpenCL backends with large-vocabulary models should see faster token generation, while CPU, CUDA, Metal, and Vulkan paths are unaffected. Simply swap the binaries or rebuild.

Analyzed Oct 31, 2026

Upgrading?

Release notes summary

Optimized OpenCL kernel routing for large Q6_K language model heads to use flat GEMV operations, fixing performance issues with large models like Gemma-4 E2B.

Key Highlights

  • Routed large q6_K lm_head to flat GEMV
  • Fixed dimension condition for large weights to prevent slowdowns
  • Addresses performance issues with Gemma-4 E2B models

Release Notes (excerpt)

<details open> opencl: route large q6_K lm_head to the flat GEMV (#26427) * add a direct size condition for `large` weights; the original dimension condition is insufficient -- q6_K lm_head for gemma-4 E2B has [1536, 262144], which is big enough to slowdown gemv_noshuffle but does not satisfy the dimension condition (ne0 >= 2048) </details> **Website:** - <https://llama.app> **macOS/iOS:** - macOS Apple Silicon (arm64) - macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED…

Published by ggml-org on GitHub.

Read the full release notes

Related Repositories