b10246
ggml-org/llama.cppb10246Aug 3, 2026by github-actions[bot]same day as b10245
What changed since b10245
OpenCL backend now routes large Q6_K lm_head weights to flat GEMV, fixing slowdowns on large models like Gemma-4 E2B.
New features
- OpenCL: large Q6_K lm_head tensors are now dispatched to the flat GEMV path instead of the shuffle variant.
Improvements
- Added a direct weight-size condition alongside the existing dimension check so oversized tensors that fail the ne0 >= 2048 test still take the faster kernel.
- Removes the performance regression seen with large Q6_K language-model heads on OpenCL devices.
Bug fixes
- Corrected kernel selection for large Q6_K lm_head weights (e.g. [1536, 262144] in Gemma-4 E2B) that previously fell through to a slow GEMV path.
Upgrade notes
This is a single-commit OpenCL-only change; no config, API, or model-format changes are involved. Users on OpenCL backends with large-vocabulary models should see faster token generation, while CPU, CUDA, Metal, and Vulkan paths are unaffected. Simply swap the binaries or rebuild.
Analyzed Oct 31, 2026
Upgrading?
Release notes summary
Optimized OpenCL kernel routing for large Q6_K language model heads to use flat GEMV operations, fixing performance issues with large models like Gemma-4 E2B.
Key Highlights
- Routed large q6_K lm_head to flat GEMV
- Fixed dimension condition for large weights to prevent slowdowns
- Addresses performance issues with Gemma-4 E2B models
Release Notes (excerpt)
<details open> opencl: route large q6_K lm_head to the flat GEMV (#26427) * add a direct size condition for `large` weights; the original dimension condition is insufficient -- q6_K lm_head for gemma-4 E2B has [1536, 262144], which is big enough to slowdown gemv_noshuffle but does not satisfy the dimension condition (ne0 >= 2048) </details> **Website:** - <https://llama.app> **macOS/iOS:** - macOS Apple Silicon (arm64) - macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED…
Published by ggml-org on GitHub.
Read the full release notesRelated Repositories
:electron: Build cross-platform desktop apps with JavaScript, HTML, and CSS
Godot Engine – Multi-platform 2D and 3D game engine
ClickHouse® is a real-time analytics database management system
Self-hosted game stream host for Moonlight.
DuckDB is a high-performance analytical database system