b9603

ggml-org/llama.cppb9603Jun 12, 2026by github-actions[bot]same day as b9601

What changed since b9601

low risk

Adds OpenCL Q5_0/Q5_1 GEMM and GEMV kernels for Adreno GPUs, enabling faster Q5 inference on Qualcomm mobile hardware.

New features

  • Q5_0 quantization support in the OpenCL backend for Adreno GPUs.
  • Q5_1 quantization support in the OpenCL backend for Adreno GPUs.
  • New GEMM and GEMV kernels targeting Adreno hardware.

Improvements

  • Faster prompt processing and token generation for Q5 models on Adreno-based mobile devices.
  • Minor cosmetic cleanup in the OpenCL kernel code.

Upgrade notes

No configuration or model-format changes are required; the new kernels are selected automatically when running Q5_0/Q5_1 models on an Adreno GPU with the OpenCL backend. Users on other backends (CPU, CUDA, Vulkan, Metal) see no behavioral change and can upgrade freely. Rebuild or pull the matching OpenCL-enabled binaries to benefit.

Analyzed Oct 31, 2026

Upgrading?

Release notes summary

Added support for Q5 quantization types on Adreno GPUs.

Key Highlights

  • Added q5_0 and q5_1 GEMM and GEMV kernels for Adreno GPU (Qualcomm)
  • Improved inference performance on mobile devices with Adreno hardware

Release Notes (excerpt)

<details open> opencl: add q5_0/q5_1 gemm and gemv kernels for Adreno (#24319) * opencl: add q5_0 adreno support * opencl: add q5_1 adreno support * opencl: cosmetic fix --------- Co-authored-by: Li He <lih@qti.qualcomm.com> </details> **macOS/iOS:** - macOS Apple Silicon (arm64) - macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED - macOS Intel (x64) - iOS XCFramework **Linux:** - Ubuntu x64 (CPU) - Ubuntu arm64 (CPU) - Ubuntu s390x (CPU) - Ubuntu x64 (Vulkan)…

Published by ggml-org on GitHub.

Read the full release notes

Related Repositories