b10209

ggml-org/llama.cppb10209Jul 31, 2026by github-actions[bot]same day as b10208

What changed since b10208

low risk

b10209 adds a CUDA-only micro-optimization for Q2_0 dequantization; otherwise identical to b10208.

Improvements

  • CUDA kernels now extract Q2_0 quantization elements using the __byte_perm intrinsic instead of the previous extraction path.
  • Faster Q2_0 dequantization on NVIDIA GPUs, which should modestly improve prompt processing and token generation for Q2_0-quantized models.
  • Change is scoped to the CUDA backend, so CPU, SYCL, Vulkan, ROCm, and Metal paths are unaffected.

Upgrade notes

This is a routine incremental build with no API, config, or model-format changes, so it can be dropped in place of b10208. Only CUDA users running Q2_0 quantized models will see any behavioral difference, and it is purely a performance optimization. No migration or environment-variable changes are required.

Analyzed Oct 31, 2026

Upgrading?

Release notes summary

This release optimizes CUDA kernel performance for Q2_0 quantization by using the __byte_perm intrinsic for efficient element extraction.

Key Highlights

  • CUDA optimization for Q2_0 quantization
  • Uses __byte_perm intrinsic for element extraction

New Features

  • CUDA Q2_0 optimization

Release Notes (excerpt)

<details open> cuda: extract Q2_0 elements via __byte_perm (#25603) </details> **Website:** - <https://llama.app> **macOS/iOS:** - macOS Apple Silicon (arm64) - macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED - macOS Intel (x64) - iOS XCFramework **Linux:** - Ubuntu x64 (CPU) - Ubuntu arm64 (CPU) - Ubuntu s390x (CPU) - Ubuntu x64 (Vulkan) - Ubuntu arm64 (Vulkan) - Ubuntu x64 (ROCm 7.2) - Ubuntu x64 (OpenVINO) - Ubuntu x64 (SYCL FP32) - Ubuntu x64 (SYCL FP16) **Android:**…

Published by ggml-org on GitHub.

Read the full release notes

Related Repositories