b10209
ggml-org/llama.cppb10209Jul 31, 2026by github-actions[bot]same day as b10208
What changed since b10208
b10209 adds a CUDA-only micro-optimization for Q2_0 dequantization; otherwise identical to b10208.
Improvements
- CUDA kernels now extract Q2_0 quantization elements using the __byte_perm intrinsic instead of the previous extraction path.
- Faster Q2_0 dequantization on NVIDIA GPUs, which should modestly improve prompt processing and token generation for Q2_0-quantized models.
- Change is scoped to the CUDA backend, so CPU, SYCL, Vulkan, ROCm, and Metal paths are unaffected.
Upgrade notes
This is a routine incremental build with no API, config, or model-format changes, so it can be dropped in place of b10208. Only CUDA users running Q2_0 quantized models will see any behavioral difference, and it is purely a performance optimization. No migration or environment-variable changes are required.
Analyzed Oct 31, 2026
Upgrading?
Release notes summary
This release optimizes CUDA kernel performance for Q2_0 quantization by using the __byte_perm intrinsic for efficient element extraction.
Key Highlights
- CUDA optimization for Q2_0 quantization
- Uses __byte_perm intrinsic for element extraction
New Features
- CUDA Q2_0 optimization
Release Notes (excerpt)
<details open> cuda: extract Q2_0 elements via __byte_perm (#25603) </details> **Website:** - <https://llama.app> **macOS/iOS:** - macOS Apple Silicon (arm64) - macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED - macOS Intel (x64) - iOS XCFramework **Linux:** - Ubuntu x64 (CPU) - Ubuntu arm64 (CPU) - Ubuntu s390x (CPU) - Ubuntu x64 (Vulkan) - Ubuntu arm64 (Vulkan) - Ubuntu x64 (ROCm 7.2) - Ubuntu x64 (OpenVINO) - Ubuntu x64 (SYCL FP32) - Ubuntu x64 (SYCL FP16) **Android:**…
Published by ggml-org on GitHub.
Read the full release notesRelated Repositories
:electron: Build cross-platform desktop apps with JavaScript, HTML, and CSS
Godot Engine – Multi-platform 2D and 3D game engine
ClickHouse® is a real-time analytics database management system
Self-hosted game stream host for Moonlight.
DuckDB is a high-performance analytical database system