b10035
ggml-org/llama.cppb10035Jul 16, 2026by github-actions[bot]same day as b10034
What changed since b10034
b10035 adds a CUDA kernel optimization that extracts Q1_0 quantization elements via the __byte_perm intrinsic.
Improvements
- CUDA Q1_0 dequantization/extraction kernel now uses the __byte_perm intrinsic instead of the previous element extraction path.
- Faster Q1_0 inference on CUDA backends as a result of the intrinsic-based element extraction.
Upgrade notes
This is a single-commit CUDA-only optimization with no API, config, or model-format changes; a straight binary swap is sufficient. Users not running Q1_0 quantized models on CUDA will see no behavioral difference. The OpenCL MoE fix from b10034 remains in place.
Analyzed Nov 1, 2026
Upgrading?
Release notes summary
Optimizes the CUDA kernel for Q1_0 quantization type using the `__byte_perm` intrinsic instruction.
Key Highlights
- Extract Q1_0 elements via `__byte_perm`
- CUDA kernel optimization for Q1_0 type
New Features
- Optimized Q1_0 extraction kernel
Release Notes (excerpt)
<details open> cuda: extract Q1_0 elements via __byte_perm (#25628) </details> **macOS/iOS:** - macOS Apple Silicon (arm64) - macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED - macOS Intel (x64) - iOS XCFramework **Linux:** - Ubuntu x64 (CPU) - Ubuntu arm64 (CPU) - Ubuntu s390x (CPU) - Ubuntu x64 (Vulkan) - Ubuntu arm64 (Vulkan) - Ubuntu x64 (ROCm 7.2) - Ubuntu x64 (OpenVINO) - Ubuntu x64 (SYCL FP32) - Ubuntu x64 (SYCL FP16) **Android:** - Android arm64 (CPU) **Windows:**…
Published by ggml-org on GitHub.
Read the full release notesRelated Repositories
:electron: Build cross-platform desktop apps with JavaScript, HTML, and CSS
Godot Engine – Multi-platform 2D and 3D game engine
ClickHouse® is a real-time analytics database management system
Self-hosted game stream host for Moonlight.
DuckDB is a high-performance analytical database system