b10035

ggml-org/llama.cppb10035Jul 16, 2026by github-actions[bot]same day as b10034

What changed since b10034

low risk

b10035 adds a CUDA kernel optimization that extracts Q1_0 quantization elements via the __byte_perm intrinsic.

Improvements

  • CUDA Q1_0 dequantization/extraction kernel now uses the __byte_perm intrinsic instead of the previous element extraction path.
  • Faster Q1_0 inference on CUDA backends as a result of the intrinsic-based element extraction.

Upgrade notes

This is a single-commit CUDA-only optimization with no API, config, or model-format changes; a straight binary swap is sufficient. Users not running Q1_0 quantized models on CUDA will see no behavioral difference. The OpenCL MoE fix from b10034 remains in place.

Analyzed Nov 1, 2026

Upgrading?

Release notes summary

Optimizes the CUDA kernel for Q1_0 quantization type using the `__byte_perm` intrinsic instruction.

Key Highlights

  • Extract Q1_0 elements via `__byte_perm`
  • CUDA kernel optimization for Q1_0 type

New Features

  • Optimized Q1_0 extraction kernel

Release Notes (excerpt)

<details open> cuda: extract Q1_0 elements via __byte_perm (#25628) </details> **macOS/iOS:** - macOS Apple Silicon (arm64) - macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED - macOS Intel (x64) - iOS XCFramework **Linux:** - Ubuntu x64 (CPU) - Ubuntu arm64 (CPU) - Ubuntu s390x (CPU) - Ubuntu x64 (Vulkan) - Ubuntu arm64 (Vulkan) - Ubuntu x64 (ROCm 7.2) - Ubuntu x64 (OpenVINO) - Ubuntu x64 (SYCL FP32) - Ubuntu x64 (SYCL FP16) **Android:** - Android arm64 (CPU) **Windows:**…

Published by ggml-org on GitHub.

Read the full release notes

Related Repositories