b9855

ggml-org/llama.cppb9855Jul 1, 2026by github-actions[bot]same day as b9852

What changed since b9852

low risk

b9855 adds AVX2-accelerated NVFP4 dot products with a UE4M3 lookup table, speeding up CPU inference.

New features

  • AVX2-optimized NVFP4 dot product path for CPU inference.

Improvements

  • NVFP4 dot products now use a UE4M3 lookup table, reducing per-element conversion overhead.
  • Faster CPU prompt processing and token generation for NVFP4-quantized models on AVX2-capable hardware.

Upgrade notes

This is a drop-in build with no config or model format changes; the gains only apply to NVFP4 models on CPUs exposing AVX2. Users on non-AVX2 or non-x86 hardware will see no difference. The OpenCL q1_0 support from b9852 remains unchanged.

Analyzed Oct 30, 2026

Upgrading?

Release notes summary

Enhances CPU performance by adding AVX2 optimizations for NVFP4 dot product operations and utilizing the UE4M3 lookup table for improved efficiency.

Key Highlights

  • AVX2 optimization for NVFP4 dot product
  • Use of UE4M3 LUT
  • CPU performance improvements

Release Notes (excerpt)

<details open> ggml-cpu: add AVX2 optimization for nvfp4 dot product and use UE4M3 LUT (#23961) </details> **macOS/iOS:** - macOS Apple Silicon (arm64) - macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED - macOS Intel (x64) - iOS XCFramework **Linux:** - Ubuntu x64 (CPU) - Ubuntu arm64 (CPU) - Ubuntu s390x (CPU) - Ubuntu x64 (Vulkan) - Ubuntu arm64 (Vulkan) - Ubuntu x64 (ROCm 7.2) - Ubuntu x64 (OpenVINO) - Ubuntu x64 (SYCL FP32) - Ubuntu x64 (SYCL FP16) **Android:**…

Published by ggml-org on GitHub.

Read the full release notes

Related Repositories