b10099

ggml-org/llama.cppb10099Jul 23, 2026by github-actions[bot]same day as b10098

What changed since b10098

low risk

b10099 adds CUDA-side NVFP4 W4A4 activation quantization optimizations on top of b10098's Hexagon activation work.

New features

  • Fused per-channel amax and quantization kernels for NVFP4 W4A4 activation quantization on CUDA.
  • Uses the nvfp4x4 intrinsic where the target hardware exposes it.

Improvements

  • Faster NVFP4 W4A4 activation quantization path on CUDA via fused kernels and 32-byte loads.
  • Pointer arithmetic on the input is hoisted so it is computed only once.
  • Scale-search re-added and optimized with intrinsics rather than scalar code.
  • Compiler-level tweaks reduce unnecessary LDC instructions in the quantization path.
  • MMQ epilogue scale handling is now restricted to NVFP4 semantics, with a host-side 64-alignment assertion on ne00.

Bug fixes

  • Guards the 32-byte aligned struct attribute to NVIDIA compilers, fixing HIP build breakage.
  • Corrects kvalues_mxfp4 to kvalues_nvfp4 in quantize_mmq_nvfp4.

Upgrade notes

This is a routine build-to-build bump with no config or API changes; CUDA users running NVFP4 W4A4 models get the speedup automatically. HIP builds are explicitly covered by the alignment-attribute guard, so no workaround is needed. Hexagon changes from b10098 remain unchanged in this release.

Analyzed Nov 1, 2026

Upgrading?

Release notes summary

Improves NVFP4 W4A4 activation quantization on CUDA with fused kernels and optimized memory access patterns.

Key Highlights

  • Improve NVFP4 W4A4 activation quantization
  • Fuse per-channel amax and quantization kernels
  • Use nvfp4x4 intrinsic when available

Release Notes (excerpt)

<details open> CUDA: Improve NVFP4 W4A4 activation quantization (#25730) * Squash history before conflict-resolution during rebase on master WIP commit Add 32-byte loads, restore per-block amax Use nvfp4x4 intrinsic when available Fuse per-channel amax and quantization kernels Do pointer arithmetic only once on x Remove unnecessary ternary in the load We assert on host side that ne00 is 64-aligned Add back scale-search, but optimize it with intrinsics Code cleanup…

Published by ggml-org on GitHub.

Read the full release notes

Related Repositories