b10099
ggml-org/llama.cppb10099Jul 23, 2026by github-actions[bot]same day as b10098
What changed since b10098
b10099 adds CUDA-side NVFP4 W4A4 activation quantization optimizations on top of b10098's Hexagon activation work.
New features
- Fused per-channel amax and quantization kernels for NVFP4 W4A4 activation quantization on CUDA.
- Uses the nvfp4x4 intrinsic where the target hardware exposes it.
Improvements
- Faster NVFP4 W4A4 activation quantization path on CUDA via fused kernels and 32-byte loads.
- Pointer arithmetic on the input is hoisted so it is computed only once.
- Scale-search re-added and optimized with intrinsics rather than scalar code.
- Compiler-level tweaks reduce unnecessary LDC instructions in the quantization path.
- MMQ epilogue scale handling is now restricted to NVFP4 semantics, with a host-side 64-alignment assertion on ne00.
Bug fixes
- Guards the 32-byte aligned struct attribute to NVIDIA compilers, fixing HIP build breakage.
- Corrects kvalues_mxfp4 to kvalues_nvfp4 in quantize_mmq_nvfp4.
Upgrade notes
This is a routine build-to-build bump with no config or API changes; CUDA users running NVFP4 W4A4 models get the speedup automatically. HIP builds are explicitly covered by the alignment-attribute guard, so no workaround is needed. Hexagon changes from b10098 remain unchanged in this release.
Analyzed Nov 1, 2026
Upgrading?
Release notes summary
Improves NVFP4 W4A4 activation quantization on CUDA with fused kernels and optimized memory access patterns.
Key Highlights
- Improve NVFP4 W4A4 activation quantization
- Fuse per-channel amax and quantization kernels
- Use nvfp4x4 intrinsic when available
Release Notes (excerpt)
<details open> CUDA: Improve NVFP4 W4A4 activation quantization (#25730) * Squash history before conflict-resolution during rebase on master WIP commit Add 32-byte loads, restore per-block amax Use nvfp4x4 intrinsic when available Fuse per-channel amax and quantization kernels Do pointer arithmetic only once on x Remove unnecessary ternary in the load We assert on host side that ne00 is 64-aligned Add back scale-search, but optimize it with intrinsics Code cleanup…
Published by ggml-org on GitHub.
Read the full release notesRelated Repositories
:electron: Build cross-platform desktop apps with JavaScript, HTML, and CSS
Godot Engine – Multi-platform 2D and 3D game engine
ClickHouse® is a real-time analytics database management system
Self-hosted game stream host for Moonlight.
DuckDB is a high-performance analytical database system