b9670
ggml-org/llama.cppb9670Jun 16, 2026by github-actions[bot]same day as b9669
What changed since b9669
b9670 is a single-PR bug fix release correcting NVFP4 quantization edge cases in llama-graph.
Improvements
- NVFP4 build_ffn is now restricted to supported layer combinations, avoiding invalid graph construction.
- Post-GEMM MUL ordering adjusted so dequantized values are used before residual and bias operations.
Bug fixes
- Fixed incorrect post-GEMM MUL placement for NVFP4 LoRA paths, aligning with expected dequant-before-residual behavior.
- Fixed NVFP4 ModelOPT bias-add to operate on fully dequantized values.
- Prevented unsupported NVFP4 FFN configurations from being built.
Upgrade notes
No config or API changes are required; this is a drop-in replacement for b9669. Users running NVFP4-quantized models with LoRA adapters or ModelOPT checkpoints should upgrade to get correct numerics. Other quantization formats and backends are unaffected.
Analyzed Oct 31, 2026
Upgrading?
Release notes summary
A bug fix release addressing edge cases in NVFP4 (NVIDIA Floating Point 4) quantization within the llama-graph implementation. The changes ensure correct order of operations for LoRA and ModelOPT, specifically moving post-GEMM MUL operations.
Key Highlights
- Fix for NVFP4 edge-cases in llama-graph
- Correction of post-GEMM MUL order for LoRA and ModelOPT
- Restriction of build_ffn for NVFP4 to supported combinations
New Features
- NVFP4 quantization bug fixes
Release Notes (excerpt)
<details open> Fix and restrict NVFP4 edge-cases in llama-graph (#24331) * Move post-GEMM MUL required for dequant b4 lora and bias add see https://github.com/ggml-org/llama.cpp/pull/23484 : 1. For lora, I would presume we want fully dequantized values before doing the residuals, but this depends on how the LORAs were generated. Literature tells me LORA happens post-mul but pre-bias add https://github.com/ggml-org/llama.cpp/pull/8332…
Published by ggml-org on GitHub.
Read the full release notesRelated Repositories
:electron: Build cross-platform desktop apps with JavaScript, HTML, and CSS
Godot Engine – Multi-platform 2D and 3D game engine
ClickHouse® is a real-time analytics database management system
Self-hosted game stream host for Moonlight.
DuckDB is a high-performance analytical database system