b9670

ggml-org/llama.cppb9670Jun 16, 2026by github-actions[bot]same day as b9669

What changed since b9669

low risk

b9670 is a single-PR bug fix release correcting NVFP4 quantization edge cases in llama-graph.

Improvements

  • NVFP4 build_ffn is now restricted to supported layer combinations, avoiding invalid graph construction.
  • Post-GEMM MUL ordering adjusted so dequantized values are used before residual and bias operations.

Bug fixes

  • Fixed incorrect post-GEMM MUL placement for NVFP4 LoRA paths, aligning with expected dequant-before-residual behavior.
  • Fixed NVFP4 ModelOPT bias-add to operate on fully dequantized values.
  • Prevented unsupported NVFP4 FFN configurations from being built.

Upgrade notes

No config or API changes are required; this is a drop-in replacement for b9669. Users running NVFP4-quantized models with LoRA adapters or ModelOPT checkpoints should upgrade to get correct numerics. Other quantization formats and backends are unaffected.

Analyzed Oct 31, 2026

Upgrading?

Release notes summary

A bug fix release addressing edge cases in NVFP4 (NVIDIA Floating Point 4) quantization within the llama-graph implementation. The changes ensure correct order of operations for LoRA and ModelOPT, specifically moving post-GEMM MUL operations.

Key Highlights

  • Fix for NVFP4 edge-cases in llama-graph
  • Correction of post-GEMM MUL order for LoRA and ModelOPT
  • Restriction of build_ffn for NVFP4 to supported combinations

New Features

  • NVFP4 quantization bug fixes

Release Notes (excerpt)

<details open> Fix and restrict NVFP4 edge-cases in llama-graph (#24331) * Move post-GEMM MUL required for dequant b4 lora and bias add see https://github.com/ggml-org/llama.cpp/pull/23484 : 1. For lora, I would presume we want fully dequantized values before doing the residuals, but this depends on how the LORAs were generated. Literature tells me LORA happens post-mul but pre-bias add https://github.com/ggml-org/llama.cpp/pull/8332…

Published by ggml-org on GitHub.

Read the full release notes

Related Repositories