b10165

ggml-org/llama.cppb10165Jul 28, 2026by github-actions[bot]same day as b10164

What changed since b10164

low risk

b10165 restores IQ4_NL quantization support in Vulkan flash-attention, fixing a regression tracked in issue #23681.

New features

  • Vulkan backend's flash-attention path accepts IQ4_NL-quantized weights again, so those models no longer need a fallback or fail outright.
  • Temporary q1_0 flash-attention support for non-coopmat2 was prototyped in the same PR but deliberately dropped before merge.

Improvements

  • Vulkan FA now spends a small amount of shared memory on the IQ4_NL lookup table, which the author judged negligible in practice.
  • Compatibility gap between Vulkan FA and IQ4_NL quantization formats is closed.

Bug fixes

  • Resolves issue #23681, where IQ4_NL models could not use flash attention on Vulkan.

Upgrade notes

This is a Vulkan-only fix layered on top of b10164's CUDA Mamba-2 prefill work; no flags, config, or model files need to change. Vulkan users running IQ4_NL quantizations with flash attention enabled should upgrade to regain that path. Users on CUDA/HIP/MUSA or CPU-only builds see no behavioral difference from b10164.

Analyzed Oct 31, 2026

Upgrading?

Release notes summary

Restores support for IQ4_NL quantization format in the Vulkan backend's Fully Attention (FA) mode to fix compatibility issues.

Key Highlights

  • Vulkan IQ4_NL support restored
  • Fixes issue #23681 regarding quantization support

Release Notes (excerpt)

<details open> vulkan: add iq4_nl support back to FA (#24585) * vulkan: add iq4_nl support back to FA I was originally concerned about wasting shared memory on the LUT, but it's small and unlikely to matter in practice. Also support q1_0 for non-coopmat2. Fixes #23681 * remove q1_0 FA support </details> **Website:** - <https://llama.app> **macOS/iOS:** - macOS Apple Silicon (arm64) - macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED - macOS Intel (x64) - iOS XCFramework…

Published by ggml-org on GitHub.

Read the full release notes

Related Repositories