b10165
ggml-org/llama.cppb10165Jul 28, 2026by github-actions[bot]same day as b10164
What changed since b10164
b10165 restores IQ4_NL quantization support in Vulkan flash-attention, fixing a regression tracked in issue #23681.
New features
- Vulkan backend's flash-attention path accepts IQ4_NL-quantized weights again, so those models no longer need a fallback or fail outright.
- Temporary q1_0 flash-attention support for non-coopmat2 was prototyped in the same PR but deliberately dropped before merge.
Improvements
- Vulkan FA now spends a small amount of shared memory on the IQ4_NL lookup table, which the author judged negligible in practice.
- Compatibility gap between Vulkan FA and IQ4_NL quantization formats is closed.
Bug fixes
- Resolves issue #23681, where IQ4_NL models could not use flash attention on Vulkan.
Upgrade notes
This is a Vulkan-only fix layered on top of b10164's CUDA Mamba-2 prefill work; no flags, config, or model files need to change. Vulkan users running IQ4_NL quantizations with flash attention enabled should upgrade to regain that path. Users on CUDA/HIP/MUSA or CPU-only builds see no behavioral difference from b10164.
Analyzed Oct 31, 2026
Upgrading?
Release notes summary
Restores support for IQ4_NL quantization format in the Vulkan backend's Fully Attention (FA) mode to fix compatibility issues.
Key Highlights
- Vulkan IQ4_NL support restored
- Fixes issue #23681 regarding quantization support
Release Notes (excerpt)
<details open> vulkan: add iq4_nl support back to FA (#24585) * vulkan: add iq4_nl support back to FA I was originally concerned about wasting shared memory on the LUT, but it's small and unlikely to matter in practice. Also support q1_0 for non-coopmat2. Fixes #23681 * remove q1_0 FA support </details> **Website:** - <https://llama.app> **macOS/iOS:** - macOS Apple Silicon (arm64) - macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED - macOS Intel (x64) - iOS XCFramework…
Published by ggml-org on GitHub.
Read the full release notesRelated Repositories
:electron: Build cross-platform desktop apps with JavaScript, HTML, and CSS
Godot Engine – Multi-platform 2D and 3D game engine
ClickHouse® is a real-time analytics database management system
Self-hosted game stream host for Moonlight.
DuckDB is a high-performance analytical database system