3.6.0

# MNN 3.6.0 版本发布说明

**发布日期**: 2026年6月

📦 **安装**: `pip install mnn==3.6.0` ｜ [PyPI](https://pypi.org/project/mnn/3.6.0/)

## 📌 版本概述

MNN 3.6.0 版本聚焦于 **多模态 LLM 模型扩展与推理加速**、**GPU 多后端深度优化** 以及 **低比特量化与硬件适配增强** 三大核心主题：

*   **多模态 LLM 模型扩展与推理加速**：新增 Gemma4（文本/视觉/语音）、Qwen3.5、Hymt2 等模型支持；推出 dflash 推测解码；Omni 模型 Thinker/Talker 交错生成将 TTFA 降低 2.7×；引入 fp16 存储的 weight-quant scale/zero-point；通过 onClone 重构 KV Cache 复用机制。
*   **GPU 多后端深度优化**：Vulkan 新增 VulkanConv1x1CoopA8（W8A8 coopMat）与 no-subgroup 回退路径；OpenCL 引入 GEMM/GEMV 启发式调度、int4 image1d_buffer_t 优化、原生 TopKV2 与 2/3-bit 权重量化；Metal 优化 Conv1x1 分支选择；CUDA 支持 Blackwell（sm_120）并优化 Eagle 推测解码；QNN 支持 Qwen3.5 与离线图多进程并行导出。
*   **低比特量化与硬件适配增强**：CPU 与 GPU 全栈支持 2/3-bit 权重量化（ARM low-bit 量化内核 + OpenCL/Metal/Vulkan）；ARMv8.2/v8.6 decode 路径深度优化；KleidiAI 升级到 v1.16.0；新增 RVV 向量化的 Int8/Norm/Gelu/SiLu/ExpC8 等核心算子；新增 fused MNNFusedGatedDelta（LinearAttention prefill）；新增 LinearAttention L=1 decode 快速路径。

---

## 🚀 版本亮点

*   **Gemma4 全模态支持**：新增 Gemma4 文本、视觉与音频模型的完整推理支持
*   **Wan2.1-T2V-1.3B 视频生成**：新增 Wan2.1-T2V-1.3B 文生视频模型的导出与推理能力
*   **LLM 推理加速**：新增 dflash 推测解码；Omni 模型 Thinker/Talker 交错生成，TTFA 降低 2.7×
*   **CPU/GPU 2/3-bit 权重量化全栈支持**：CPU 新增 low-bit ARM 量化内核，OpenCL/Metal/Vulkan 同步支持 2/3-bit 权重量化
*   **Vulkan W8A8 coopMat**：新增 VulkanConv1x1CoopA8，支持动态量化 W8A8 + Cooperative Matrix

---

## ✨ 新功能

### LLM / VLM
*   **Gemma4 模型支持**：新增 Gemma4 文本、视觉、音频全模态推理
*   **dflash 推测解码**：新增 dflash 推测解码方案
*   **Omni Thinker/Talker 交错生成**：Omni 模型 Thinker 与 Talker 交错执行，TTFA 降低 2.7×
*   **fp16 Scale/Zero-Point 存储**：权重量化的 scale/zero-point 支持 fp16 存储
*   **KV Cache onClone 重构**：通过 onClone 统一各后端 KV Cache 复用逻辑
*   **LLM log 保存到 context**：支持将 LLM 推理日志保存到 context
*   **linear_attention 视为 full attention**：性能优化路径

### 模型导出 / 模型支持
*   **Gemma4 模型导出**：支持 Gemma4 多模态导出
*   **Stable Diffusion 3.5 Medium**：新增 SD 3.5 Medium 模型支持
*   **Wan2.1-T2V-1.3B 视频模型**：新增视频生成模型的导出与推理

### GPU 后端
*   **Vulkan VulkanConv1x1CoopA8**：新增 W8A8 动态量化 + Cooperative Matrix 卷积
*   **Vulkan LinearAttention 无 subgroup 回退**：兼容不支持 subgroup 的设备
*   **Vulkan gemv decode 无 subgroup 回退**：decode 路径的兼容性增强
*   **Vulkan 持久状态跨响应复用**：支持持久状态在多次响应间复用
*   **Vulkan LayerNorm onClone**：支持 LayerNorm onClone，opt pipeline 构造时初始化
*   **OpenCL GEMM/GEMV 启发式调度**：量化 GEMM/GEMV 启发式 + XGEMM FP local size 设置
*   **OpenCL int4 GEMM/GEMV 优化**：使用 image1d_buffer_t 优化 int4 GEMM/GEMV，免除 dequant 时额外的 -8
*   **OpenCL 原生 TopKV2**：OpenCL IMAGE 与 Vulkan（image/buffer）新增原生 TopKV2 实现
*   **OpenCL LinearAttention chunk 分支优化**：优化 chunk 分支性能
*   **OpenCL/Metal/Vulkan 2/3-bit 权重量化**：GPU 三后端统一支持 2/3-bit 权重量化（CPU 端通过 low-bit ARM 量化内核支持）
*   **Metal Conv1x1 分支选择优化**：MetalConvolution1x1 分支路径优化
*   **CUDA Blackwell (sm_120) 支持**：新增 sm_120 计算架构支持
*   **CUDA Eagle 推测解码优化**：CUDA Eagle 推测解码性能优化
*   **QNN Qwen3.5 支持**：QNN 后端支持 Qwen3.5 及视觉模型修复
*   **QNN 离线图多进程并行导出**：使用多进程并行加速离线图导出
*   **QNN Binary 算子优化**：加速视觉模型的 binary op 路径

### CPU 后端
*   **RVV Int8 加速**：新增 RVV 加速的 Int8 函数
*   **RVV 通用 opt 函数**：新增 RVV 向量化的 MNNNorm、MNNGelu、MNNSiLu、MNNExpC8 等
*   **ARM v8.2/v8.6 Decode 优化**：手机端 LLM decode 性能显著提升
*   **LinearAttention prefill fused kernel**：新增 MNNFusedGatedDelta fused kernel
*   **LinearAttention L=1 decode 快速路径**：新增 decode 快速路径
*   **Low-bit ARM 量化内核**：CPU 新增 2/3-bit 等低比特 ARM 量化内核支持
*   **fp16 alpha lowMemory 优化**：跳过 fp32 round-trip
*   **int4 reorder 性能优化**：减少非 fast path 的 int4 reorder 开销

### 构建 / 测试 / 工具
*   **MNN 发布工作流升级**：新增 LLM/OMNI、Harmony、Android 16K 页大小支持
*   **test_ci.sh**：新增统一 CI 入口脚本
*   **CMake 生成 podspec**：从 CMake 自动生成 podspec
*   **GEMM 性能基准**：新增 FP32/INT8/Block32-INT4 GEMM 基准
*   **GEMV 带宽基准**：新增 CPU/Metal w8/w4/w3/w2 GEMV 带宽基准

### 应用
*   **MNNChat Android 0.8.3 模型市场**：为 Android 用户准备 0.8.3 版本模型目录

---

## 🐛 缺陷修复

### LLM / VLM
*   **Qwen3.5 head_dim 自动检测**：修复 gated attention 模型的 head_dim 自动检测
*   **LinearAttention prefix disk cache**：修复 LinearAttention prefix disk cache 精度及混合模型支持问题
*   **prefix kvcache 损坏崩溃**：修复 prefix kvcache 损坏导致的崩溃以及 expandKVCacheInDisk 中的 size_t 截断
*   **推测解码 cache mismatch**：修复推测解码中的 cache mismatch 与 mrope 支持
*   **vision encoder 内存泄露**：修复 vision encoder 内存泄露与延迟增长问题
*   **llm_demo linear-attention 状态重置**：在 llm_demo 中重置 linear-attention 状态
*   **kvmeta 内存泄露**：修复 kvmeta 内存泄露
*   **stop_ids fallback**：修复 list 形式 eos_token_id 的 stop_ids fallback
*   **GPTQ weight_dict 重初始化**：修复 GPTQ loader 中的 weight_dict 重初始化问题
*   **GTE embedding/reranker RoPE**：恢复 GTE embedding/reranker 导出中清零的 RoPE inv_freq buffer
*   **Gemma3 dual-RoPE 导出**：修复 Gemma3 dual-RoPE 导出与 Qwen3-VL bilinear 采样
*   **Gemma4 导出**：修复 HF transformers>=5.5.4 纯 KV sharing 导致的 gemma4 导出崩溃；通过 mapper 隔离 k_eq_v 与 head_dim 自动检测
*   **Hymt2 qk norm 导出**：修复 Hymt2 qk norm 导出处理
*   **Qwen2.5-Omni 音频预处理**：修复 Qwen2.5-Omni 音频预处理的边界情况
*   **Qwen3-VL DeepStack dtype mismatch**：修复 Qwen3-VL DeepStack dtype 不一致问题
*   **Qwen3-VL 重复输出**：通过 per-RTM KVCACHE_INFO meta 修复 Qwen3-VL 重复输出问题
*   **_maskQK gapLen**：将 gapLen 计算移动到 null mask early return 之后

### GPU 后端
*   **Vulkan attention 及其他算子修复**
*   **Vulkan 静态符号查找改为动态符号**
*   **Metal output reuse offset 错误**：修复 Metal 输出复用 offset 错误
*   **Metal API 兼容性保护**：新增 Metal API 兼容性保护
*   **OpenCL Attention 越界访问**：修复 Attention 算子在内存重分配前的越界访问
*   **OpenCL LinearAttention GQA bug**：修复 OpenCL LinearAttention 的 GQA bug
*   **CUDA Softmax echo / FP16FP32Mix**：修复 FP16FP32Mix 模式下 Softmax 回声伪影与 MatMul precisionType 问题
*   **CUDA Windows / MNN_CUDA_NATIVE_ARCH 编译**：修复 CUDA 在 Windows 与 MNN_CUDA_NATIVE_ARCH 下的 CMake 构建失败
*   **QNN MSVC 编译**：修复 QNN 后端及 MNN2QNNModel 在 Windows MSVC 下的编译错误

### CPU 后端
*   **VNNI w4 < 64 计算错误**：修复 VNNI w4 < 64 的计算错误
*   **KleidiAI int4 dequant**：升级 KleidiAI 到 v1.16.0，修复 block-quantized int4 模型的反量化错误
*   **KleidiAI 路径映射**：修复将 MNN 作为子项目时的 KleidiAI 路径映射
*   **KleidiAI depthwise SME2 guard**：为 KleidiAI depthwise conv 路径添加 SME2 guard
*   **RVV pack/unpack 错误**：修复 RVV pack/unpack 函数错误
*   **CPU sdot fallback**：避免在 Memory_Low 外强制 fp32 fallback（CPU 无 sdot 时）
*   **cstdint 包含缺失**：补全 cstdint 头文件
*   **cpufreq 扫描错误日志**：修复扫描 cpufreq policy 目录时的错误日志

### 算子 / Converter
*   **Unique 算子**：修复 Unique 算子使输出符合 ONNX 排序规范；修复 index 输出问题
*   **TFLite binary activation**：修复 TFLite 转换器中 binary 激活的转换
*   **ONNX Resize / Einsum**：修复 ONNX Resize 转换器回归与 Einsum 4D 非对齐 batch 维快速路径
*   **cumsum int 类型不一致**：修复 cumsum int 类型不一致与 virtual type command 错误

### Diffusion
*   **SD1.5 resize cache warmup**：修复 SD1.5 resize cache warmup
*   **tokenizer.hpp cstdint**：修复 tokenizer.hpp 中 uint8_t 未声明错误
*   **diffusion llm link sep-build**：修复 sep-build 模式下的 llm 链接问题

### 编译 / Embedding / 其它
*   **embedding_demo 崩溃**：修复 mMeta->add==0 导致的 embedding_demo 崩溃
*   **PyMNN llm 子模块**：使 PyMNN 中的 llm 子模块可选导入
*   **Windows PyMNN .pdb 膨胀**：移除 Windows 上的 /Zi 和 /DEBUG 标志，避免 .pdb 膨胀 wheel
*   **dflash Windows MSVC 编译**：修复 dflash 在 Windows MSVC 下的编译错误
*   **通用编译问题修复**：修复 commit f40b3d2a7 引入的若干编译问题

---

## 📚 其他改进

*   **AGENTS.md**：将 CLAUDE.md 重命名为 AGENTS.md 并通过 include 转发
*   **VirtualFaceCapture-MNN 加入 demo 列表**
*   **MNN_CUDA_NATIVE_ARCH 文档**：补充 MNN_CUDA_NATIVE_ARCH cmake 选项的文档
*   **fp16 scale schema 同步**
*   **代码清理：删除 `MNN_CPU_DEQUANT_GEMM`**：移除已废弃的 MNN_CPU_DEQUANT_GEMM 相关代码
*   **代码清理：删除 `WEIGHT_DEQUANT_GEMM` 宏**：移除废弃宏，精简代码

---

## 📊 性能基准

<img width="4200" height="1800" alt="combined_bench" src="https://github.com/user-attachments/assets/958885de-bd8a-4749-b774-8e20f505f113" />



> 测试环境: Mac mini M4 (16GB) / OnePlus PJZ110 (Snapdragon 8 Elite) | 模型: Qwen3-0.6B / Qwen3-4B | 量化: W4 (block=64)

## 🙏 致谢

衷心感谢所有贡献者对本版本的宝贵贡献（按 GitHub login 列出）：

*   **@jxt1234** — ARMv8.2/v8.6 decode 优化 (#4389)、Metal Conv1x1 分支选择优化 (#4520)、Metal output reuse offset 修复 (#4365)、VNNI w4 < 64 修复 (#4366)、int4 reorder 优化 (#4511)、Geometry cumsum virtual type 修复 (#4391)
*   **@Fatemanx (xly)** — Wan2.1-T2V-1.3B 视频模型 (#4469)、CUDA Blackwell sm_120 (#4467)、Qwen2.5-Omni 音频边界修复 (#4459)、Qwen3-VL DeepStack dtype 修复 (#4504)、Hymt2 qk norm 导出 (#4490)、SD1.5 resize cache warmup (#4530)、PyMNN llm 可选导入 (#4468)、Converter Einsum 4D fast-path (#4466)
*   **@EricMoin** — Omni Thinker/Talker 交错生成，TTFA ↓2.7× (#4397)
*   **@luckysmg** — LinearAttention L=1 decode fast path (#4356)、MNNFusedGatedDelta prefill fused kernel (#4434)、LinearAttention prefix disk cache 精度修复 (#4495)、prefix kvcache 崩溃 / expandKVCacheInDisk size_t 修复 (#4413)
*   **@hebangwen** — vision encoder 内存泄露与延迟增长修复 (#4441)、llm_demo linear-attention 状态重置 (#4424)
*   **@LudovicoYIN (YinHanke)** — ONNX Resize / Einsum lowering 修复 (#4254)
*   **@cgisky1980 (顾真牛)** — CUDA Softmax echo / FP16FP32Mix 修复 (#4476)、CUDA Windows / MNN_CUDA_NATIVE_ARCH 构建修复 (#4477)
*   **@yydhYYDH (YYDH)** — GPTQ loader weight_dict 重初始化修复 (#4411)、QNN 离线图多进程并行导出 (#4414)
*   **@HenryDen** — KleidiAI v1.16.0 升级修复 int4 dequant (#4474)、KleidiAI 子项目路径映射修复 (#4542)
*   **@luke-opus (liushengtong)** — RVV 向量化 MNNNorm/MNNGelu (#4508)、RVV 向量化 MNNSiLu/MNNExpC8 (#4484)
*   **@Sherlockzhangjinge** — RVV 加速 Int8 函数 (#4359)
*   **@typer-J (XJJ)** — RVV 通用 opt 函数优化 (#4433)
*   **@ihb2032** — RVV pack/unpack 函数错误修复 (#4531)
*   **@Juude (Jinde.Song)** — MNNChat Android 0.8.3 模型市场准备 (#4405)
*   **@devin-lai (Devin Lai)** — Unique index 输出修复 (#4529)、TFLite binary activation 修复 (#4540)
*   **@EduMenges (Eduardo Menges Mattje)** — Vulkan 静态符号查找改为动态 (#4348)
*   **@DavidVentura (David Ventura)** — CPU sdot fallback 修复 (#4534)
*   **@katolikov (Artyom Katolikov)** — test_ci.sh 与 Vulkan/OpenCL 算子修复 (#4422)
*   **@zlaazlaa (zlaa)** — Stable Diffusion 3.5 Medium 支持 (#4072)
*   **@Voine** — VirtualFaceCapture-MNN 加入 demo 列表 (#4497)
*   **@Dugtx** — Diffusion sep-build 模式 llm 链接修复 (#4438)
*   **@Saisimon** — cstdint 头文件修复 (#4338)
*   **@Dts0 (dream zhao)** — tokenizer.hpp cstdint 修复 (#4373)
*   **@tumuyan** — cpufreq policy 错误日志修复 (#4400)

---

# MNN 3.6.0 Release Notes

**Release Date**: June 2026

📦 **Install**: `pip install mnn==3.6.0` ｜ [PyPI](https://pypi.org/project/mnn/3.6.0/)

## 📌 Overview

MNN 3.6.0 focuses on three core themes: **Multimodal LLM Model Expansion & Inference Acceleration**, **Deep Multi-Backend GPU Optimization**, and **Low-bit Quantization & Hardware Adaptation Enhancement**:

*   **Multimodal LLM Model Expansion & Inference Acceleration**: Added support for Gemma4 (text/vision/audio), Qwen3.5, Qwen3-VL DeepStack, and Hymt2 models; introduced dflash speculative decoding; Omni Thinker/Talker interleaved generation reduces TTFA by 2.7×; new fp16 storage for weight-quant scale/zero-point; refactored KV Cache reuse via `onClone`.
*   **Deep Multi-Backend GPU Optimization**: Vulkan adds VulkanConv1x1CoopA8 (W8A8 coopMat) and no-subgroup fallback paths; OpenCL adds heuristic GEMM/GEMV scheduling, int4 image1d_buffer_t optimization, native TopKV2 and 2/3-bit weight quantization; Metal optimizes Conv1x1 branch selection; CUDA supports Blackwell (sm_120) and optimizes Eagle speculative decoding; QNN supports Qwen3.5 and multi-process parallel offline graph export.
*   **Low-bit Quantization & Hardware Adaptation Enhancement**: Full-stack 2/3-bit weight quantization across CPU and GPU (ARM low-bit quant kernels + OpenCL/Metal/Vulkan); deep optimization of ARMv8.2/v8.6 decode path; KleidiAI upgraded to v1.16.0; new RVV-vectorized Int8/Norm/Gelu/SiLu/ExpC8 kernels; new fused MNNFusedGatedDelta (LinearAttention prefill); new LinearAttention L=1 decode fast path.

---

## 🚀 Highlights

*   **Gemma4 Multimodal Support**: Full inference support for Gemma4 text, vision, and audio
*   **Wan2.1-T2V-1.3B Video Generation**: Export and inference support for Wan2.1-T2V-1.3B text-to-video model
*   **LLM Inference Acceleration**: New dflash speculative decoding; Omni Thinker/Talker interleaved generation reduces TTFA by 2.7×
*   **Full-Stack 2/3-bit Weight Quantization**: 2/3-bit weight quantization across CPU (low-bit ARM kernels) and GPU (OpenCL/Metal/Vulkan)
*   **Vulkan W8A8 coopMat**: New VulkanConv1x1CoopA8 with dynamic-quant W8A8 + Cooperative Matrix

---

## ✨ New Features

### LLM / VLM
*   **Gemma4 Model Support**: Full text + vision + audio multimodal inference
*   **dflash Speculative Decoding**: New dflash speculative decoding
*   **Omni Interleaved Thinker/Talker**: TTFA reduced by 2.7×
*   **fp16 Scale/Zero-Point Storage**: weight-quant scale/zero-point can be stored in fp16
*   **KV Cache `onClone` Refactor**: Unified KV Cache reuse via `onClone` across backends
*   **LLM log save to context**: Persist LLM inference logs to context
*   **Treat linear_attention as full attention**: Performance optimization path

### Model Export / Model Support
*   **Gemma4 Export**: Multimodal export support
*   **Stable Diffusion 3.5 Medium**
*   **Wan2.1-T2V-1.3B**: Video generation model export and inference

### GPU Backends
*   **Vulkan VulkanConv1x1CoopA8**: Dynamic-quant W8A8 + Cooperative Matrix conv
*   **Vulkan LinearAttention no-subgroup fallback**
*   **Vulkan gemv decode no-subgroup fallback**
*   **Vulkan persistent states across responses**
*   **Vulkan LayerNorm `onClone`**: Move opt pipeline creation into constructor
*   **OpenCL Heuristic GEMM/GEMV Scheduling**: Heuristic guidance for quantized GEMM/GEMV + XGEMM FP local size
*   **OpenCL int4 GEMM/GEMV Optimization**: image1d_buffer_t-based int4 GEMM/GEMV, removing the extra `-8` during dequant
*   **OpenCL Native TopKV2**: For OpenCL IMAGE and Vulkan (image/buffer)
*   **OpenCL LinearAttention Chunk Branch Optimization**
*   **OpenCL/Metal/Vulkan 2/3-bit Weight Quantization**: Unified across GPU backends (CPU supported via low-bit ARM quant kernels)
*   **Metal Conv1x1 Branch Selection Optimization**
*   **CUDA Blackwell (sm_120) Support**
*   **CUDA Eagle Speculative Decoding Optimization**
*   **QNN Qwen3.5 Support** with abnormal quant params and vision model fixes
*   **QNN Multi-Process Parallel Offline Graph Export**
*   **QNN Binary Op Optimization**: Speeds up vision models

### CPU Backend
*   **RVV Int8 Acceleration**
*   **RVV General Opt Functions**: MNNNorm, MNNGelu, MNNSiLu, MNNExpC8
*   **ARM v8.2/v8.6 Decode Optimization**
*   **LinearAttention prefill fused kernel**: MNNFusedGatedDelta
*   **LinearAttention L=1 Decode Fast Path**
*   **Low-bit ARM Quant Kernels**: CPU 2/3-bit and other low-bit ARM quant kernel support
*   **fp16 alpha lowMemory Optimization**: Skip fp32 round-trip
*   **int4 Reorder Optimization**: Reduce non-fast-path overhead

### Build / Testing / Tools
*   **Release Workflow Upgrade**: LLM/OMNI, Harmony, Android 16K page size
*   **test_ci.sh**: Unified CI entry script
*   **CMake-generated podspec**
*   **GEMM Benchmarks**: FP32 / INT8 / Block32-INT4
*   **GEMV Bandwidth Benchmarks**: CPU/Metal w8/w4/w3/w2

### Apps
*   **MNNChat Android 0.8.3 Model Catalog Preparation**

---

## 🐛 Bug Fixes

### LLM / VLM
*   **Qwen3.5 head_dim Auto-Detection**: Fixed head_dim auto-detection for gated attention models
*   **LinearAttention prefix disk cache**: Fixed accuracy and hybrid model support
*   **Crash on corrupted prefix kvcache**: Fixed crash on corrupted prefix kvcache and `size_t` narrowing in `expandKVCacheInDisk`
*   **Speculative decoding cache mismatch**: Fixed cache mismatch and mrope support
*   **Vision encoder memory leak and latency growth**: Fixed
*   **`llm_demo` linear-attention state reset**: Reset linear-attention states in `llm_demo`
*   **kvmeta memory leak**: Fixed
*   **stop_ids fallback**: Fixed list-form `eos_token_id` stop_ids fallback
*   **GPTQ weight_dict reinitialization**: Fixed
*   **GTE embedding/reranker RoPE**: Restored zeroed RoPE `inv_freq` buffer
*   **Gemma3 dual-RoPE export & Qwen3-VL bilinear sampling**: Fixed
*   **Gemma4 export**: Fixed crash with HF transformers>=5.5.4 pure KV sharing; isolated `k_eq_v` and head_dim auto-detection via mapper
*   **Hymt2 qk norm export**: Fixed
*   **Qwen2.5-Omni audio preprocessing**: Fixed edge cases
*   **Qwen3-VL DeepStack dtype mismatch**: Fixed
*   **Qwen3-VL repeating output**: Fixed via per-RTM `KVCACHE_INFO` meta
*   **`_maskQK` gapLen**: Moved gapLen computation after null mask early return

### GPU Backends
*   **Vulkan attention and other op fixes**
*   **Vulkan dynamic symbol lookup**
*   **Metal output reuse offset error**: Fixed
*   **Metal API compatibility protection**: Added
*   **OpenCL Attention OOB access**: Fixed OOB access before memory reallocation
*   **OpenCL LinearAttention GQA bug**: Fixed
*   **CUDA Softmax echo / FP16FP32Mix**: Fixed echo artifacts and MatMul precisionType in FP16FP32Mix mode
*   **CUDA Windows / `MNN_CUDA_NATIVE_ARCH` Build**: Fixed CMake build failures
*   **QNN MSVC Build**: Fixed QNN backend and `MNN2QNNModel` MSVC compile errors

### CPU Backend
*   **VNNI w4 < 64 Compute Error**: Fixed
*   **KleidiAI int4 dequant**: Upgraded to v1.16.0 to fix incorrect block-quantized int4 dequant
*   **KleidiAI Path Mapping**: Fixed when using MNN as a subproject
*   **KleidiAI depthwise SME2 guard**: Added
*   **RVV pack/unpack Errors**: Fixed
*   **CPU sdot Fallback**: Don't force fp32 fallback outside `Memory_Low` on CPUs without sdot
*   **Missing cstdint Include**: Added
*   **cpufreq Scan Error Log**: Fixed spurious error log when scanning cpufreq policy directories

### Ops / Converter
*   **Unique Op**: Fixed to produce sorted output per ONNX spec; fixed index outputs
*   **TFLite binary activation conversion**: Fixed
*   **ONNX Resize / Einsum**: Fixed ONNX Resize converter regressions and Einsum 4D fast-path for non-aligned batch dims
*   **cumsum int Type Mismatch**: Fixed cumsum int type mismatch and virtual type command bug

### Diffusion
*   **SD1.5 resize cache warmup**: Fixed
*   **`tokenizer.hpp` cstdint**: Added include to fix `uint8_t` undeclared
*   **Diffusion llm link in sep-build mode**: Fixed

### Build / Embedding / Misc
*   **`embedding_demo` crash**: Fixed crash caused by `mMeta->add==0`
*   **PyMNN llm submodule**: Made llm submodule import optional
*   **Windows PyMNN .pdb bloat**: Removed `/Zi` and `/DEBUG` flags on Windows
*   **dflash Windows MSVC Build**: Fixed dflash Windows MSVC compile error
*   **General compilation fixes**: Fixed compilation issues introduced by commit f40b3d2a7

---

## 📚 Other Improvements

*   **AGENTS.md**: Renamed `CLAUDE.md` to `AGENTS.md` and forwarded via include
*   **VirtualFaceCapture-MNN added to demo list**
*   **`MNN_CUDA_NATIVE_ARCH` Documentation**
*   **fp16 scale schema sync**
*   **Code cleanup: removed `MNN_CPU_DEQUANT_GEMM`**: Deleted deprecated MNN_CPU_DEQUANT_GEMM code
*   **Code cleanup: removed `WEIGHT_DEQUANT_GEMM` macro**: Removed deprecated macro

---

## 🙏 Acknowledgements

We sincerely thank all contributors for their valuable contributions to this release (listed by GitHub login):

*   **@jxt1234** — ARMv8.2/v8.6 decode optimization (#4389), Metal Conv1x1 branch selection (#4520), Metal output reuse offset fix (#4365), VNNI w4 < 64 fix (#4366), int4 reorder optimization (#4511), Geometry cumsum virtual type fix (#4391)
*   **@Fatemanx (xly)** — Wan2.1-T2V-1.3B video model (#4469), CUDA Blackwell sm_120 (#4467), Qwen2.5-Omni audio edge cases (#4459), Qwen3-VL DeepStack dtype fix (#4504), Hymt2 qk norm export (#4490), SD1.5 resize cache warmup (#4530), PyMNN llm optional import (#4468), Converter Einsum 4D fast-path (#4466)
*   **@EricMoin** — Omni Thinker/Talker interleaved generation, TTFA ↓2.7× (#4397)
*   **@luckysmg** — LinearAttention L=1 decode fast path (#4356), MNNFusedGatedDelta prefill fused kernel (#4434), LinearAttention prefix disk cache accuracy fix (#4495), prefix kvcache crash / expandKVCacheInDisk size_t fix (#4413)
*   **@hebangwen** — Vision encoder memory leak / latency growth fix (#4441), `llm_demo` linear-attention state reset (#4424)
*   **@LudovicoYIN (YinHanke)** — ONNX Resize / Einsum lowering fix (#4254)
*   **@cgisky1980 (顾真牛)** — CUDA Softmax echo / FP16FP32Mix fix (#4476), CUDA Windows / MNN_CUDA_NATIVE_ARCH build fix (#4477)
*   **@yydhYYDH (YYDH)** — GPTQ loader weight_dict reinit fix (#4411), QNN multi-process parallel offline graph export (#4414)
*   **@HenryDen** — KleidiAI v1.16.0 upgrade fixing int4 dequant (#4474), KleidiAI subproject path mapping fix (#4542)
*   **@luke-opus (liushengtong)** — RVV vectorized MNNNorm/MNNGelu (#4508), RVV vectorized MNNSiLu/MNNExpC8 (#4484)
*   **@Sherlockzhangjinge** — RVV accelerated Int8 functions (#4359)
*   **@typer-J (XJJ)** — RVV common opt functions optimization (#4433)
*   **@ihb2032** — RVV pack/unpack functions fix (#4531)
*   **@Juude (Jinde.Song)** — MNNChat Android 0.8.3 model catalog preparation (#4405)
*   **@devin-lai (Devin Lai)** — Unique index outputs fix (#4529), TFLite binary activation fix (#4540)
*   **@EduMenges (Eduardo Menges Mattje)** — Vulkan static → dynamic symbol lookup (#4348)
*   **@DavidVentura (David Ventura)** — CPU sdot fallback fix (#4534)
*   **@katolikov (Artyom Katolikov)** — `test_ci.sh` and Vulkan/OpenCL op fixes (#4422)
*   **@zlaazlaa (zlaa)** — Stable Diffusion 3.5 Medium support (#4072)
*   **@Voine** — Added VirtualFaceCapture-MNN to demo list (#4497)
*   **@Dugtx** — Diffusion sep-build mode llm link fix (#4438)
*   **@Saisimon** — cstdint header fix (#4338)
*   **@Dts0 (dream zhao)** — `tokenizer.hpp` cstdint fix (#4373)
*   **@tumuyan** — cpufreq policy error log fix (#4400)

---

**Full Changelog**: https://github.com/alibaba/MNN/compare/3.5.0...3.6.0