Starstruk Logo MarkStarstruk

2.4.0

alibaba/MNN2.4.0Mar 1, 2023by jxt1234

AI Summary

MNN 2.4.0 is a significant release introducing NNAPI int8 quantization support, online operator Fuse for OpenCL/Metal backends, and automated CI/CD via GitHub Actions. The release also includes major CUDA optimizations for Softmax/DepthwiseConv, OpenCL 3x3 convolution improvements, and removes deprecated LLVMJit/Codegen backends.

Key Highlights

  • NNAPI now supports int8 quantized models
  • OpenCL/Metal backends support online operator Fuse and code generation
  • CUDA backend optimized for Softmax/DepthwiseConv operators
  • OpenCL 3x3 convolution kernel performance significantly improved
  • Removed deprecated LLVMJit and C Codegen backends
  • Vulkan and OpenCL backends now support GELU operator

Breaking Changes

  • Removed LLVMJit and C Codegen backends - users relying on these backends will need to migrate
  • Test scripts renamed: fastTestOnnx.py → testMNNFromOnnx.py, fastTestTf.py → testMNNFromTf.py, fastTestTflite.py → testMNNFromTflite.py, fastTestTorch.py → testMNNFromTorch.py

New Features

  • NNAPI int8 quantization model support
  • OpenCL/Metal online operator Fuse and code generation
  • Python Wheel package building via cibuildwheel
  • GitHub Actions automated multi-end library and Whl package building
  • (Testing) CUDA backend quantized model inference support
  • GELU operator support for Vulkan and OpenCL backends
  • OpenCL low priority inference configuration
  • OpenCL async updateCache to reduce blocking
  • NetModule destructor gc function for memory optimization

Full Release Notes

一、新特性

- NNAPI 支持int8 量化模型；
- MNN OpenCL/Metal支持算子在线Fuse与代码生成；
- 支持使用cibuildwheel构建Python Wheel包；
- Github Action支持自动化构建多端库与Whl包；
- (测试中）CUDA后端支持量化模型运行

二、重构/优化
- CUDA优化Softmax/DepthwiseConv算子
- 优化 KernelSize = 3x3 的 OpenCL 卷积算子性能
- 优化了MaxPool/AvgPool的int8量化计算；
- 移除原来的LLVMJit, C等Codegen后端；
- 更新MNN.podspec, MNNBridge.podspec；
- 增加GELU模块Fuse为GELU算子的功能，Vulkan 和 OpenCL 后端支持 GELU 算子
- NetModule析构函数中增加gc函降低内存占用；
- OpenCL支持设置推理低优先级配置；
- OpenCL updateCache支持异步，降低阻塞时间；
- fastTestOnnx.py / fastTestTf.py / fastTestTflite.py / fastTestTorch.py 分别更名为 testMNNFromOnnx.py / testMNNFromTf.py / testMNNFromTflite.py / testMNNFromTorch.py
- Android Demo新增使用README文档；

三、Bugfix
- 修复Android Demo运行Crash的问题；
- 修复Metal中的onSync的Bug;
- 修复Metal多段模型推理的Bug；
- 修复在Windows下MNN Train的编译问题；
- 修复perm值非法时的Crash问题；
- 修复Pad的输入参数为负数时（此时等效为Crop），计算出错的问题
- 修正 Relu Int8 不支持非对称量化的问题
- 修正部分AVX2架构的机器上运行量化模型crash的问题
- 修正Module API 运行静态模型crash的问题
- 修正Winograd量化过程未使用相同变换矩阵的问题
- 修正Winograd量化计算多Batch输入错误的问题
- 修正 OpenCL Relu 算子在 AMD GPU 上段错误的问题
- 修正 OpenCL ROIPooling 算子实现错误