Starstruk Logo MarkStarstruk

runtime-llamacpp-v0.2.6

modelscope/FunASRruntime-llamacpp-v0.2.6Aug 30, 2026by github-actions[bot]

AI Summary

This runtime release provides prebuilt binaries for the llama.cpp backend, specifically adding support for the Blackwell GPU architecture and Vulkan backends on Windows and Linux.

Key Highlights

  • Support for CUDA Blackwell architecture (sm_120) on Windows.
  • Vulkan backend support for Linux and Windows.
  • Self-contained binaries with bundled cuBLAS libraries for easy deployment.
  • Includes SenseVoice, Paraformer, and Fun-ASR-Nano models with built-in FSMN-VAD.

New Features

  • Windows CUDA Blackwell runtime support.
  • Vulkan backend support.

Full Release Notes

Prebuilt binaries for the FunASR llama.cpp / GGUF runtime: SenseVoice, Paraformer, and Fun-ASR-Nano with built-in FSMN-VAD.

## Choose an asset

- Default CPU packages maximize compatibility. Use the x64 AVX2 package only on CPUs with AVX2/FMA/F16C/BMI2.
- Vulkan packages are available for Linux x64 and Windows x64. They require a working Vulkan driver/ICD and accelerate SenseVoiceSmall with `--backend vulkan`.
- `windows-x64-cuda` targets CUDA architecture 86.
- `windows-x64-cuda-blackwell` targets CUDA architecture 120 (`sm_120`) for RTX 50 / Blackwell GPUs.

Both Windows CUDA ZIPs bundle `cublas64_13.dll`, `cublasLt64_13.dll`, and the NVIDIA license, and statically link the MSVC runtime. Running them does not require a separate CUDA Toolkit or Visual C++ redistributable installation, but it does require a compatible NVIDIA driver. Build from source for other GPU architectures.

## Verification boundary

The signed tag resolves to exact commit `a57c05bfe2a91b5e0cb0983479634eba3e28ede5`. Actions run 33290555297 built and published all ten assets. Each public asset was downloaded again; archive integrity and GitHub-provided SHA-256 digests were verified. The Blackwell ZIP SHA-256 is `e32961a753f40888182f352fa551159c5165a6a77718ae4ade316aedfea4b1c2`.

The CUDA executables import only the system DLLs and bundled `cublas64_13.dll`; they do not dynamically import MSVC or OpenMP runtimes. The bundled cuBLAS DLLs match NVIDIA official `libcublas-windows-x86_64-13.3.0.5` byte-for-byte.

A successful CI build proves architecture 120 code generation and package integrity. It does **not** prove successful inference on physical Blackwell hardware. Hardware-specific reports remain open until users retest the matching archive.

Download the default quantized model with `bash download-funasr-model.sh <sensevoice|paraformer|nano>` (requires `pip install -U huggingface_hub`), then run the matching CLI. No Python ASR runtime or local compilation is required.

Documentation: https://github.com/modelscope/FunASR/blob/runtime-llamacpp-v0.2.6/runtime/llama.cpp/README.md