runtime-llamacpp-v0.2.3

modelscope/FunASRruntime-llamacpp-v0.2.3Aug 28, 2026by github-actions[bot]

AI Summary

This release enhances diagnostic capabilities for the llama.cpp runtime by extending SenseVoice error reporting with granular stage boundaries, allowing users to pinpoint failures during backend initialization, model loading, or graph execution. It also prepares Windows Vulkan packages for specific AMD GPU retesting.

Key Highlights

Extended SenseVoice diagnostics with new failure stage boundaries: 'model ready', 'graph allocated', and 'compute starting'.
Improved error reporting for CUDA and Vulkan backends to narrow down failure points.
Windows Vulkan packages ready for Radeon 780M and RX 9070 XT retesting.
Built from commit 820f1c64a3123112a3099bc3fdc373dafa381768 with successful platform builds.

New Features

Granular failure diagnostics for SenseVoice runtime
Enhanced backend initialization error handling
Windows Vulkan support for specific AMD GPUs

Full Release Notes

Prebuilt self-contained binaries for the FunASR llama.cpp / GGUF runtime: SenseVoice, Paraformer, and Fun-ASR-Nano with built-in FSMN-VAD.

## v0.2.3

- Extends SenseVoice diagnostics beyond backend initialization with flushed `model ready`, `graph allocated`, and `compute starting` stage boundaries.
- These boundaries separate model loading, graph allocation, and graph execution failures after a CUDA or Vulkan backend reports ready.
- Built from exact commit `820f1c64a3123112a3099bc3fdc373dafa381768`; all nine platform builds and the release job succeeded.
- The Windows Vulkan package is ready for Radeon 780M and RX 9070 XT retesting. This diagnostic release does **not** claim to fix the AMD Windows `0xC0000005` crash.
- Android/Mali remains outside the official prebuilt and continuously validated targets.

For `--backend vulkan` or `--backend cuda`, the last stderr boundary reached narrows the failure stage:

- Before `backend ready`: backend selection, initialization, or default buffer-type resolution.
- `backend ready` without `model ready`: model loading.
- `model ready` without `graph allocated`: graph allocation.
- `graph allocated` without `compute starting`: preparation immediately before graph execution.
- `compute starting`: the failure is inside graph execution or a lower driver/backend layer.

## Quick start

Download the default quantized model with:

```bash
bash download-funasr-model.sh <sensevoice|paraformer|nano>
```

The helper requires the Hugging Face CLI (`pip install -U huggingface_hub`). Then run `llama-funasr-cli`, `llama-funasr-sensevoice`, or `llama-funasr-paraformer`.

Use the default x64 asset for maximum CPU compatibility. Use x64-avx2 on CPUs with AVX2/FMA/F16C/BMI2. Vulkan assets require a working Vulkan driver/ICD. The Windows CUDA asset targets CUDA architecture 86 and requires a compatible NVIDIA driver; build from source for other GPU architectures.

## SHA-256

```text
54abd8dbdcfc200a64a62b657f40d6e4c123c423988707058ce2771096f2921a funasr-llamacpp-linux-arm64.tar.gz
139616c8adf6b5b306cefcfb1813fabc0f4b7727d655d79102163c5c146df200 funasr-llamacpp-linux-x64-avx2.tar.gz
ea8eeb9e334598a59c4cdb2dba282855b9a3b909cd05684d30250914f418bf43 funasr-llamacpp-linux-x64-vulkan.tar.gz
5b7cf0c2339ab76d8a56594455b1607e02f1eab34392b2543b95a5273413f088 funasr-llamacpp-linux-x64.tar.gz
532da7810e9311b275e3dd2e6f693811421ab49881e0194a788a6db189df4000 funasr-llamacpp-macos-arm64.tar.gz
55fa78e2a7522e54ead84532452afe40041a60014ba0261add611124897a7fe8 funasr-llamacpp-windows-x64-avx2.zip
ee25ed9cb4dff763f94de7db5812dd69fd7e748d2940dfbf82f29c408a4d419e funasr-llamacpp-windows-x64-cuda.zip
1c90daf2292fcf9041c14a8017be65a05acc8e42443ee0bbc8a8dfddb78cbdc7 funasr-llamacpp-windows-x64-vulkan.zip
54520b705ffedfc2b3505f337bfff64d7890e28b660e78e7255b27e1dc34dc0e funasr-llamacpp-windows-x64.zip
```

Build evidence: https://github.com/modelscope/FunASR/actions/runs/33197306623

Docs: https://github.com/modelscope/FunASR/blob/runtime-llamacpp-v0.2.3/runtime/llama.cpp/README.md