Starstruk Logo MarkStarstruk

runtime-llamacpp-v0.2.0

moeru-ai/airiruntime-llamacpp-v0.2.0Aug 11, 2026by github-actions[bot]

AI Summary

This release provides prebuilt llama.cpp binaries for FunASR models (Fun-ASR-Nano, SenseVoiceSmall, Paraformer) with built-in FSMN-VAD. It introduces SRT subtitle output, improves Vulkan submission batching for AMD GPUs, and updates troubleshooting documentation.

Key Highlights

  • SRT subtitle output is now available across Fun-ASR-Nano, SenseVoiceSmall, and Paraformer models.
  • llama.cpp revision refreshes Vulkan submission batching for smaller AMD GPUs and improves diagnostics for VK_ERROR_DEVICE_LOST.
  • AMD Vulkan troubleshooting documentation now includes GGML_VK_MAX_NODES_PER_SUBMIT and GGML_VK_SERIALIZE_SUBMISSIONS settings.
  • CPU fallback options are now available for Vulkan troubleshooting.

New Features

  • SRT subtitle output support
  • Vulkan submission batching improvements for AMD GPUs
  • Enhanced AMD Vulkan troubleshooting documentation
  • CPU fallback availability

Full Release Notes

# FunASR llama.cpp runtime v0.2.0

Prebuilt, self-contained FunASR llama.cpp / GGUF binaries for Fun-ASR-Nano, SenseVoiceSmall, and Paraformer, with built-in FSMN-VAD. No Python ASR runtime or local C++ build is required.

## Highlights

- SRT subtitle output is available across Fun-ASR-Nano, SenseVoiceSmall, and Paraformer. Pass `--srt` to emit timestamped subtitle cues; the default plain-text output remains unchanged.
- The bundled llama.cpp revision refreshes Vulkan submission batching for smaller AMD GPUs and improves `VK_ERROR_DEVICE_LOST` diagnostics.
- AMD Vulkan troubleshooting now documents `GGML_VK_MAX_NODES_PER_SUBMIT` and `GGML_VK_SERIALIZE_SUBMISSIONS`, plus a CPU fallback. These changes improve diagnostics and batching behavior, but this release does not claim that every vendor-driver crash is fixed; please report the GPU model, driver version, command, and full log if a device still fails.

## Download

Nine platform archives are attached:

- Linux: `linux-arm64`, `linux-x64`, `linux-x64-avx2`, `linux-x64-vulkan`
- macOS: `macos-arm64`
- Windows: `windows-x64`, `windows-x64-avx2`, `windows-x64-vulkan`, `windows-x64-cuda`

Use the default x64 archive for maximum CPU compatibility. Choose x64-avx2 on CPUs with AVX2, FMA, F16C, and BMI2. The Vulkan archives require a working Vulkan driver/ICD and currently accelerate the SenseVoiceSmall graph with `llama-funasr-sensevoice ... --backend vulkan`. The Windows CUDA archive targets CUDA architecture 86 and requires a compatible NVIDIA driver; build from source for other GPU architectures.

## Quick Start

Download the default quantized model with:

```bash
bash download-funasr-model.sh <sensevoice|paraformer|nano>
```

The helper requires the Hugging Face CLI (`pip install -U huggingface_hub`). Then run `llama-funasr-cli`, `llama-funasr-sensevoice`, or `llama-funasr-paraformer`.

Full documentation: https://github.com/modelscope/FunASR/blob/runtime-llamacpp-v0.2.0/runtime/llama.cpp/README.md