runtime-llamacpp-v0.2.0

# FunASR llama.cpp runtime v0.2.0

Prebuilt, self-contained FunASR llama.cpp / GGUF binaries for Fun-ASR-Nano, SenseVoiceSmall, and Paraformer, with built-in FSMN-VAD. No Python ASR runtime or local C++ build is required.

## Highlights

- SRT subtitle output is available across Fun-ASR-Nano, SenseVoiceSmall, and Paraformer. Pass `--srt` to emit timestamped subtitle cues; the default plain-text output remains unchanged.
- The bundled llama.cpp revision refreshes Vulkan submission batching for smaller AMD GPUs and improves `VK_ERROR_DEVICE_LOST` diagnostics.
- AMD Vulkan troubleshooting now documents `GGML_VK_MAX_NODES_PER_SUBMIT` and `GGML_VK_SERIALIZE_SUBMISSIONS`, plus a CPU fallback. These changes improve diagnostics and batching behavior, but this release does not claim that every vendor-driver crash is fixed; please report the GPU model, driver version, command, and full log if a device still fails.

## Download

Nine platform archives are attached:

- Linux: `linux-arm64`, `linux-x64`, `linux-x64-avx2`, `linux-x64-vulkan`
- macOS: `macos-arm64`
- Windows: `windows-x64`, `windows-x64-avx2`, `windows-x64-vulkan`, `windows-x64-cuda`

Use the default x64 archive for maximum CPU compatibility. Choose x64-avx2 on CPUs with AVX2, FMA, F16C, and BMI2. The Vulkan archives require a working Vulkan driver/ICD and currently accelerate the SenseVoiceSmall graph with `llama-funasr-sensevoice ... --backend vulkan`. The Windows CUDA archive targets CUDA architecture 86 and requires a compatible NVIDIA driver; build from source for other GPU architectures.

## Quick Start

Download the default quantized model with:

```bash
bash download-funasr-model.sh <sensevoice|paraformer|nano>
```

The helper requires the Hugging Face CLI (`pip install -U huggingface_hub`). Then run `llama-funasr-cli`, `llama-funasr-sensevoice`, or `llama-funasr-paraformer`.

Full documentation: https://github.com/modelscope/FunASR/blob/runtime-llamacpp-v0.2.0/runtime/llama.cpp/README.md