Starstruk Logo MarkStarstruk

runtime-llamacpp-v0.2.0

TimoBolkart/FLAME-Universeruntime-llamacpp-v0.2.0Aug 11, 2026by github-actions[bot]

AI Summary

This release introduces prebuilt llama.cpp binaries for Fun-ASR-Nano, SenseVoiceSmall, and Paraformer models, removing the need for local C++ builds. It adds SRT subtitle output support and improves Vulkan submission batching for AMD GPUs.

Key Highlights

  • Prebuilt llama.cpp/GGUF binaries available for Fun-ASR-Nano, SenseVoiceSmall, and Paraformer.
  • New SRT subtitle output with timestamps available across all three models.
  • llama.cpp revision refreshes Vulkan submission batching for AMD GPUs and improves diagnostics.
  • AMD Vulkan troubleshooting documentation updated with new environment variables and CPU fallback options.

New Features

  • SRT subtitle output support
  • Vulkan backend acceleration for SenseVoiceSmall

Full Release Notes

# FunASR llama.cpp runtime v0.2.0

Prebuilt, self-contained FunASR llama.cpp / GGUF binaries for Fun-ASR-Nano, SenseVoiceSmall, and Paraformer, with built-in FSMN-VAD. No Python ASR runtime or local C++ build is required.

## Highlights

- SRT subtitle output is available across Fun-ASR-Nano, SenseVoiceSmall, and Paraformer. Pass `--srt` to emit timestamped subtitle cues; the default plain-text output remains unchanged.
- The bundled llama.cpp revision refreshes Vulkan submission batching for smaller AMD GPUs and improves `VK_ERROR_DEVICE_LOST` diagnostics.
- AMD Vulkan troubleshooting now documents `GGML_VK_MAX_NODES_PER_SUBMIT` and `GGML_VK_SERIALIZE_SUBMISSIONS`, plus a CPU fallback. These changes improve diagnostics and batching behavior, but this release does not claim that every vendor-driver crash is fixed; please report the GPU model, driver version, command, and full log if a device still fails.

## Download

Nine platform archives are attached:

- Linux: `linux-arm64`, `linux-x64`, `linux-x64-avx2`, `linux-x64-vulkan`
- macOS: `macos-arm64`
- Windows: `windows-x64`, `windows-x64-avx2`, `windows-x64-vulkan`, `windows-x64-cuda`

Use the default x64 archive for maximum CPU compatibility. Choose x64-avx2 on CPUs with AVX2, FMA, F16C, and BMI2. The Vulkan archives require a working Vulkan driver/ICD and currently accelerate the SenseVoiceSmall graph with `llama-funasr-sensevoice ... --backend vulkan`. The Windows CUDA archive targets CUDA architecture 86 and requires a compatible NVIDIA driver; build from source for other GPU architectures.

## Quick Start

Download the default quantized model with:

```bash
bash download-funasr-model.sh <sensevoice|paraformer|nano>
```

The helper requires the Hugging Face CLI (`pip install -U huggingface_hub`). Then run `llama-funasr-cli`, `llama-funasr-sensevoice`, or `llama-funasr-paraformer`.

Full documentation: https://github.com/modelscope/FunASR/blob/runtime-llamacpp-v0.2.0/runtime/llama.cpp/README.md