runtime-llamacpp-v0.1.2

zaidmukaddam/sciraruntime-llamacpp-v0.1.2Jun 21, 2026by LauraGPT

AI Summary

Introduces prebuilt binaries for running local ASR (SenseVoice, Paraformer) using the llama.cpp runtime. This release introduces support for q8 GGUF models which offer half the file size with equivalent accuracy.

Key Highlights

New q8 GGUF models are ~half the size of f16 with same accuracy
Built-in FSMN-VAD and whisper.cpp-style on-device ASR
Support for Linux (x64/arm64), macOS (arm64), and Windows (x64)
No Python or build required

New Features

Added support for q8 GGUF models
Self-contained binaries for SenseVoice, Paraformer, and Fun-ASR-Nano

Full Release Notes

Prebuilt self-contained binaries for running **SenseVoice** (and Paraformer / Fun-ASR-Nano) locally with the FunASR llama.cpp / GGUF runtime — built-in FSMN-VAD, whisper.cpp-style on-device ASR, strong on Chinese & Cantonese.

**New:** q8 GGUF models are ~half the size of f16 with the same accuracy.

```bash
bash download-funasr-model.sh sensevoice ./gguf
llama-funasr-sensevoice -m ./gguf/sensevoice-small-q8.gguf --vad ./gguf/fsmn-vad.gguf -a audio.wav
```

No Python, no build. Linux (x64/arm64), macOS (arm64), Windows (x64). Docs: `runtime/llama.cpp/README.md`.