Starstruk Logo MarkStarstruk

runtime-llamacpp-v0.1.2

PaddlePaddle/Paddleruntime-llamacpp-v0.1.2Jun 21, 2026by LauraGPT

AI Summary

This release introduces the ability to run SenseVoice (and Paraformer / Fun-ASR-Nano) locally using the llama.cpp/GGUF runtime. It emphasizes on-device ASR capabilities and introduces a new quantization option for smaller model sizes.

Key Highlights

  • Supports running SenseVoice, Paraformer, and Fun-ASR-Nano locally
  • Includes built-in FSMN-VAD and whisper.cpp-style on-device ASR
  • New q8 GGUF models offer half the size of f16 with similar accuracy

New Features

  • SenseVoice local execution
  • Paraformer and Fun-ASR-Nano local execution
  • FSMN-VAD integration
  • q8 GGUF model quantization support

Full Release Notes

Prebuilt self-contained binaries for running **SenseVoice** (and Paraformer / Fun-ASR-Nano) locally with the FunASR llama.cpp / GGUF runtime — built-in FSMN-VAD, whisper.cpp-style on-device ASR, strong on Chinese & Cantonese.

**New:** q8 GGUF models are ~half the size of f16 with the same accuracy.

```bash
bash download-funasr-model.sh sensevoice ./gguf
llama-funasr-sensevoice -m ./gguf/sensevoice-small-q8.gguf --vad ./gguf/fsmn-vad.gguf -a audio.wav
```

No Python, no build. Linux (x64/arm64), macOS (arm64), Windows (x64). Docs: `runtime/llama.cpp/README.md`.