Starstruk Logo MarkStarstruk

runtime-llamacpp-v0.1.2

Mintplex-Labs/anything-llmruntime-llamacpp-v0.1.2Jun 21, 2026by LauraGPT

AI Summary

This runtime update introduces support for q8 GGUF models, which provide half the memory footprint of f16 models with comparable accuracy, while maintaining prebuilt binaries for all major platforms.

Key Highlights

  • New q8 GGUF model support
  • Prebuilt binaries for Linux, macOS, and Windows
  • Built-in FSMN-VAD
  • Strong performance for Chinese and Cantonese

New Features

  • q8 GGUF model support
  • SenseVoice and Paraformer support
  • Fun-ASR-Nano support
  • whisper.cpp-style on-device ASR

Full Release Notes

Prebuilt self-contained binaries for running **SenseVoice** (and Paraformer / Fun-ASR-Nano) locally with the FunASR llama.cpp / GGUF runtime — built-in FSMN-VAD, whisper.cpp-style on-device ASR, strong on Chinese & Cantonese.

**New:** q8 GGUF models are ~half the size of f16 with the same accuracy.

```bash
bash download-funasr-model.sh sensevoice ./gguf
llama-funasr-sensevoice -m ./gguf/sensevoice-small-q8.gguf --vad ./gguf/fsmn-vad.gguf -a audio.wav
```

No Python, no build. Linux (x64/arm64), macOS (arm64), Windows (x64). Docs: `runtime/llama.cpp/README.md`.