Starstruk Logo MarkStarstruk

v1.0.0

QuentinFuxa/WhisperLiveKitv1.0.0May 25, 2026by LauraGPT

AI Summary

The first official release of the SenseVoiceSmall model, providing multilingual speech understanding, emotion recognition, and audio event detection.

Key Highlights

  • Supports Mandarin, Cantonese, English, Japanese, and Korean.
  • Includes speech emotion recognition and audio event detection.
  • Low-latency architecture running 5-15x faster than Whisper models.
  • Optional composition with FunASR's FSMN-VAD and CAM++ models.

New Features

  • Multilingual ASR and language identification
  • Speech emotion recognition
  • Audio event detection (music, applause, laughter, etc.)
  • Non-autoregressive low-latency inference

Full Release Notes

## SenseVoice v1.0.0

The first official release of the SenseVoiceSmall checkpoint for multilingual speech understanding.

### Released checkpoint scope

SenseVoiceSmall supports ASR and language identification for Mandarin, Cantonese, English, Japanese, and Korean. It also returns emotion and audio-event tags. The broader SenseVoice research reports training on more than 400,000 hours and support for more than 50 languages; those research results should not be read as the released SenseVoiceSmall checkpoint's language list.

### Highlights

- ASR and language identification for the five released languages.
- Speech emotion recognition with happy, sad, angry, and neutral labels.
- Audio event detection for events such as background music, applause, laughter, crying, coughing, and sneezing.
- A non-autoregressive architecture for low-latency inference. In the benchmark configuration documented in the repository, SenseVoiceSmall runs more than 5 times faster than Whisper-Small and 15 times faster than Whisper-Large at a similar parameter count.
- Optional composition with FunASR's separate FSMN-VAD and CAM++ models for segmented, speaker-attributed workflows. Speaker diarization is not a native SenseVoiceSmall checkpoint output.

### Quick start

```python
from funasr import AutoModel
from funasr.utils.postprocess_utils import rich_transcription_postprocess

model = AutoModel(
    model="iic/SenseVoiceSmall",
    trust_remote_code=True,
    vad_model="fsmn-vad",
    vad_kwargs={"max_single_segment_time": 30000},
    device="cuda:0",
)
result = model.generate(
    input="audio.wav",
    cache={},
    language="auto",
    use_itn=True,
    batch_size_s=60,
    merge_vad=True,
)
print(rich_transcription_postprocess(result[0]["text"]))
```

### Model

| Model | Languages | Parameters | Download |
|---|---|---:|---|
| SenseVoiceSmall | Mandarin, Cantonese, English, Japanese, Korean | 234M | [ModelScope](https://modelscope.cn/models/iic/SenseVoiceSmall) / [Hugging Face](https://huggingface.co/FunAudioLLM/SenseVoiceSmall) |

### Links

- [Repository and benchmark details](https://github.com/FunAudioLLM/SenseVoice)
- [FunAudioLLM paper](https://arxiv.org/abs/2407.04051)
- [ModelScope demo](https://modelscope.cn/studios/iic/SenseVoice)
- [Hugging Face Space](https://huggingface.co/spaces/FunAudioLLM/SenseVoice)
- [FunASR toolkit](https://github.com/modelscope/FunASR)