3.0.0

pyannote/pyannote-audio3.0.0Sep 26, 2023by hbredin

Full Release Notes

## TL;DR

### Better pretrained pipeline and model

* Much better overlapping speech detection with _powerset_ [pyannote/segmentation-3.0](https://hf.co/pyannote/segmentation-3.0) 
* Much better speaker diarization performance with [pyannote/speaker-diarization-3.0](https://hf.co/pyannote/speaker-diarization-3.0)

| Benchmark  (DER %)       | [v2.1](https://hf.co/pyannote/speaker-diarization-2.1) | [v3.0](https://hf.co/pyannote/speaker-diarization-3.0) |
| ---------------------- | ---- | ---- | 
| AISHELL-4              |  14.1  |  12.3  |
| AliMeeting (channel 1) |  27.4  |  24.3  |
| AMI (IHM)              | 18.9  |  19.0  | 
| AMI (SDM)              |  27.1  |  22.2  | 
| AVA-AVD                |  -     |  49.1  | 
| DIHARD 3 (full)        |   26.9  |  21.7  | 
| MSDWild                |   -     |  24.6  | 
| REPERE (phase2)        |     8.2  |   7.8  | 
| VoxConverse (v0.3)     |    11.2  |  11.3  |

### Major breaking changes

* BREAKING: pipelines now run on CPU by default
   Use `pipeline.to(torch.device('cuda'))` to use GPU
* BREAKING: removed `SpeakerSegmentation` pipeline
   Use `SpeakerDiarization` pipeline instead
* BREAKING: removed support for `prodi.gy` recipes

## Full changelog

### Features and improvements

  - feat(pipeline): send pipeline to device with `pipeline.to(device)`
  - feat(pipeline): add `return_embeddings` option to `SpeakerDiarization` pipeline
  - feat(pipeline): make `segmentation_batch_size` and `embedding_batch_size` mutable in `SpeakerDiarization` pipeline (they now default to `1`)
  - feat(pipeline): add progress hook to pipelines
  - feat(task): add [powerset](https://www.isca-speech.org/archive/interspeech_2023/plaquet23_interspeech.html) support to `SpeakerDiarization` task
  - feat(task): add support for multi-task models
  - feat(task): add support for label scope in speaker diarization task
  - feat(task): add support for missing classes in multi-label segmentation task
  - feat(model): add segmentation model based on torchaudio self-supervised representation
  - feat(pipeline): check version compatibility at load time
  - improve(task): load metadata as tensors rather than pyannote.core instances
  - improve(task): improve error message on missing specifications

### Breaking changes

  - BREAKING(task): rename `Segmentation` task to `SpeakerDiarization`
  - BREAKING(pipeline): pipeline defaults to CPU (use `pipeline.to(device)`)
  - BREAKING(pipeline): remove `SpeakerSegmentation` pipeline (use `SpeakerDiarization` pipeline)
  - BREAKING(pipeline): remove `segmentation_duration` parameter from `SpeakerDiarization` pipeline (defaults to `duration` of segmentation model)
  - BREAKING(task): remove support for variable chunk duration for segmentation tasks
  - BREAKING(pipeline): remove support for `FINCHClustering` and `HiddenMarkovModelClustering`
  - BREAKING(setup): drop support for Python 3.7
  - BREAKING(io): channels are now 0-indexed (used to be 1-indexed)
  - BREAKING(io): multi-channel audio is no longer downmixed to mono by default.
    You should update how `pyannote.audio.core.io.Audio` is instantiated:
    * replace `Audio()` by `Audio(mono="downmix")`;
    * replace `Audio(mono=True)` by `Audio(mono="downmix")`;
    * replace `Audio(mono=False)` by `Audio()`.
  - BREAKING(model): get rid of (flaky) `Model.introspection`
    If, for some weird reason, you wrote some custom code based on that,
    you should instead rely on `Model.example_output`.
  - BREAKING(interactive): remove support for Prodigy recipes


### Fixes and improvements

  - fix(pipeline): fix reproducibility issue with Ampere CUDA devices
  - fix(pipeline): fix support for IOBase audio
  - fix(pipeline): fix corner case with no speaker
  - fix(train): prevent metadata preparation to happen twice
  - fix(task): fix support for "balance" option
  - improve(task): shorten and improve structure of Tensorboard tags

### Dependencies update

  - setup: switch to torch 2.0+, torchaudio 2.0+, soundfile 0.12+, lightning 2.0+, torchmetrics 0.11+
  - setup: switch to pyannote.core 5.0+, pyannote.database 5.0+, and pyannote.pipeline 3.0+
  - setup: switch to speechbrain 0.5.14+