v3.1.1

casey/justv3.1.1Sep 22, 2024by Flux9665

AI Summary

A quality-focused release that significantly improves the vocoder component by training it end-to-end on synthetic spectrograms, resulting in significantly better audio output.

Key Highlights

Major audio quality improvement via end-to-end vocoder training.
Increased vocoder capacity (more parameters) without sacrificing inference speed.
Eliminated the gap between synthetic spectrograms and ground-truth audio.
Improved performance comparable to using expensive tires on a sports car.

New Features

End-to-end trained vocoder
Increased vocoder capacity

Full Release Notes

This is a minor release that does not change much, however the impact on the audio quality is pretty big. Previously, we trained the vocoder on spectrograms extracted from ground-truth audio files. Despite our best efforts of making the synthetic spectrograms as natural as possible, there always remained a gap between the synthetic spectrograms and the genuine spectrograms. We previously thought, that the impact of this was very small and unimportant, but upon experimentation we now noticed, that it actually does make a big difference. The vocoder is the last piece of the pipeline, it is what actually connects the generated spectrograms to your ears. So using a bad vocoder is a bit like putting cheap tires on an expensive sports car. 

We have finally fixed this by generating synthetic spectrograms and then training the vocoder on these. We also increased the capacity of the vocoder by giving it more parameters. The inference speed is still just as fast, but the quality is improved greatly. This release is part of the cleanup towards finishing my PhD thesis and is not associated with its own paper or proper evaluation besides internal pilot studies.

Update your code and run the model downloader script for an immediate major quality boost.

**Full Changelog**: https://github.com/DigitalPhonetics/IMS-Toucan/compare/v3.1...v3.1.1