r1.4.0

the-dev-tools/dev-toolsr1.4.0Mar 15, 2023by zxcd

AI Summary

This update focuses on expanding multi-lingual support (Cantonese), adding new models like VITS and StarGANv2-VC, and improving inference deployment capabilities (Static, ONNX).

Key Highlights

Added Cantonese TTS support with Static inference and CLI
Added VITS inference pipeline and diffusion module for DiffSinger
Added StarGANv2-VC model scripts and synthesis
Added Wfst decoder and batch recognizer decode support
Removed deprecated Fluid API in ASR

Breaking Changes

Removed fluid API in ASR

New Features

Wav2vec2-zh finetune pipeline
Cantonese TTS dygraph to static inference
VITS inference pipeline
StarGANv2-VC model scripts
Wfst decoder
Batch recognizer decode
PaddleAudio refactor (replaced kaldi fbank)
TTS ARM Linux C++ Demo
SSML support for Chinese

Full Release Notes

# S2T
- Add wav2vec2-zh finetune pipeline. https://github.com/PaddlePaddle/PaddleSpeech/pull/3012 https://github.com/PaddlePaddle/PaddleSpeech/pull/2916  by @zxcd 
- Fix some bugs in Whisper. https://github.com/PaddlePaddle/PaddleSpeech/pull/2900  https://github.com/PaddlePaddle/PaddleSpeech/pull/2828https://github.com/PaddlePaddle/PaddleSpeech/pull/2825  by @zxcd 
- Add code-switch asr tal_cs recipe. https://github.com/PaddlePaddle/PaddleSpeech/pull/2816 https://github.com/PaddlePaddle/PaddleSpeech/pull/2796 by @zxcd 


# T2S
- Add dygraph to static、PaddleInference、Paddle2ONNX and ONNXRuntime Infer for Cantonese TTS. https://github.com/PaddlePaddle/PaddleSpeech/pull/2990 by @JiehangXie 
- Add Cantonese test examples. https://github.com/PaddlePaddle/PaddleSpeech/pull/2937 by @JiehangXie
- Add VITS inference pipeline. https://github.com/PaddlePaddle/PaddleSpeech/pull/3002 https://github.com/PaddlePaddle/PaddleSpeech/pull/2972  https://github.com/PaddlePaddle/PaddleSpeech/pull/2883 by @yt605155624 
- Rearrange encoder_infer param's order. https://github.com/PaddlePaddle/PaddleSpeech/pull/2983 by @443127316 
- Add male speaker and Chinese-English mix ONNXRuntime infer in CLI. https://github.com/PaddlePaddle/PaddleSpeech/pull/2945 by @lym0302 
- Add Cantonese TTS example. https://github.com/PaddlePaddle/PaddleSpeech/pull/2950  https://github.com/PaddlePaddle/PaddleSpeech/pull/2927 https://github.com/PaddlePaddle/PaddleSpeech/pull/2924 https://github.com/PaddlePaddle/PaddleSpeech/pull/2907  https://github.com/PaddlePaddle/PaddleSpeech/pull/2899  by @WongLaw 
- Fix PWGAN TIPC. https://github.com/PaddlePaddle/PaddleSpeech/pull/2882 by @yt605155624 
- Add a case in not_erhua. https://github.com/PaddlePaddle/PaddleSpeech/pull/2863 by @QuanZ9
- Fix data prepare for PaddleSlim PTQ of TTS. https://github.com/PaddlePaddle/PaddleSpeech/pull/2862 by @yt605155624 
- Avoid using variable "attn_loss" before assignment. https://github.com/PaddlePaddle/PaddleSpeech/pull/2860 by @hopingZ
- add soft link for shell in example, Add skip_copy_wave in norm stage of GANVocoders to save disk. https://github.com/PaddlePaddle/PaddleSpeech/pull/2851 by @yt605155624 
- Optimize the training of  VITS.  https://github.com/PaddlePaddle/PaddleSpeech/pull/2843 https://github.com/PaddlePaddle/PaddleSpeech/pull/2809  https://github.com/PaddlePaddle/PaddleSpeech/pull/2791 https://github.com/PaddlePaddle/PaddleSpeech/pull/2770 by @WongLaw 
- Add StarGANv2-VC model scripts and synthsize scripts. https://github.com/PaddlePaddle/PaddleSpeech/pull/2842 by @yt605155624 
- Add diffusion module for training diffsinger. https://github.com/PaddlePaddle/PaddleSpeech/pull/2868  https://github.com/PaddlePaddle/PaddleSpeech/pull/2832 by @HighCWu 
- Fix some Text Frontend bugs. https://github.com/PaddlePaddle/PaddleSpeech/pull/2831 by @yt605155624 
- For mixed Chinese and English speech synthesis, add SSML support for Chinese. https://github.com/PaddlePaddle/PaddleSpeech/pull/2830 by @jindongyi011039
- Add mkldnn and trt config for TTS Inference. https://github.com/PaddlePaddle/PaddleSpeech/pull/2748 by @yt605155624 
- Fix dygraph to static for tacotron2. https://github.com/PaddlePaddle/PaddleSpeech/pull/2426 by @yt605155624 

# Server
- Add  static infer for multi-spk tts. https://github.com/PaddlePaddle/PaddleSpeech/pull/2779 by @lym0302 

# Engine
- Add wfst decoder. https://github.com/PaddlePaddle/PaddleSpeech/pull/2886 by @SmileGoat 
- Add batch recognizer decode. https://github.com/PaddlePaddle/PaddleSpeech/pull/2866 by @SmileGoat 
- Add nnet prob cache && make 2 thread decode work. https://github.com/PaddlePaddle/PaddleSpeech/pull/2769 by @SmileGoat 
- Engine directory refactor. https://github.com/PaddlePaddle/PaddleSpeech/pull/2746 by @SmileGoat 
- Fix openfst download error. https://github.com/PaddlePaddle/PaddleSpeech/pull/2742 by @SmileGoat 

# Audio
- Replace kaldi fbank with kaldi-native-fbank in paddleaudio. https://github.com/PaddlePaddle/PaddleSpeech/pull/2799 by @SmileGoat 
- Fix load paddleaudio fail. https://github.com/PaddlePaddle/PaddleSpeech/pull/2815 by @SmileGoat 
- Update paddleaudio readme. https://github.com/PaddlePaddle/PaddleSpeech/pull/2801 by @SmileGoat 

# Demos 
- Add TTS ARM Linux C++ Demo. https://github.com/PaddlePaddle/PaddleSpeech/pull/2991 by @SwimmingTiger 
- Add Cantonese TTS in CLI. https://github.com/PaddlePaddle/PaddleSpeech/pull/2977 by @WongLaw 
- Add ONNXRuntime infer for Cantonese  TTS in CLI. https://github.com/PaddlePaddle/PaddleSpeech/pull/2990 by @JiehangXie 

# Docs
- Add u2pp_wenetspeech_static_quant to released_model.md. https://github.com/PaddlePaddle/PaddleSpeech/pull/2973 @zxcd
- Remove redundant dependencies and Fix some bugs in setup.py. https://github.com/PaddlePaddle/PaddleSpeech/pull/2970 https://github.com/PaddlePaddle/PaddleSpeech/pull/2871 https://github.com/PaddlePaddle/PaddleSpeech/pull/2867 https://github.com/PaddlePaddle/PaddleSpeech/pull/2853 https://github.com/PaddlePaddle/PaddleSpeech/pull/2771 https://github.com/PaddlePaddle/PaddleSpeech/pull/2767  https://github.com/PaddlePaddle/PaddleSpeech/pull/2764 by @yt605155624 

# Others
- Remove fluid API in ASR. https://github.com/PaddlePaddle/PaddleSpeech/pull/2944 https://github.com/PaddlePaddle/PaddleSpeech/pull/2859 https://github.com/PaddlePaddle/PaddleSpeech/pull/2852  by @zxcd 
- Add python simple adadelta optimizer. https://github.com/PaddlePaddle/PaddleSpeech/pull/2925 by @zxcd 
- Add encoding=utf-8 for text. https://github.com/PaddlePaddle/PaddleSpeech/pull/2896 by @zxcd https://github.com/PaddlePaddle/PaddleSpeech/pull/2865 by @yt605155624 
- Fix Tensor.numpy()[0] to float(Tensor) to adapt 0D. https://github.com/PaddlePaddle/PaddleSpeech/pull/2884 by @zhouwei25
- Fix libsndfile.so not found in ubuntu18-cpu/Dockerfile. https://github.com/PaddlePaddle/PaddleSpeech/pull/2763 by @linkec
- Fix AttributeError "module 'distutils' has no attribute 'ccompiler'" in setup.py in ctc_decoders. https://github.com/PaddlePaddle/PaddleSpeech/pull/2745  by @GreatV 


# New Contributors
* @GreatV made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/2745
* @linkec made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/2763
* @cxumol made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/2828
* @jindongyi011039 made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/2830
* @QuanZ9 made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/2863
* @hopingZ made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/2860
* @zhouwei25 made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/2884
* @EscaticZheng made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/2915
* @chinobing made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/2922
* @lance6716 made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/2924
* @443127316 made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/2983

**Full Changelog**: https://github.com/PaddlePaddle/PaddleSpeech/compare/r1.3.0...r1.4.0