Starstruk Logo MarkStarstruk

r1.4.0

microsoft/playwright-clir1.4.0Mar 15, 2023by zxcd

AI Summary

This release adds comprehensive Cantonese TTS support including inference pipelines and CLI tools, alongside the introduction of the VITS inference pipeline and diffusion module for DiffSinger. It also refactors the engine directory and removes deprecated Fluid APIs from the ASR components.

Key Highlights

  • Add Cantonese TTS support (static inference, CLI, demos)
  • Add VITS inference pipeline and diffusion module
  • Add Wfst decoder and batch recognizer decode
  • Remove Fluid API from ASR
  • Replace kaldi fbank with kaldi-native-fbank

Breaking Changes

  • Removed fluid API in ASR

New Features

  • wav2vec2-zh finetune pipeline
  • Cantonese TTS
  • VITS inference pipeline
  • StarGANv2-VC model scripts
  • Wfst decoder
  • kaldi-native-fbank

Full Release Notes

# S2T
- Add wav2vec2-zh finetune pipeline. https://github.com/PaddlePaddle/PaddleSpeech/pull/3012 https://github.com/PaddlePaddle/PaddleSpeech/pull/2916  by @zxcd 
- Fix some bugs in Whisper. https://github.com/PaddlePaddle/PaddleSpeech/pull/2900  https://github.com/PaddlePaddle/PaddleSpeech/pull/2828https://github.com/PaddlePaddle/PaddleSpeech/pull/2825  by @zxcd 
- Add code-switch asr tal_cs recipe. https://github.com/PaddlePaddle/PaddleSpeech/pull/2816 https://github.com/PaddlePaddle/PaddleSpeech/pull/2796 by @zxcd 


# T2S
- Add dygraph to static、PaddleInference、Paddle2ONNX and ONNXRuntime Infer for Cantonese TTS. https://github.com/PaddlePaddle/PaddleSpeech/pull/2990 by @JiehangXie 
- Add Cantonese test examples. https://github.com/PaddlePaddle/PaddleSpeech/pull/2937 by @JiehangXie
- Add VITS inference pipeline. https://github.com/PaddlePaddle/PaddleSpeech/pull/3002 https://github.com/PaddlePaddle/PaddleSpeech/pull/2972  https://github.com/PaddlePaddle/PaddleSpeech/pull/2883 by @yt605155624 
- Rearrange encoder_infer param's order. https://github.com/PaddlePaddle/PaddleSpeech/pull/2983 by @443127316 
- Add male speaker and Chinese-English mix ONNXRuntime infer in CLI. https://github.com/PaddlePaddle/PaddleSpeech/pull/2945 by @lym0302 
- Add Cantonese TTS example. https://github.com/PaddlePaddle/PaddleSpeech/pull/2950  https://github.com/PaddlePaddle/PaddleSpeech/pull/2927 https://github.com/PaddlePaddle/PaddleSpeech/pull/2924 https://github.com/PaddlePaddle/PaddleSpeech/pull/2907  https://github.com/PaddlePaddle/PaddleSpeech/pull/2899  by @WongLaw 
- Fix PWGAN TIPC. https://github.com/PaddlePaddle/PaddleSpeech/pull/2882 by @yt605155624 
- Add a case in not_erhua. https://github.com/PaddlePaddle/PaddleSpeech/pull/2863 by @QuanZ9
- Fix data prepare for PaddleSlim PTQ of TTS. https://github.com/PaddlePaddle/PaddleSpeech/pull/2862 by @yt605155624 
- Avoid using variable "attn_loss" before assignment. https://github.com/PaddlePaddle/PaddleSpeech/pull/2860 by @hopingZ
- add soft link for shell in example, Add skip_copy_wave in norm stage of GANVocoders to save disk. https://github.com/PaddlePaddle/PaddleSpeech/pull/2851 by @yt605155624 
- Optimize the training of  VITS.  https://github.com/PaddlePaddle/PaddleSpeech/pull/2843 https://github.com/PaddlePaddle/PaddleSpeech/pull/2809  https://github.com/PaddlePaddle/PaddleSpeech/pull/2791 https://github.com/PaddlePaddle/PaddleSpeech/pull/2770 by @WongLaw 
- Add StarGANv2-VC model scripts and synthsize scripts. https://github.com/PaddlePaddle/PaddleSpeech/pull/2842 by @yt605155624 
- Add diffusion module for training diffsinger. https://github.com/PaddlePaddle/PaddleSpeech/pull/2868  https://github.com/PaddlePaddle/PaddleSpeech/pull/2832 by @HighCWu 
- Fix some Text Frontend bugs. https://github.com/PaddlePaddle/PaddleSpeech/pull/2831 by @yt605155624 
- For mixed Chinese and English speech synthesis, add SSML support for Chinese. https://github.com/PaddlePaddle/PaddleSpeech/pull/2830 by @jindongyi011039
- Add mkldnn and trt config for TTS Inference. https://github.com/PaddlePaddle/PaddleSpeech/pull/2748 by @yt605155624 
- Fix dygraph to static for tacotron2. https://github.com/PaddlePaddle/PaddleSpeech/pull/2426 by @yt605155624 

# Server
- Add  static infer for multi-spk tts. https://github.com/PaddlePaddle/PaddleSpeech/pull/2779 by @lym0302 

# Engine
- Add wfst decoder. https://github.com/PaddlePaddle/PaddleSpeech/pull/2886 by @SmileGoat 
- Add batch recognizer decode. https://github.com/PaddlePaddle/PaddleSpeech/pull/2866 by @SmileGoat 
- Add nnet prob cache && make 2 thread decode work. https://github.com/PaddlePaddle/PaddleSpeech/pull/2769 by @SmileGoat 
- Engine directory refactor. https://github.com/PaddlePaddle/PaddleSpeech/pull/2746 by @SmileGoat 
- Fix openfst download error. https://github.com/PaddlePaddle/PaddleSpeech/pull/2742 by @SmileGoat 

# Audio
- Replace kaldi fbank with kaldi-native-fbank in paddleaudio. https://github.com/PaddlePaddle/PaddleSpeech/pull/2799 by @SmileGoat 
- Fix load paddleaudio fail. https://github.com/PaddlePaddle/PaddleSpeech/pull/2815 by @SmileGoat 
- Update paddleaudio readme. https://github.com/PaddlePaddle/PaddleSpeech/pull/2801 by @SmileGoat 

# Demos 
- Add TTS ARM Linux C++ Demo. https://github.com/PaddlePaddle/PaddleSpeech/pull/2991 by @SwimmingTiger 
- Add Cantonese TTS in CLI. https://github.com/PaddlePaddle/PaddleSpeech/pull/2977 by @WongLaw 
- Add ONNXRuntime infer for Cantonese  TTS in CLI. https://github.com/PaddlePaddle/PaddleSpeech/pull/2990 by @JiehangXie 

# Docs
- Add u2pp_wenetspeech_static_quant to released_model.md. https://github.com/PaddlePaddle/PaddleSpeech/pull/2973 @zxcd
- Remove redundant dependencies and Fix some bugs in setup.py. https://github.com/PaddlePaddle/PaddleSpeech/pull/2970 https://github.com/PaddlePaddle/PaddleSpeech/pull/2871 https://github.com/PaddlePaddle/PaddleSpeech/pull/2867 https://github.com/PaddlePaddle/PaddleSpeech/pull/2853 https://github.com/PaddlePaddle/PaddleSpeech/pull/2771 https://github.com/PaddlePaddle/PaddleSpeech/pull/2767  https://github.com/PaddlePaddle/PaddleSpeech/pull/2764 by @yt605155624 

# Others
- Remove fluid API in ASR. https://github.com/PaddlePaddle/PaddleSpeech/pull/2944 https://github.com/PaddlePaddle/PaddleSpeech/pull/2859 https://github.com/PaddlePaddle/PaddleSpeech/pull/2852  by @zxcd 
- Add python simple adadelta optimizer. https://github.com/PaddlePaddle/PaddleSpeech/pull/2925 by @zxcd 
- Add encoding=utf-8 for text. https://github.com/PaddlePaddle/PaddleSpeech/pull/2896 by @zxcd https://github.com/PaddlePaddle/PaddleSpeech/pull/2865 by @yt605155624 
- Fix Tensor.numpy()[0] to float(Tensor) to adapt 0D. https://github.com/PaddlePaddle/PaddleSpeech/pull/2884 by @zhouwei25
- Fix libsndfile.so not found in ubuntu18-cpu/Dockerfile. https://github.com/PaddlePaddle/PaddleSpeech/pull/2763 by @linkec
- Fix AttributeError "module 'distutils' has no attribute 'ccompiler'" in setup.py in ctc_decoders. https://github.com/PaddlePaddle/PaddleSpeech/pull/2745  by @GreatV 


# New Contributors
* @GreatV made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/2745
* @linkec made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/2763
* @cxumol made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/2828
* @jindongyi011039 made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/2830
* @QuanZ9 made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/2863
* @hopingZ made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/2860
* @zhouwei25 made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/2884
* @EscaticZheng made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/2915
* @chinobing made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/2922
* @lance6716 made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/2924
* @443127316 made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/2983

**Full Changelog**: https://github.com/PaddlePaddle/PaddleSpeech/compare/r1.3.0...r1.4.0