r1.2.0

PaddlePaddle/PaddleSpeechr1.2.0Oct 10, 2022by lym0302

AI Summary

A significant update introducing VITS voice cloning, ERNIE-SAT synthesis, BiTransformer decoders, and various TTS/ASR fixes.

Key Highlights

  • VITS support and voice cloning training on AISHELL-3
  • ERNIE-SAT end-to-end synthesis
  • BiTransformer decoder support
  • AISHELL-3 Voice Cloning with ECAPA-TDNN

New Features

  • VITS model
  • ERNIE-SAT model
  • BiTransformer decoder
  • TTS finetune examples
  • Chinese English mixed TTS frontend

Full Release Notes

# S2T
 - Fix conformer/transformer multi GPU training. #2327 #2334 #2336 #2372 by @Zth9730
 - Fix deepspeech2 decode_wav. #2351 by @Zth9730
 - Support BiTransformer decoder. #2415 by @Zth9730
 

# T2S
 - Update VITS to support VITS and its voice cloning training on AISHELL-3. #2268 by @HighCWu
 - Add ERNIE-SAT synthesize_e2e. #2287 #2316 #2355 #2378 #2432 by @yt605155624
 - Specify the input data type of G2PW. #2288 by @kslz
 - Add TTS finetune example. #2297 #2385 #2418 #2430 by @lym0302
 - Fix Chinese English mixed TTS frontend. #2299 #2493 by @lym0302
 - Add words into polyphonic.yaml for g2pW. #2300 by @david-95
 - Update the quantifier unit in Text Normalization. #2308 by @pengzhendong
 - Fix Chinese frontend bugs. #2312 #2323 by @david-95
 - Add AISHELL-3 Voice Cloning with ECAPA-TDNN speaker encoder. #2359 #2429 by @yt605155624
 - Add pre-install doc for G2P and TN, update version of pypinyin. #2364 by @WongLaw
 - Add tools to compare two test results of G2P to show differences. #2367 by @david-95
 - Revise must_neural_tone_words. #2370 by @WongLaw
 - Add type-hint for g2pW. #2390 by @yt605155624
 - Replaced fixed path with path variable in MFA. #2416 by @WongLaw
 - Solve "unknown format: 3" for wavfile.write(). #2422 by @zhoupc2015


# Text
 - Create preprocess.py for Punctuation Restoration. #2295 by @THUzyt21
 
# Demo
 - Add Voice Cloning, TTS finetune, and ERNIE-SAT in speech_web. #2412 #2451 by @iftaken

# Server
 - Add num_decoding_left_chunks in streaming_asr_server's config. #2337 by @THUzyt21
 - Removed useless spk_id in speech_server and streaming_tts_server, support Chinese English mixed TTS server engine. #2380 by @WongLaw


# Doc
 - Add Chinese doc and language switcher for metaverse, style_fs2 and story_talker. #2357 by @WongLaw
 - Update API docs. #2406 by @yt605155624
 - Add finetune demos in readthedocs. #2411 by @yt605155624

# Test
 - Add barrier for distributed training using multiple machines. #2309 #2311 by @sneaxiy
 - Fix prepare.sh for PWGAN TIPC. #2376 by @yuehuayingxueluo

# Other
 - Format paddlespeech with pre-commit. #2331 by @yt605155624
 

# Acknowledgements
Special thanks to @yt605155624  @lym0302  @THUzyt21  @iftaken  @Zth9730  @zhoupc2015  @WongLaw  @david-95  @pengzhendong  @kslz  @HighCWu  @yuehuayingxueluo  @sneaxiy @SmileGoat 

## New Contributors
- @HighCWu made their first contribution in #2268
- @pengzhendong made their first contribution in #2308 
- @Zth9730 made their first contribution in #2327
- @WongLaw made their first contribution in #2357
- @yuehuayingxueluo made their first contribution in #2376
- @zhoupc2015 made their first contribution in #2422


**Full Changelog**: https://github.com/PaddlePaddle/PaddleSpeech/compare/r1.1.0...r1.2.0