Starstruk Logo MarkStarstruk

r1.3.0

expressjs/expressr1.3.0Dec 14, 2022by SmileGoat

AI Summary

This release significantly expands the project's scope by integrating Self-Supervised Learning models such as Whisper and Wav2vec2 for ASR, while enhancing TTS with prosody prediction, streaming serving, and Paddle-Lite integration.

Key Highlights

  • Added Whisper CLI and Demos for multi-language recognition and translation.
  • Added Wav2vec2 ASR and Feature Extraction support.
  • Added end-to-end Prosody Prediction pipeline for TTS.
  • Added Streaming TTS FastDeploy serving.
  • Moved audio functionality to paddleaudio package.

New Features

  • Whisper CLI and Demos
  • Wav2vec2ASR-en and wav2vec2-zh models
  • Streaming TTS FastDeploy serving
  • End-to-end Prosody Prediction pipeline
  • Adversarial Loss for Chinese-English mixed TTS
  • TTS Paddle-Lite x86 inference
  • TTS Android demo
  • Mix finetune support
  • SSML for Chinese Text Frontend

Full Release Notes

## HighLIght

### S2T
- Support U2/U2++ Conformer dy2static, and [U2/U2++ C++ High Performance Streaming ASR Deployment](https://github.com/PaddlePaddle/PaddleSpeech/blob/develop/speechx/examples/u2pp_ol/wenetspeech). @zh794390558 
- Add [Wav2vec2ASR-en](https://github.com/PaddlePaddle/PaddleSpeech/blob/develop/examples/librispeech/asr3), wav2vec2.0 fine-tuning for ASR on LibriSpeech.  @Zth9730 
- Add [Whisper CLI and Demos](https://github.com/PaddlePaddle/PaddleSpeech/pull/2640), support multi language recognition and translation.  @zxcd 
- Add [Wav2vec2 CLI and Demos](https://github.com/PaddlePaddle/PaddleSpeech/blob/develop/demos/speech_ssl), support ASR and Feature Extraction.  @Zth9730 
- Add whisper. #2640 #2704 by @zxcd
- Fix gpu training hang. #2478 by @Zth9730 
- Support u2++ based cli and server. #2489 #2510 by @Zth9730 
- Add wav2vec2-en. #2518 #2527 #2637 by @Zth9730 
- Add wav2vec2-zh cli. #2697 by @Zth9730 

### T2S
- Add seek for BytesIO. https://github.com/PaddlePaddle/PaddleSpeech/pull/2484 by @ZapBird 
- Add mix finetune. https://github.com/PaddlePaddle/PaddleSpeech/pull/2525 https://github.com/PaddlePaddle/PaddleSpeech/pull/2647 by @lym0302 
- Add streaming TTS fastdeploy serving. https://github.com/PaddlePaddle/PaddleSpeech/pull/2528 by @HexToString 
- Add SSML for Chinese Text Frontend. https://github.com/PaddlePaddle/PaddleSpeech/pull/2531 by @david-95 
- Add [end-to-end Prosody Prediction pipeline](https://github.com/PaddlePaddle/PaddleSpeech/tree/develop/examples/csmsc/tts3_rhy) (including using prosody labels in Acoustic Model). https://github.com/PaddlePaddle/PaddleSpeech/pull/2548 https://github.com/PaddlePaddle/PaddleSpeech/pull/2615 https://github.com/PaddlePaddle/PaddleSpeech/pull/2693 by @WongLaw 
- Add [Adversarial Loss](https://arxiv.org/pdf/1907.04448.pdf) for [Chinese English mixed TTS](https://github.com/lym0302/PaddleSpeech/blob/develop/examples/zh_en_tts/tts3). https://github.com/PaddlePaddle/PaddleSpeech/pull/2588 by @lym0302
- Fix frontend bugs. https://github.com/PaddlePaddle/PaddleSpeech/pull/2539 https://github.com/PaddlePaddle/PaddleSpeech/pull/2606 by @yt605155624 
- Add TN for English unit. https://github.com/PaddlePaddle/PaddleSpeech/pull/2629 by @WongLaw 
- Add male voice for TTS. https://github.com/PaddlePaddle/PaddleSpeech/pull/2660 by @lym0302 
- Add double byte char for zh normalization. https://github.com/PaddlePaddle/PaddleSpeech/pull/2661 by @david-95 
- Add TTS Paddle-Lite x86 inference. https://github.com/PaddlePaddle/PaddleSpeech/pull/2636 https://github.com/PaddlePaddle/PaddleSpeech/pull/2667 by @yt605155624 
- Add greek char and fix #2571. https://github.com/PaddlePaddle/PaddleSpeech/pull/2683 by @david-95 
- Add Slim for TTS. https://github.com/PaddlePaddle/PaddleSpeech/pull/2729 by @yt605155624 

### Audio
- Move paddlespeech/audio to paddleaudio. https://github.com/PaddlePaddle/PaddleSpeech/pull/2706 by @SmileGoat 

### Demo
- Add TTSAndroid demo. https://github.com/PaddlePaddle/PaddleSpeech/pull/2703 by @yt605155624 

## New Contributors
* @ZapBird made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/2484
* @HexToString made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/2528
* @dahu1 made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/2554
* @kFoodie made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/2664
* @zxcd made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/2640
* @michael-skynorth made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/2666
* @heyudage made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/2688

**Full Changelog**: https://github.com/PaddlePaddle/PaddleSpeech/compare/r1.2.0...r1.3.0