r1.1.0

PaddlePaddle/PaddleSpeechr1.1.0Aug 19, 2022by yt605155624

AI Summary

A major overhaul introducing VITS models, ONNX support for TTS, SpeechX refactoring, and a new speech web demo.

Key Highlights

  • VITS model release and integration
  • ONNX models for TTS vocoders
  • SpeechX refactoring and DeepSpeech2 to ONNX conversion
  • Speech Web Demo

New Features

  • VITS model
  • ONNX inference
  • Chinese English mixed TTS
  • Speech Web Demo
  • SpeechX refactoring

Full Release Notes

# S2T
  - Add wer tools. https://github.com/PaddlePaddle/PaddleSpeech/pull/1709
  - Add optimize attention cache used for attention ; 0-dim tensor for model export. https://github.com/PaddlePaddle/PaddleSpeech/pull/2124
  - Fix cnn cache dy2st shape.  https://github.com/PaddlePaddle/PaddleSpeech/pull/2168

# TTS
  - Fix random speaker embedding bug in voice clone. https://github.com/PaddlePaddle/PaddleSpeech/pull/1828 by @jerryuhoo
  - Add VITS model. https://github.com/PaddlePaddle/PaddleSpeech/pull/1855 https://github.com/PaddlePaddle/PaddleSpeech/pull/1957 https://github.com/PaddlePaddle/PaddleSpeech/pull/2040 
  - Add kunlun support for speedyspeech. https://github.com/PaddlePaddle/PaddleSpeech/pull/1879 by @QingshuChen
  - Normalize wav max value to 1 in preprocess. https://github.com/PaddlePaddle/PaddleSpeech/pull/1887 by @jerryuhoo
  - Remove fluid dependence in TTS. https://github.com/PaddlePaddle/PaddleSpeech/pull/1940
  - Add onnx models for aishell3/ljspeech/vctk's tts3/voc1/voc5. https://github.com/PaddlePaddle/PaddleSpeech/pull/2068
  - Add TTS static/onnx models in pretrained_models.py. https://github.com/PaddlePaddle/PaddleSpeech/pull/2074 
  - Add Ernie SAT model. https://github.com/PaddlePaddle/PaddleSpeech/pull/2052 https://github.com/PaddlePaddle/PaddleSpeech/pull/2117
  - Add Chinese English mixed TTS frontend. https://github.com/PaddlePaddle/PaddleSpeech/pull/2143
  - Add Chinese English mixed TTS example. https://github.com/PaddlePaddle/PaddleSpeech/pull/2234
  - Fix English text frontend bug. https://github.com/PaddlePaddle/PaddleSpeech/pull/2235 by @david-95
  - Add g2pW to Chinese frontend. https://github.com/PaddlePaddle/PaddleSpeech/pull/2230 by @BarryKCL
  - Fix text frontend bugs. https://github.com/PaddlePaddle/PaddleSpeech/pull/1912 https://github.com/PaddlePaddle/PaddleSpeech/pull/2250 https://github.com/PaddlePaddle/PaddleSpeech/pull/2254 https://github.com/PaddlePaddle/PaddleSpeech/pull/2255 https://github.com/PaddlePaddle/PaddleSpeech/pull/2272

# Speechx
- add custom asr script. https://github.com/PaddlePaddle/PaddleSpeech/pull/1946
- refactor frontend. https://github.com/PaddlePaddle/PaddleSpeech/pull/2003
- deepspeech2 to onnx https://github.com/PaddlePaddle/PaddleSpeech/pull/2034
- Refactor audio/data/feature cache. https://github.com/PaddlePaddle/PaddleSpeech/pull/1638
-  Frontend refactor .  https://github.com/PaddlePaddle/PaddleSpeech/pull/1640
- Fix nnet itf header. https://github.com/PaddlePaddle/PaddleSpeech/pull/1641
- Refactor speech egs. https://github.com/PaddlePaddle/PaddleSpeech/pull/1707
- Refactor egs and more egs for TLG wfst graph build.  https://github.com/PaddlePaddle/PaddleSpeech/pull/1715
- Speedup ngram building .  https://github.com/PaddlePaddle/PaddleSpeech/pull/1729
- Update speechx install doc. https://github.com/PaddlePaddle/PaddleSpeech/pull/1736
- Fix nnet input and output name. https://github.com/PaddlePaddle/PaddleSpeech/pull/1740
- Update wfst graph. https://github.com/PaddlePaddle/PaddleSpeech/pull/1742
- Fix model params path name.  https://github.com/PaddlePaddle/PaddleSpeech/pull/1750
- Remove fluid tools for onnx export. https://github.com/PaddlePaddle/PaddleSpeech/pull/2116

# Audio
  - Refactor paddleaudio to paddlespeech.audio. https://github.com/PaddlePaddle/PaddleSpeech/pull/2007
  - Add webdataset in paddlespeech.audio. https://github.com/PaddlePaddle/PaddleSpeech/pull/2062


# Server
  - Remove extra logs. https://github.com/PaddlePaddle/PaddleSpeech/pull/2111 https://github.com/PaddlePaddle/PaddleSpeech/pull/2113 
  - Change streaming tts servers' fs from 24k to models' fs. https://github.com/PaddlePaddle/PaddleSpeech/pull/2121
  - Fix bug in engine_warmup. https://github.com/PaddlePaddle/PaddleSpeech/pull/2171 by @Betterman-qs
  - Replace default vocoder in seerver to mb_melgan. https://github.com/PaddlePaddle/PaddleSpeech/pull/2214
  - Fix bug in streaming_asr_server with punctuation restoration. https://github.com/PaddlePaddle/PaddleSpeech/pull/2244
  -  Rename time_s and time_ns to time_b and time_nb. https://github.com/PaddlePaddle/PaddleSpeech/pull/2133
  - More accuracy decoding somthing. https://github.com/PaddlePaddle/PaddleSpeech/pull/2128

# CLI
  - Add paddlespeech.resource module. https://github.com/PaddlePaddle/PaddleSpeech/pull/1917
  - Dynamic cli commands registration. https://github.com/PaddlePaddle/PaddleSpeech/pull/1959
  - Fix unnecessary download. https://github.com/PaddlePaddle/PaddleSpeech/pull/2103
  - Remove extra logs. https://github.com/PaddlePaddle/PaddleSpeech/pull/2084 https://github.com/PaddlePaddle/PaddleSpeech/pull/2085 https://github.com/PaddlePaddle/PaddleSpeech/pull/2107 
  - Add Chinese English mixed TTS CLI. https://github.com/PaddlePaddle/PaddleSpeech/pull/2249
  - Add onnxruntime infer for CLI. https://github.com/PaddlePaddle/PaddleSpeech/pull/2222 
    

# Demo
  - Add speech web demo. https://github.com/PaddlePaddle/PaddleSpeech/pull/2039 https://github.com/PaddlePaddle/PaddleSpeech/pull/2080
  - Add kws cli and demo. https://github.com/PaddlePaddle/PaddleSpeech/pull/2063
  - Use paddle web for streaming asr. https://github.com/PaddlePaddle/PaddleSpeech/pull/2105
  - add custom asr script https://github.com/PaddlePaddle/PaddleSpeech/pull/1946
  - More cli for speech demos. https://github.com/PaddlePaddle/PaddleSpeech/pull/2138
 

# Doc
  - Add API doc. https://github.com/PaddlePaddle/PaddleSpeech/pull/2075
  - Format tts doc string for read the docs. https://github.com/PaddlePaddle/PaddleSpeech/pull/2115

# Others
  - Fix CPU Dockerfile. https://github.com/PaddlePaddle/PaddleSpeech/pull/2172 by @BrightXiaoHan
  - Add PaddleSpeech Dockerfile for hard mode of installation. https://github.com/PaddlePaddle/PaddleSpeech/pull/2127 by @buchongyu2

# Acknowledgements
Special thanks to  @buchongyu2 @BrightXiaoHan @BarryKCL @Betterman-qs @david-95 @jerryuhoo  @QingshuChen @iftaken @zh794390558 @Jackwaterveg  @lym0302  @SmileGoat  @yt605155624

## New Contributors
* @QingshuChen made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/1879
* @Zhangjingyu06 made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/1951
* @ryanrussell made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/1976
* @freeliuzc made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/2044
* @vpegasus made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/2043
* @dependabot made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/2061
* @raycool made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/2109
* @YDX-2147483647 made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/2125
* @chenkui164 made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/2130
* @0x45f made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/2162
* @Doubledongli made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/2167
* @Betterman-qs made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/2171
* @BrightXiaoHan made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/2172
* @THUzyt21 made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/2202
* @david-95 made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/2235
* @BarryKCL made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/2230

**Full Changelog**: https://github.com/PaddlePaddle/PaddleSpeech/compare/r1.0.0...r1.1.0