r1.0.0a

PaddlePaddle/PaddleSpeechr1.0.0aApr 28, 2022by Jackwaterveg

AI Summary

The alpha release leading up to v1.0.0, establishing core streaming ASR/TTS infrastructure, WebSocket support, and speaker verification features.

Key Highlights

  • Streaming ASR and TTS systems for industrial application
  • Speaker verification (VPR) pipeline
  • WebSocket support for ASR
  • ONNX inference for FastSpeech2 and vocoders

New Features

  • Streaming ASR
  • Streaming TTS
  • VPR
  • WebSocket support
  • ONNX inference

Full Release Notes

# Highlight 
* Release Streaming ASR and Streaming TTS system for industrial application.
* Support KWS model
* Deepspeech2 streaming model aishell cer 6.66%
* Conformer aishell cer 4.64%
* Conformer streaming model aishell cer 5.44%
* SpeechX Deepspeech2 streaming with WFST

# What's Changed
* [speechx] refactor audio/data/feature cache by @zh794390558 in https://github.com/PaddlePaddle/PaddleSpeech/pull/1638
* [speechx] Frontend refactor by @zh794390558 in https://github.com/PaddlePaddle/PaddleSpeech/pull/1640
* [speechx] fix nnet itf header by @zh794390558 in https://github.com/PaddlePaddle/PaddleSpeech/pull/1641
* [TTS]add license and reference for some models by @yt605155624 in https://github.com/PaddlePaddle/PaddleSpeech/pull/1642
* [Doc] supplement note by @Jackwaterveg in https://github.com/PaddlePaddle/PaddleSpeech/pull/1643
* [vec][search] update search demo README by @qingen in https://github.com/PaddlePaddle/PaddleSpeech/pull/1644
* [speechx]refactor linear feature:unify vector & remove redundant function & add remained_wav cache shift wav by @SmileGoat in https://github.com/PaddlePaddle/PaddleSpeech/pull/1649
* [Audio] Fix mcd issue. by @KPatr1ck in https://github.com/PaddlePaddle/PaddleSpeech/pull/1658
* [Audio] Remove mcd. by @KPatr1ck in https://github.com/PaddlePaddle/PaddleSpeech/pull/1659
* [vec]update the speaker verification model by @Honei in https://github.com/PaddlePaddle/PaddleSpeech/pull/1663
* [ASR] update ds2 online model by @Jackwaterveg in https://github.com/PaddlePaddle/PaddleSpeech/pull/1668
* [TTS]fix preprocess bug, test=tts by @yt605155624 in https://github.com/PaddlePaddle/PaddleSpeech/pull/1660
* update README, test=doc by @iftaken in https://github.com/PaddlePaddle/PaddleSpeech/pull/1672
* [Punc] Update RESULTS.md. by @KPatr1ck in https://github.com/PaddlePaddle/PaddleSpeech/pull/1675
* [CLI] update ds2 online model in cli by @Jackwaterveg in https://github.com/PaddlePaddle/PaddleSpeech/pull/1674
* [CLI] ASR: Add duration limitation for asr by @Jackwaterveg in https://github.com/PaddlePaddle/PaddleSpeech/pull/1666
* [vec]add speaker verification score method by @Honei in https://github.com/PaddlePaddle/PaddleSpeech/pull/1646
* [TTS]add onnx inference for fastspeech2 + hifigan/mb_melgan by @yt605155624 in https://github.com/PaddlePaddle/PaddleSpeech/pull/1665
* [doc]update readme by @yt605155624 in https://github.com/PaddlePaddle/PaddleSpeech/pull/1680
* [WebSocket] fixed online model md5 error , test=doc by @WilliamZhang06 in https://github.com/PaddlePaddle/PaddleSpeech/pull/1682
* [speechx]add aishell test script & json parser & no db norm linear feature & json2kaldi type cmvn by @SmileGoat in https://github.com/PaddlePaddle/PaddleSpeech/pull/1676
* [server] add stream tts server by @lym0302 in https://github.com/PaddlePaddle/PaddleSpeech/pull/1652
* [speechx]remove mutable in audio_cache by @SmileGoat in https://github.com/PaddlePaddle/PaddleSpeech/pull/1687
* [Doc] update readem for aishell/asr0 by @Jackwaterveg in https://github.com/PaddlePaddle/PaddleSpeech/pull/1677
* [vec] add speaker diarization pipeline by @ccrrong in https://github.com/PaddlePaddle/PaddleSpeech/pull/1651
* [vec]voxceleb convert dataset format to paddlespeech by @Honei in https://github.com/PaddlePaddle/PaddleSpeech/pull/1630
* [Speechx] add tlg decoder by @SmileGoat in https://github.com/PaddlePaddle/PaddleSpeech/pull/1599
* [vec]add vector necessary note, test=doc by @Honei in https://github.com/PaddlePaddle/PaddleSpeech/pull/1690
* Revert "[WebSocket] fixed online model md5 error , test=doc" by @zh794390558 in https://github.com/PaddlePaddle/PaddleSpeech/pull/1691
* [WebSocket] added online web client, test=doc by @WilliamZhang06 in https://github.com/PaddlePaddle/PaddleSpeech/pull/1692
* 修复 example/aishell  目录中speech单词拼写错误问题 by @buchongyu2 in https://github.com/PaddlePaddle/PaddleSpeech/pull/1694
* 修改hack 单词拼写错误 by @buchongyu2 in https://github.com/PaddlePaddle/PaddleSpeech/pull/1697
* [TTS]change NLC to NCL in speedyspeech, test=tts by @yt605155624 in https://github.com/PaddlePaddle/PaddleSpeech/pull/1693
* [doc]fix typo, test=doc by @yt605155624 in https://github.com/PaddlePaddle/PaddleSpeech/pull/1698
* [doc]add pwgan onnx model, test=doc by @yt605155624 in https://github.com/PaddlePaddle/PaddleSpeech/pull/1700
* [WebSocket] added online asr doc and online asr command line, test=doc by @WilliamZhang06 in https://github.com/PaddlePaddle/PaddleSpeech/pull/1701
* [vec][server] vpr demo support by @qingen in https://github.com/PaddlePaddle/PaddleSpeech/pull/1696
* [speechx] refactor speech egs by @zh794390558 in https://github.com/PaddlePaddle/PaddleSpeech/pull/1707
* [asr]add wer tools by @zh794390558 in https://github.com/PaddlePaddle/PaddleSpeech/pull/1709
* [asr][websocket]fix the ws send bug, cache buffer,  text=doc by @Honei in https://github.com/PaddlePaddle/PaddleSpeech/pull/1710
* [TTS]add fastspeech2 cnndecoder onnx model by @yt605155624 in https://github.com/PaddlePaddle/PaddleSpeech/pull/1712
* [speechx] refactor egs and more egs for TLG wfst graph build by @zh794390558 in https://github.com/PaddlePaddle/PaddleSpeech/pull/1715
* [vec][score] add plda model by @qingen in https://github.com/PaddlePaddle/PaddleSpeech/pull/1681
* [CLI]update cli, test=doc by @yt605155624 in https://github.com/PaddlePaddle/PaddleSpeech/pull/1716
* [server] add streaming am infer by @lym0302 in https://github.com/PaddlePaddle/PaddleSpeech/pull/1713
* [speechx] Add websocket & make it work by @SmileGoat in https://github.com/PaddlePaddle/PaddleSpeech/pull/1720
* [asr][websocket] add asr conformer websocket server by @Honei in https://github.com/PaddlePaddle/PaddleSpeech/pull/1704
* [vec][loss] add NCE Loss from RNNLM by @qingen in https://github.com/PaddlePaddle/PaddleSpeech/pull/1719
* [vec][loss] add FocalLoss to deal with class imbalances by @qingen in https://github.com/PaddlePaddle/PaddleSpeech/pull/1722
* [TTS]restructure syn_utils.py, test=tts by @yt605155624 in https://github.com/PaddlePaddle/PaddleSpeech/pull/1723
* [TTS]add paddle device set for ort and inference by @yt605155624 in https://github.com/PaddlePaddle/PaddleSpeech/pull/1727
* [vec] add GRL to domain adaptation by @qingen in https://github.com/PaddlePaddle/PaddleSpeech/pull/1725
* [speechx] speedup ngram building by @zh794390558 in https://github.com/PaddlePaddle/PaddleSpeech/pull/1729
* [asr] Add new cer tools by @Jackwaterveg in https://github.com/PaddlePaddle/PaddleSpeech/pull/1673
* [speechx]add websocket lib by @SmileGoat in https://github.com/PaddlePaddle/PaddleSpeech/pull/1732
* [speechx]update speechx install doc by @zh794390558 in https://github.com/PaddlePaddle/PaddleSpeech/pull/1736
* [Doc] prefect the packing scripts by @Jackwaterveg in https://github.com/PaddlePaddle/PaddleSpeech/pull/1735
* [Doc]renew the released mode by @Jackwaterveg in https://github.com/PaddlePaddle/PaddleSpeech/pull/1739
* [asr][websocket]add streaming asr demo by @Honei in https://github.com/PaddlePaddle/PaddleSpeech/pull/1737
* [speechx] fix nnet input and output name by @zh794390558 in https://github.com/PaddlePaddle/PaddleSpeech/pull/1740
* [ASR] remove redundant log by @Jackwaterveg in https://github.com/PaddlePaddle/PaddleSpeech/pull/1741
* [speechx] update wfst graph by @zh794390558 in https://github.com/PaddlePaddle/PaddleSpeech/pull/1742
* [speechx] Add recognizer_test_main script by @SmileGoat in https://github.com/PaddlePaddle/PaddleSpeech/pull/1743
* [vec][doc]update the voxceleb readme.md, test=doc by @Honei in https://github.com/PaddlePaddle/PaddleSpeech/pull/1744
* [ASR] fix CER tools by @Jackwaterveg in https://github.com/PaddlePaddle/PaddleSpeech/pull/1747
* [Doc] Fix release_model info by @Jackwaterveg in https://github.com/PaddlePaddle/PaddleSpeech/pull/1746
* [Doc] Updata released model info by @Jackwaterveg in https://github.com/PaddlePaddle/PaddleSpeech/pull/1748
* Updata released model info by @Jackwaterveg in https://github.com/PaddlePaddle/PaddleSpeech/pull/1749
* [speechx] fix model params path name by @zh794390558 in https://github.com/PaddlePaddle/PaddleSpeech/pull/1750
* [speechx] fix linear-spectrogram-wo-db-norm-ol read feature issue by @SmileGoat in https://github.com/PaddlePaddle/PaddleSpeech/pull/1751
* [TTS]fix wavernn white noise bug for paddle develop(2.3) by @yt605155624 in https://github.com/PaddlePaddle/PaddleSpeech/pull/1752
* [server] add onnx tts engine by @lym0302 in https://github.com/PaddlePaddle/PaddleSpeech/pull/1733
* [TTS]Update paddle2onnx by @yt605155624 in https://github.com/PaddlePaddle/PaddleSpeech/pull/1754
* [Setup] to r1.0.0a by @Jackwaterveg in https://github.com/PaddlePaddle/PaddleSpeech/pull/1759
* [audio] rename paddleaudio to audio, since confilict with pkg name by @zh794390558 in https://github.com/PaddlePaddle/PaddleSpeech/pull/1758
* [speechx] to_float32, fix shell script by @zh794390558 in https://github.com/PaddlePaddle/PaddleSpeech/pull/1757
* [vec] bug fix to adapt VUE by @qingen in https://github.com/PaddlePaddle/PaddleSpeech/pull/1760
* [asr][weboscket]fix the streaming asr server bug, server client by @Honei in https://github.com/PaddlePaddle/PaddleSpeech/pull/1761
* [speechx] fbank and mfcc by @zh794390558 in https://github.com/PaddlePaddle/PaddleSpeech/pull/1765
* format code by @zh794390558 in https://github.com/PaddlePaddle/PaddleSpeech/pull/1764
* [CLI] Add conformer_aishell, conformer_online_aishell by @Jackwaterveg in https://github.com/PaddlePaddle/PaddleSpeech/pull/1767
* [speechx]make cmvn global in run.sh by @SmileGoat in https://github.com/PaddlePaddle/PaddleSpeech/pull/1768
* [ASR] ds2: add log_interval and  fix lr problem when resume training by @Jackwaterveg in https://github.com/PaddlePaddle/PaddleSpeech/pull/1766
* [speechx] set nnet param by flags by @zh794390558 in https://github.com/PaddlePaddle/PaddleSpeech/pull/1769
* [server] add streaming tts demos by @lym0302 in https://github.com/PaddlePaddle/PaddleSpeech/pull/1771
* [server] fix tts streaming server by @lym0302 in https://github.com/PaddlePaddle/PaddleSpeech/pull/1774
* [KWS]Add kws example on HeySnips dataset. by @KPatr1ck in https://github.com/PaddlePaddle/PaddleSpeech/pull/1558
* [text][server]add text punc server by @Honei in https://github.com/PaddlePaddle/PaddleSpeech/pull/1772
* [ASR] fix asr cli infer by @Jackwaterveg in https://github.com/PaddlePaddle/PaddleSpeech/pull/1770
* [vec] add GE2E to support unlabeled data training by @qingen in https://github.com/PaddlePaddle/PaddleSpeech/pull/1731
* [ASR] fix time restricion in test_cli.sh by @Jackwaterveg in https://github.com/PaddlePaddle/PaddleSpeech/pull/1777
* [ASR] Replace fbank by @Jackwaterveg in https://github.com/PaddlePaddle/PaddleSpeech/pull/1776
* [CLI] add color for test_cli by @Jackwaterveg in https://github.com/PaddlePaddle/PaddleSpeech/pull/1778
* [speechx] add sucess log in run.sh by @SmileGoat in https://github.com/PaddlePaddle/PaddleSpeech/pull/1779
* [KWS]Update KWS example. by @KPatr1ck in https://github.com/PaddlePaddle/PaddleSpeech/pull/1783
* [server] update readme by @lym0302 in https://github.com/PaddlePaddle/PaddleSpeech/pull/1782
* [Doc] Update ds2online model info by @Jackwaterveg in https://github.com/PaddlePaddle/PaddleSpeech/pull/1781
* [CLI] renew ds2 online model by @Jackwaterveg in https://github.com/PaddlePaddle/PaddleSpeech/pull/1786
* [speechx] fix speechx ws server to return dummpy partial  result by @zh794390558 in https://github.com/PaddlePaddle/PaddleSpeech/pull/1787
* [asr][server]asr client add punctuatjion server by @Honei in https://github.com/PaddlePaddle/PaddleSpeech/pull/1784
* [asr] patch func to var by @zh794390558 in https://github.com/PaddlePaddle/PaddleSpeech/pull/1788
* [asr][server]fix client parse the asr result bug by @Honei in https://github.com/PaddlePaddle/PaddleSpeech/pull/1789
* [Bug fix] fix test_cli by @Jackwaterveg in https://github.com/PaddlePaddle/PaddleSpeech/pull/1794
* [vec] update readme by @qingen in https://github.com/PaddlePaddle/PaddleSpeech/pull/1796
* [R1.0]update the streaming output and punc default ip, port by @Honei in https://github.com/PaddlePaddle/PaddleSpeech/pull/1800
* Renew ds2 online model [cer 6.66%] by @Jackwaterveg in https://github.com/PaddlePaddle/PaddleSpeech/pull/1802
* [R1.0] update the streaming asr server readme by @Honei in https://github.com/PaddlePaddle/PaddleSpeech/pull/1810
* [R1.0] Renew ds2 online doc info  by @Jackwaterveg in https://github.com/PaddlePaddle/PaddleSpeech/pull/1809
* [server] update streaming demos readme by @lym0302 in https://github.com/PaddlePaddle/PaddleSpeech/pull/1806
* [R1.0]update the paddlespeech_client asr_online cli by @Honei in https://github.com/PaddlePaddle/PaddleSpeech/pull/1818
* [r1.0][doc] fix readme by @zh794390558 in https://github.com/PaddlePaddle/PaddleSpeech/pull/1825

# New Contributors
* @iftaken made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/1672
* @ccrrong made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/1651
* @buchongyu2 made their first contribution in https://github.com/PaddlePaddle/PaddleSpeech/pull/1694


# Acknowledgements
Special thanks to @zh794390558  @Honei  @Jackwaterveg  @lym0302  @qingen  @GT-ZhangAcer @yt605155624 @WilliamZhang06 @SmileGoat  @ccrrong 

**Full Changelog**: https://github.com/PaddlePaddle/PaddleSpeech/compare/r0.2.0...r1.0.0a