WenetSpeech-Yue: A Large-Scale Cantonese Speech Corpus with Multi-dimensional Annotation

The development of speech understanding and generation has been significantly accelerated by the availability of large-scale, high-quality speech datasets. Among these, ASR and TTS are regarded as the most established and fundamental tasks. However, for Cantonese (Yue Chinese), spoken by approximately 84.9 million native speakers worldwide, limited annotated resources have hindered progress and resulted in suboptimal ASR and TTS performance. To address this challenge, we propose WenetSpeech-Pipe, an integrated pipeline for building large-scale speech corpus with multi-dimensional annotation tailored for speech understanding and generation. Based on this pipeline, we release WenetSpeech-Yue, the first large-scale Cantonese speech corpus with multi-dimensional annotation for ASR and TTS, covering 21,800 hours across 10 domains with annotations including ASR transcription, text confidence, speaker identity, age, gender, speech quality scores, among other annotations. We also release WSYue-eval, a comprehensive Cantonese benchmark with two components: WSYue-ASR-eval, a manually annotated set for evaluating ASR on short and long utterances, code-switching, and diverse acoustic conditions, and WSYue-TTS-eval, with base and coverage subsets for standard and generalization testing. Experimental results show that models trained on WenetSpeech-Yue achieve competitive results against state-of-the-art (SOTA) Cantonese ASR and TTS systems, including commercial and LLM-based models, highlighting the value of our dataset and pipeline.

Authors

• Longhao Li Audio, Speech and Language Processing Group (ASLP@NPU), Northwestern Polytechnical University
• Zhao Guo Audio, Speech and Language Processing Group (ASLP@NPU), Northwestern Polytechnical University
• Hongjie Chen
• Yuhang Dai Audio, Speech and Language Processing Group (ASLP@NPU), Northwestern Polytechnical University
• Ziyu Zhang Audio, Speech and Language Processing Group (ASLP@NPU), Northwestern Polytechnical University
• Hongfei Xue University of North Carolina at Charlotte
• Tianlun Zuo Audio, Speech and Language Processing Group (ASLP@NPU), Northwestern Polytechnical University
• Chengyou Wang Audio, Speech and Language Processing Group (ASLP@NPU), Northwestern Polytechnical University
• Shuiyuan Wang Audio, Speech and Language Processing Group (ASLP@NPU), Northwestern Polytechnical University
• Xin Xu
• Hui Bu Beijing AIShell Technology Co. Ltd, China
• Jie Li Shanghai Jiao Tong University Shanghai Jiao Tong University (Wuxi) Blockchain Advanced Research Center
• Jian Kang
• Binbin Zhang WeNet Open Source Community
• Ruibin Yuan
• Ziya Zhou
• Wei Xue Hong Kong University of Science and Technology
• Lei Xie Northwest Polytechnical University

Keywords

No keywords are indexed for this paper.