TTA-Bench: A Comprehensive Benchmark for Evaluating Text-to-Audio Models

Text-to-Audio (TTA) generation has made rapid progress, but current evaluation methods remain narrow, focusing mainly on perceptual quality while overlooking robustness, generalization, and ethical concerns. We present TTA-Bench, a comprehensive benchmark for evaluating TTA models across functional performance, reliability, and social responsibility. It covers seven dimensions including accuracy, robustness, fairness, and toxicity, and includes 2,999 diverse prompts generated through automated and manual methods. We introduce a unified evaluation protocol that combines objective metrics with over 118,000 human annotations from both experts and general users. Ten state-of-the-art models are benchmarked under this framework, offering detailed insights into their strengths and limitations. TTA-Bench establishes a new standard for holistic evaluation of TTA systems.

Authors

• Hui Wang
• Cheng Liu Chongqing Key Laboratory of Computational Intelligence Chongqing University of Posts and Telecommunications, Chongqing, China Key Laboratory of Cyberspace Big Data Intelligent Security, Ministry of Education
• Junyang Chen College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, China
• Haoze Liu College of Computer Science, Nankai University
• Yuhang Jia Research Center of Ubiquitous Computing and Intelligent Systems, Harbin Institute of Technology, China
• Shiwan Zhao
• Jiaming Zhou
• Haoqin Sun College of Computer Science, Nankai University
• Hui Bu Beijing AIShell Technology Co. Ltd, China
• Yong Qin

Keywords

No keywords are indexed for this paper.