Author name cluster

Chung-Cheng Chiu

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

5 papers

2 author rows

ICLR Conference 2025 Conference Paper

Talking Turns: Benchmarking Audio Foundation Models on Turn-Taking Dynamics

Siddhant Arora
Zhiyun Lu
Chung-Cheng Chiu
Ruoming Pang
Shinji Watanabe 0001

The recent wave of audio foundation models (FMs) could provide new capabilities for conversational modeling. However, there have been limited efforts to evaluate these audio FMs comprehensively on their ability to have natural and interactive conversations. To engage in meaningful conversation with the end user, we would want the FMs to additionally perform a fluent succession of turns without too much overlapping speech or long stretches of silence. Inspired by this, we ask whether the recently proposed audio FMs can understand, predict, and perform turn-taking events? To answer this, we propose a novel evaluation protocol that can assess spoken dialog system's turn-taking capabilities using a supervised model as a judge that has been trained to predict turn-taking events in human-human conversations. Using this protocol, we present the first comprehensive user study that evaluates existing spoken dialogue systems on their ability to perform turn-taking events and reveal many interesting insights, such as they sometimes do not understand when to speak up, can interrupt too aggressively and rarely backchannel. We further evaluate multiple open-source and proprietary audio FMs accessible through APIs on carefully curated test benchmarks from Switchboard to measure their ability to understand and predict turn-taking events and identify significant room for improvement. We will open source our evaluation platform to promote the development of advanced conversational AI systems.

Details

ICML Conference 2022 Conference Paper

Self-supervised learning with random-projection quantizer for speech recognition

Chung-Cheng Chiu
James Qin
Yu Zhang
Jiahui Yu
Yonghui Wu

We present a simple and effective self-supervised learning approach for speech recognition. The approach learns a model to predict the masked speech signals, in the form of discrete labels generated with a random-projection quantizer. In particular the quantizer projects speech inputs with a randomly initialized matrix, and does a nearest-neighbor lookup in a randomly-initialized codebook. Neither the matrix nor the codebook are updated during self-supervised learning. Since the random-projection quantizer is not trained and is separated from the speech recognition model, the design makes the approach flexible and is compatible with universal speech recognition architecture. On LibriSpeech our approach achieves similar word-error-rates as previous work using self-supervised learning with non-streaming models, and provides lower word-error-rates than previous work with streaming models. On multilingual tasks the approach also provides significant improvement over wav2vec 2. 0 and w2v-BERT.

Details

ICLR Conference 2021 Conference Paper

Dual-mode ASR: Unify and Improve Streaming ASR with Full-context Modeling

Jiahui Yu
Wei Han 0002
Anmol Gulati
Chung-Cheng Chiu
Bo Li 0028
Tara N. Sainath
Yonghui Wu
Ruoming Pang

Streaming automatic speech recognition (ASR) aims to emit each hypothesized word as quickly and accurately as possible, while full-context ASR waits for the completion of a full speech utterance before emitting completed hypotheses. In this work, we propose a unified framework, Dual-mode ASR, to train a single end-to-end ASR model with shared weights for both streaming and full-context speech recognition. We show that the latency and accuracy of streaming ASR significantly benefit from weight sharing and joint training of full-context ASR, especially with inplace knowledge distillation during the training. The Dual-mode ASR framework can be applied to recent state-of-the-art convolution-based and transformer-based ASR networks. We present extensive experiments with two state-of-the-art ASR networks, ContextNet and Conformer, on two datasets, a widely used public dataset LibriSpeech and a large-scale dataset MultiDomain. Experiments and ablation studies demonstrate that Dual-mode ASR not only simplifies the workflow of training and deploying streaming and full-context ASR models, but also significantly improves both emission latency and recognition accuracy of streaming ASR. With Dual-mode ASR, we achieve new state-of-the-art streaming ASR results on both LibriSpeech and MultiDomain in terms of accuracy and latency.

Details

AAMAS Conference 2011 Conference Paper

A Style Controller for Generating Virtual Human Behaviors

Chung-Cheng Chiu
Stacy Marsella

Creating a virtual character that exhibits realistic physical behaviors requires a rich set of animations. To mimic the variety as well as the subtlety of human behavior, we may need to animate not only a wide range of behaviors but also variations of the same type of behavior influenced by the environment and the state of the character, including the emotional and physiological state. A general approach to this challenge is to gather a set of animations produced by artists or motion capture. However, this approach can be extremely costly in time and effort. In this work, we propose a model that can learn styled motion generation and an algorithm that produce new styles of motions via style interpolation. The model takes a set of styled motions as training samples, and can create new motions that are the generalization among given styles of motions. Our style interpolation algorithm can blend together motions with distinct styles, and it also helps improve the performance of previous work. We verify our algorithm using walking motions of different styles, and the experimental results show that our method is significantly better than previous work.

PDF

AIIM Journal 2010 Journal Article

Analysis of adverse drug reactions using drug and drug target interactions and graph-based methods

Shih-Fang Lin
Ke-Ting Xiao
Yu-Ting Huang
Chung-Cheng Chiu
Von-Wun Soo

Objective The purpose of this study was to integrate knowledge about drugs, drug targets, and topological methods. The goals were to build a system facilitating the study of adverse drug events, to make it easier to find possible explanations, and to group similar drug–drug interaction cases in the adverse drug reaction reports from the US Food and Drug Administration (FDA). Methods We developed a system that analyses adverse drug reaction (ADR) cases reported by the FDA. The system contains four modules. First, we integrate drug and drug target databases that provide information related to adverse drug reactions. Second, we classify drug and drug targets according to anatomical therapeutic chemical classification (ATC) and drug target ontology (DTO). Third, we build drug target networks based on drug and drug target databases. Finally, we apply topological analysis to reveal drug interaction complexity for each ADR case reported by the FDA. Results We picked 1952 ADR cases from the years 2005–2006. Our dataset consisted of 1952 cases, of which 1471 cases involved ADR targets, 845 cases involved absorption, distribution, metabolism, and excretion (ADME) targets, and 507 cases involved some drugs acting on the same targets, namely, common targets (CTs). We then investigated the cases involving ADR targets, ADME targets, and CTs using the ATC system and DTO. In the cases that led to death, the average number of common targets (NCTs) was 0. 879 and the average of average clustering coefficient (ACC) was 0. 067. In cases that did not lead to death, the average NCTs was 0. 551, and the average of ACC was 0. 039. Conclusions We implemented a system that can find possible explanations and cluster similar ADR cases reported by the FDA. We found that the average of ACC and the average NCTs in cases leading to death are higher than in cases not leading to death, suggesting that the interactions in cases leading to death are generally more complicated than in cases not leading to death. This indicates that our system can help not only in analysing ADRs in terms of drug–drug interactions but also by providing drug target assessments early in the drug discovery process.

Details DOI