Author name cluster

Yu Yao

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

19 papers

2 author rows

AAAI Conference 2026 Conference Paper

Early Warning of Intraoperative Adverse Events via Transformer-Driven Multi-Label Learning

Xueyao Wang
Xiuding Cai
Honglin Shang
Yaoyao Zhu
Yu Yao

Early warning of intraoperative adverse events plays a vital role in reducing surgical risk and improving patient safety. While deep learning has shown promise in predicting the single adverse event, several key challenges remain: overlooking adverse event dependencies, underutilizing heterogeneous clinical data, and suffering from the class imbalance inherent in medical datasets. To address these issues, we construct the first Multi-label Adverse Events dataset (MuAE) for intraoperative adverse events prediction, covering six critical events. Next, we propose a novel Transformer-based multi-label learning framework (IAENet) that combines an improved Time-Aware Feature-wise Linear Modulation (TAFiLM) module for static covariates and dynamic variables robust fusion and complex temporal dependencies modeling. Furthermore, we introduce a Label-Constrained Reweighting Loss (LCRLoss) with co-occurrence regularization to effectively mitigate intra-event imbalance and enforce structured consistency among frequently co-occurring events. Extensive experiments demonstrate that IAENet consistently outperforms strong baselines on 5, 10, and 15-minute early warning tasks, achieving improvements of +5.05%, +2.82%, and +7.57% on average F1 score. These results highlight the potential of IAENet for supporting intelligent intraoperative decision-making in clinical practice.

PDF Details DOI

ICLR Conference 2025 Conference Paper

A Conditional Independence Test in the Presence of Discretization

Boyang Sun
Yu Yao
Guang-Yuan Hao
Yumou Qiu
Kun Zhang

Testing conditional independence (CI) has many important applications, such as Bayesian network learning and causal discovery. Although several approaches have been developed for learning CI structures for observed variables, those existing methods generally fail to work when the variables of interest can not be directly observed and only discretized values of those variables are available. For example, if $X_1$, $\tilde{X}_2$ and $X_3$ are the observed variables, where $\tilde{X}_2$ is a discretization of the latent variable $X_2$, applying the existing methods to the observations of $X_1$, $\tilde{X}_2$ and $X_3$ would lead to a false conclusion about the underlying CI of variables $X_1$, $X_2$ and $X_3$. Motivated by this, we propose a CI test specifically designed to accommodate the presence of discretization. To achieve this, a bridge equation and nodewise regression are used to recover the precision coefficients reflecting the conditional dependence of the latent continuous variables under the nonparanormal model. An appropriate test statistic has been proposed, and its asymptotic distribution under the null hypothesis of CI has been derived. Theoretical analysis, along with empirical validation on various datasets, rigorously demonstrates the effectiveness of our testing methods.

Details

IROS Conference 2025 Conference Paper

AGENTS-LLM: Augmentative GENeration of Challenging Traffic Scenarios with an Agentic LLM Framework

Yu Yao
Salil Bhatnagar
Markus Mazzola
Vasileios Belagiannis
Igor Gilitschenski
Luigi Palmieri
Simon Razniewski
Marcel Hallgarten

Rare, yet critical, scenarios pose a significant challenge in testing and evaluating autonomous driving planners. Relying solely on real-world driving scenes requires collecting massive datasets to capture these scenarios. While automatic generation of traffic scenarios appears promising, data-driven models require extensive training data and often lack fine-grained control over the output. Moreover, generating novel scenarios from scratch can introduce a distributional shift from the original training scenes which undermines the validity of evaluations especially for learning-based planners. To sidestep this, recent work proposes to generate challenging scenarios by augmenting original scenarios from the test set. However, this involves the manual augmentation of scenarios by domain experts. An approach that is unable to meet the demands for scale in the evaluation of self-driving systems. Therefore, this paper introduces a novel LLM-agent based framework for augmenting real-world traffic scenarios using natural language descriptions, addressing the limitations of existing methods. A key innovation is the use of an agentic design, enabling fine-grained control over the output and maintaining high performance even with smaller, cost-effective LLMs. Extensive human expert evaluation demonstrates our framework’s ability to accurately adhere to user intent, generating high quality augmented scenarios comparable to those created manually.

Details

NeurIPS Conference 2025 Conference Paper

Aligning What Matters: Masked Latent Adaptation for Text-to-Audio-Video Generation

Jiyang Zheng
Siqi Pan
Yu Yao
Zhaoqing Wang
Dadong Wang
Tongliang Liu

Text-to-Audio-Video (T2AV) generation aims to produce temporally and semantically aligned visual and auditory content from natural language descriptions. While recent progress in text-to-audio and text-to-video models has improved generation quality within each modality, jointly modeling them remains challenging due to incomplete and asymmetric correspondence: audio often reflects only a subset of the visual scene, and vice versa. Naively enforcing full alignment introduces semantic noise and temporal mismatches. To address this, we propose a novel framework that performs selective cross-modal alignment through a learnable masking mechanism, enabling the model to isolate and align only the shared latent components relevant to both modalities. This mechanism is integrated into an adaptation module that interfaces with pretrained encoders and decoders from latent video and audio diffusion models, preserving their generative capacity with reduced training overhead. Theoretically, we show that our masked objective provably recovers the minimal set of shared latent variables across modalities. Empirically, our method achieves state-of-the-art performance on standard T2AV benchmarks, demonstrating significant improvements in audiovisual synchronization and semantic consistency.

PDF Details

NeurIPS Conference 2025 Conference Paper

Can Dependencies Induced by LLM-Agent Workflows Be Trusted?

Yu Yao
Yiliao (Lia) Song
Yian Xie
Mengdan Fan
Mingyu Guo
Tongliang Liu

LLM-agent systems often decompose high-level objectives into subtask dependency graphs, assuming that each subtask’s output is reliable and conditionally independent of others given its parent responses. However, this assumption frequently breaks during execution, as ground-truth responses are inaccessible, leading to inter-agent misalignment—failures caused by inconsistencies and coordination breakdowns among agents. To address this, we propose SeqCV, a dynamic framework for reliable execution under violated conditional independence. SeqCV executes subtasks sequentially, each conditioned on all prior verified responses, and performs consistency checks immediately after agents generate short token sequences. At each checkpoint, a token sequence is accepted only if it represents shared knowledge consistently supported across diverse LLM models; otherwise, it is discarded, triggering recursive subtask decomposition for finer-grained reasoning. Despite its sequential nature, SeqCV avoids repeated corrections on the same misalignment and achieves higher effective throughput than parallel pipelines. Across multiple reasoning and coordination tasks, SeqCV improves accuracy by up to 30\% over existing LLM-agent systems. Code is available at https: //github. com/tmllab/2025 NeurIPS SeqCV.

PDF Details

ICML Conference 2025 Conference Paper

SafeAuto: Knowledge-Enhanced Safe Autonomous Driving with Multimodal Foundation Models

Jiawei Zhang 0013
Xuan Yang
Taiqi Wang
Yu Yao
Aleksandr Petiushko
Bo Li 0026

Traditional autonomous driving systems often struggle to connect high-level reasoning with low-level control, leading to suboptimal and sometimes unsafe behaviors. Recent advances in multimodal large language models (MLLMs), which process both visual and textual data, offer an opportunity to unify perception and reasoning. However, effectively embedding precise safety knowledge into MLLMs for autonomous driving remains a significant challenge. To address this, we propose SafeAuto, a framework that enhances MLLM-based autonomous driving by incorporating both unstructured and structured knowledge. First, we introduce a Position-Dependent Cross-Entropy (PDCE) loss to improve low-level control signal predictions when values are represented as text. Second, to explicitly integrate safety knowledge, we develop a reasoning component that translates traffic rules into first-order logic (e. g. , "red light =$>$ stop") and embeds them into a probabilistic graphical model (e. g. , Markov Logic Network) to verify predicted actions using recognized environmental attributes. Additionally, our Multimodal Retrieval-Augmented Generation (RAG) model leverages video, control signals, and environmental attributes to learn from past driving experiences. Integrating PDCE, MLN, and Multimodal RAG, SafeAuto outperforms existing baselines across multiple datasets, enabling more accurate, reliable, and safer autonomous driving. The code is available at https: //github. com/AI-secure/SafeAuto.

Details

JBHI Journal 2024 Journal Article

A SwinTransformer-Based Segmentation Framework With Self-Supervised Strategy for Post-Operative Prostate Cancer Radiotherapy

Dong Miao
Jielang Li
Meng Dou
Linjie Fu
Yu Yao
Xin Wang
Feng Wen
Yali Shen

Radical prostatectomy (prostate removal) is a standard treatment for clinically localized prostate cancer and is often followed by postoperative radiotherapy. Postoperative radiotherapy requires accurate delineation of the clinical target volume (CTV) and lymph node drainage area (LNA) on computed tomography (CT) images. However, the CTV contour cannot be determined by the simple prostate expansion after resection of the prostate in the CT image. Constrained by this factor, the manual delineation process in postoperative radiotherapy is more time-consuming and challenging than in radical radiotherapy. In addition, CTV and LNA have no boundaries that can be distinguished by pixel values in CT images, and existing automatic segmentation models cannot get satisfactory results. Radiation oncologists generally determine CTV and LNA profiles according to clinical consensus and guidelines regarding surrounding organs at risk (OARs). In this work, we design a cascade segmentation block to explicitly establish correlations between CTV, LNA, and OARs, leveraging OARs features to guide CTV and LNA segmentation. Furthermore, inspired by the success of the self-attention mechanism and self-supervised learning, we adopt SwinTransformer as our backbone and propose a pure SwinTransformer-based segmentation network with self-supervised learning strategies. We performed extensive quantitative and qualitative evaluations of the proposed method. Compared to other competitive segmentation models, our model shows higher dice scores with minor standard deviations, and the detailed visualization results are more consistent with the ground truth. We believe this work can provide a feasible solution to this problem, making the postoperative radiotherapy process more efficient.

Details DOI

EAAI Journal 2024 Journal Article

Adaptive neighborhood-perceived contrastive network for early stage fault diagnosis of rolling bearing with limited labeled data

Yu Yao
Jian Feng
Huaguang Zhang
Yitong Xing

Early stage fault diagnosis is vital for rotating machinery to reduce accidents and increase stability. It is a challenging task since weak fault pattern is presented in signals and limited labeled data escalate the hardness of learning salient fault features. To address the gap, we propose a novel adaptive neighborhood-perceived contrastive network (ANPCN). It enhances features by adaptively aggregating representations of the samples that have high anchor-based correlations, and restraining the connections among samples that have low correlations. Specifically, based on mapped time–frequency representations, a graph with dynamic edges is constructed by measuring designed anchor-based similarity based on limited labels. In ANPCN, a graph learning path learns fault features using a graph neural network (GNN) that can perceive neighborhood from the graph. A contrastive learning path learns prototype features by integrating data views and the edge topology of the graph, utilizing a key parameter α to combine them. The prototype features serve as a supplement to the limited labels, guiding the learning of the GNN and updating dynamic edges. Fault features are enhanced by alternatively optimizing the two paths. The effectiveness of the proposed approach is validated on a private bearing testbed and two public benchmarks, reaching a diagnosis accuracy of 96. 63% on our dataset with only 5% labeled data.

Details DOI

NeurIPS Conference 2024 Conference Paper

Identifying Latent State-Transition Processes for Individualized Reinforcement Learning

Yuewen Sun
Biwei Huang
Yu Yao
Donghuo Zeng
Xinshuai Dong
Songyao Jin
Boyang Sun
Roberto Legaspi

The application of reinforcement learning (RL) involving interactions with individuals has grown significantly in recent years. These interactions, influenced by factors such as personal preferences and physiological differences, causally influence state transitions, ranging from health conditions in healthcare to learning progress in education. As a result, different individuals may exhibit different state-transition processes. Understanding individualized state-transition processes is essential for optimizing individualized policies. In practice, however, identifying these state-transition processes is challenging, as individual-specific factors often remain latent. In this paper, we establish the identifiability of these latent factors and introduce a practical method that effectively learns these processes from observed state-action trajectories. Experiments on various datasets show that the proposed method can effectively identify latent state-transition processes and facilitate the learning of individualized RL policies.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

Learning the Latent Causal Structure for Modeling Label Noise

Yexiong Lin
Yu Yao
Tongliang Liu

In label-noise learning, the noise transition matrix reveals how an instance transitions from its clean label to its noisy label. Accurately estimating an instance's noise transition matrix is crucial for estimating its clean label. However, when only a noisy dataset is available, noise transition matrices can be estimated only for some "special" instances. To leverage these estimated transition matrices to help estimate the transition matrices of other instances, it is essential to explore relations between the matrices of these "special" instances and those of others. Existing studies typically build the relation by explicitly defining the similarity between the estimated noise transition matrices of "special" instances and those of other instances. However, these similarity-based assumptions are hard to validate and may not align with real-world data. If these assumptions fail, both noise transition matrices and clean labels cannot be accurately estimated. In this paper, we found that by learning the latent causal structure governing the generating process of noisy data, we can estimate noise transition matrices without the need for similarity-based assumptions. Unlike previous generative label-noise learning methods, we consider causal relations between latent causal variables and model them with a learnable graphical model. Utilizing only noisy data, our method can effectively learn the latent causal structure. Experimental results on various noisy datasets demonstrate that our method achieves state-of-the-art performance in estimating noise transition matrices, which leads to improved classification accuracy. The code is available at: https: //github. com/tmllab/2024 NeurIPS CSGN.

PDF Details DOI

AAAI Conference 2024 Conference Paper

Rethinking the Paradigm of Content Constraints in Unpaired Image-to-Image Translation

Xiuding Cai
Yaoyao Zhu
Dong Miao
Linjie Fu
Yu Yao

In an unpaired setting, lacking sufficient content constraints for image-to-image translation (I2I) tasks, GAN-based approaches are usually prone to model collapse. Current solutions can be divided into two categories, reconstruction-based and Siamese network-based. The former requires that the transformed or transforming image can be perfectly converted back to the original image, which is sometimes too strict and limits the generative performance. The latter involves feeding the original and generated images into a feature extractor and then matching their outputs. This is not efficient enough, and a universal feature extractor is not easily available. In this paper, we propose EnCo, a simple but efficient way to maintain the content by constraining the representational similarity in the latent space of patch-level features from the same stage of the encoder and decoder of the generator. For the similarity function, we use a simple MSE loss instead of contrastive loss, which is currently widely used in I2I tasks. Benefits from the design, EnCo training is extremely efficient, while the features from the encoder produce a more positive effect on the decoding, leading to more satisfying generations. In addition, we rethink the role played by discriminators in sampling patches and propose a discriminative attention-guided (DAG) patch sampling strategy to replace random sampling. DAG is parameter-free and only requires negligible computational overhead, while significantly improving the performance of the model. Extensive experiments on multiple datasets demonstrate the effectiveness and advantages of EnCo, and we achieve multiple state-of-the-art compared to previous methods.

PDF Details DOI

JBHI Journal 2024 Journal Article

Towards Real-World Applications of Personalized Anesthesia Using Policy Constraint Q Learning for Propofol Infusion Control

Xiuding Cai
Jiao Chen
Yaoyao Zhu
Beimin Wang
Yu Yao

Automated anesthesia promises to enable more personalized and precise anesthetic administration and free anesthesiologists from repetitive tasks, allowing them to focus on the most critical aspects of surgical care for patients. Current research has typically focused on creating simulated environments from which agents can learn. These approaches have demonstrated good experimental results, but are still far from clinical application. In this paper, Policy Constraint Q-Learning (PCQL), a data-driven reinforcement learning algorithm for solving the problem of learning strategies on real world anesthesia data, is proposed. Conservative Q-Learning is first introduced to alleviate the problem of Q function overestimation in an offline context. A policy constraint term is then added to agent training to keep the action distribution of the agent and the anesthesiologist consistent, ensuring the agent makes safer decisions in anesthesia scenarios. The effectiveness of PCQL was validated by extensive experiments on a real clinical anesthesia dataset we collected. Experimental results show that PCQL is predicted to achieve higher gains than the baseline approaches while maintaining good agreement with the reference dose given by the anesthesiologist, using less total dose, and being more responsive to the patient's vital signs. In addition, the confidence intervals of the agent were investigated and were founded to cover most of the clinical decisions of the anesthesiologist. Finally, an interpretable method, SHAP, was used to analyze the contributing components of the model predictions, increasing the transparency of the model.

Details DOI

NeurIPS Conference 2023 Conference Paper

CS-Isolate: Extracting Hard Confident Examples by Content and Style Isolation

Yexiong Lin
Yu Yao
Xiaolong Shi
Mingming Gong
Xu Shen
Dong Xu
Tongliang Liu

Label noise widely exists in large-scale image datasets. To mitigate the side effects of label noise, state-of-the-art methods focus on selecting confident examples by leveraging semi-supervised learning. Existing research shows that the ability to extract hard confident examples, which are close to the decision boundary, significantly influences the generalization ability of the learned classifier. In this paper, we find that a key reason for some hard examples being close to the decision boundary is due to the entanglement of style factors with content factors. The hard examples become more discriminative when we focus solely on content factors, such as semantic information, while ignoring style factors. Nonetheless, given only noisy data, content factors are not directly observed and have to be inferred. To tackle the problem of inferring content factors for classification when learning with noisy labels, our objective is to ensure that the content factors of all examples in the same underlying clean class remain unchanged as their style information changes. To achieve this, we utilize different data augmentation techniques to alter the styles while regularizing content factors based on some confident examples. By training existing methods with our inferred content factors, CS-Isolate proves their effectiveness in learning hard examples on benchmark datasets. The implementation is available at https: //github. com/tmllab/2023 NeurIPS CS-isolate.

PDF Details

IJCAI Conference 2021 Conference Paper

Coupling Intent and Action for Pedestrian Crossing Behavior Prediction

Yu Yao
Ella Atkins
Matthew Johnson-Roberson
Ram Vasudevan
Xiaoxiao Du

Accurate prediction of pedestrian crossing behaviors by autonomous vehicles can significantly improve traffic safety. Existing approaches often model pedestrian behaviors using trajectories or poses but do not offer a deeper semantic interpretation of a person's actions or how actions influence a pedestrian's intention to cross in the future. In this work, we follow the neuroscience and psychological literature to define pedestrian crossing behavior as a combination of an unobserved inner will (a probabilistic representation of binary intent of crossing vs. not crossing) and a set of multi-class actions (e. g. , walking, standing, etc. ). Intent generates actions, and the future actions in turn reflect the intent. We present a novel multi-task network that predicts future pedestrian actions and uses predicted future action as a prior to detect the present intent and action of the pedestrian. We also designed an attention relation network to incorporate external environmental contexts thus further improve intent and action detection performance. We evaluated our approach on two naturalistic driving datasets, PIE and JAAD, and extensive experiments show significantly improved and more explainable results for both intent detection and action prediction over state-of-the-art approaches. Our code is available at: https: //github. com/umautobots/pedestrian_intent_action_detection

PDF Details DOI

NeurIPS Conference 2021 Conference Paper

Instance-dependent Label-noise Learning under a Structural Causal Model

Yu Yao
Tongliang Liu
Mingming Gong
Bo Han
Gang Niu
Kun Zhang

Label noise generally degenerates the performance of deep learning algorithms because deep neural networks easily overfit label errors. Let $X$ and $Y$ denote the instance and clean label, respectively. When $Y$ is a cause of $X$, according to which many datasets have been constructed, e. g. , \textit{SVHN} and \textit{CIFAR}, the distributions of $P(X)$ and $P(Y|X)$ are generally entangled. This means that the unsupervised instances are helpful to learn the classifier and thus reduce the side effect of label noise. However, it remains elusive on how to exploit the causal information to handle the label-noise problem. We propose to model and make use of the causal process in order to correct the label-noise effect. Empirically, the proposed method outperforms all state-of-the-art methods on both synthetic and real-world label-noise datasets.

PDF Details

NeurIPS Conference 2020 Conference Paper

Dual T: Reducing Estimation Error for Transition Matrix in Label-noise Learning

Yu Yao
Tongliang Liu
Bo Han
Mingming Gong
Jiankang Deng
Gang Niu
Masashi Sugiyama

The transition matrix, denoting the transition relationship from clean labels to noisy labels, is essential to build statistically consistent classifiers in label-noise learning. Existing methods for estimating the transition matrix rely heavily on estimating the noisy class posterior. However, the estimation error for noisy class posterior could be large because of the randomness of label noise. The estimation error would lead the transition matrix to be poorly estimated. Therefore in this paper, we aim to solve this problem by exploiting the divide-and-conquer paradigm. Specifically, we introduce an intermediate class to avoid directly estimating the noisy class posterior. By this intermediate class, the original transition matrix can then be factorized into the product of two easy-to-estimated transition matrices. We term the proposed method as the dual $T$-estimator. Both theoretical analyses and empirical results illustrate the effectiveness of the dual $T$-estimator for estimating transition matrices, leading to better classification performances.

PDF Details

EAAI Journal 2019 Journal Article

Segmentation of casting defect regions for the extraction of microstructural properties

Jinhua Lin
Lin Ma
Yu Yao

Extracting microstructural properties of casting defect regions plays an important role in quality control efforts for casting production. However, it is not easy to extract microstructural properties via the existing extraction strategies because the microstructures of casting defect regions are extremely complex and irregular. In this paper, a 3D convolutional neural network, a nonlinear topological dimension parameter and an empirical model are proposed for extracting the microstructural properties of casting defect regions efficiently. First, taking the 3D region proposal network (RPN), the instance segmentation network (ISN) and the 3D RoIAlign layer as three subnetworks, a 3D convolutional neural network is constructed for the initial segmentation of casting defect regions, and the geometric features of casting defect regions are further characterized according to the nonlinear topological dimension parameter. In the end, based on the nonlinear topological dimension parameter, an empirical model is established for extracting four important microstructural properties of casting defect regions. The experimental results demonstrate that microstructural properties of casting defect regions can be extracted via this method.

Details DOI

YNIMG Journal 2018 Journal Article

Variational Bayesian inversion for hierarchical unsupervised generative embedding (HUGE)

Yu Yao
Sudhir S. Raman
Michael Schiek
Alex Leff
Stefan Frässle
Klaas E. Stephan

A recently introduced hierarchical generative model unified the inference of effective connectivity in individual subjects and the unsupervised identification of subgroups defined by connectivity patterns. This hierarchical unsupervised generative embedding (HUGE) approach combined a hierarchical formulation of dynamic causal modelling (DCM) for fMRI with Gaussian mixture models and relied on Markov chain Monte Carlo (MCMC) sampling for inference. While well suited for the inversion of complex hierarchical models, MCMC-based sampling suffers from a computational burden that is prohibitive for many applications. To address this problem, this paper derives an efficient variational Bayesian (VB) inversion scheme for HUGE that simultaneously provides approximations to the posterior distribution over model parameters and to the log model evidence. The face validity of the VB scheme was tested using two synthetic fMRI datasets with known ground truth. Additionally, an empirical fMRI dataset of stroke patients and healthy controls was used to evaluate the practical utility of the method in application to real-world problems. Our analyses demonstrate good performance of our VB scheme, with a marked speed-up of model inversion by two orders of magnitude compared to MCMC, while maintaining a similar level of accuracy. Notably, additional acceleration would be possible if parallel computing techniques were applied. Generally, our VB implementation of HUGE is fast enough to support multi-start procedures for whole-group analyses, a useful strategy to ameliorate problems with local extrema. HUGE thus represents a potentially useful practical solution for an important problem in clinical neuromodeling and computational psychiatry, i. e. , the unsupervised detection of subgroups in heterogeneous populations that are defined by effective connectivity.

Details DOI

JBHI Journal 2014 Journal Article

Model-Based Verification of a Non-Linear Separation Scheme for Ballistocardiography

Yu Yao
Christoph Brüser
Uwe Pietrzyk
Steffen Leonhardt
Stefan van Waasen
Michael Schiek

The current rise in popularity of ballisto-cardiography-related research has led to the development of new sensor concepts and recording methods. Measuring the ballistocardiogram using bed mounted pressure sensors opens up new possibilities for home monitoring applications. The signals measured with these sensors contain a mixture of cardiac and respiratory components, which can be used for detection of comorbidities of heart failure like apnea or arrhythmia. However, the separation of the cardiac and respiratory components has proven to be difficult, since there is significant overlap in the spectra of both components. In this paper, an algorithm for the separation task is presented, which can overcome the problem of overlapping spectra. Additionally, a model has been developed for the generation of artificial ballistocardiograms, which are used to analyze the separation performance. Furthermore, the algorithm is tested on preliminary data from a clinical study.

Details DOI