Author name cluster

Kai He

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

11 papers

1 author row

AAAI Conference 2026 Conference Paper

MARS: Multi-Agent Adaptive Reasoning with Socratic Guidance for Automated Prompt Optimization

Jian Zhang
Zhangqi Wang
Haiping Zhu
Kangda Cheng
Kai He
Bo Li
Qika Lin
Jun Liu

Large language models (LLMs) typically operate in a question-answering paradigm, where the quality of the input prompt critically affects the response. Automated Prompt Optimization (APO) aims to overcome the cognitive biases of manually crafted prompts and explore a broader prompt design space. However, existing APO methods often suffer from rigid template structures and inefficient exploration in the prompt space. To this end, we propose a Multi-Agent Adaptive Reasoning with Socratic guidance framework (MARS) for APO. MARS consists of five complementary agents and formulates the optimization process as a Partially Observable Markov Decision Process (POMDP), enabling adaptive prompt refinement through explicit state modeling and interactive feedback. Specifically, a Planner agent generates flexible optimization trajectories, a Teacher-Critic-Student triad engages in Socratic-style dialogue to iteratively optimize the prompt based on pseudo-gradient signals in the text space, and a Target agent executes the prompt in downstream tasks to provide performance feedback. MARS integrates reasoning, feedback, and state transition into a unified hidden-state evolution process, improving both the effectiveness and interpretability of optimization. Extensive experiments on multiple datasets demonstrate that MARS outperforms existing APO methods in terms of optimization performance, search efficiency, and interpretability.

PDF Details DOI

JBHI Journal 2026 Journal Article

Multi-Atlas Brain Network Classification Through Consistency Distillation and Complementary Information Fusion

Jiaxing Xu
Mengcheng Lan
Xia Dong
Kai He
Wei Zhang
Qingtian Bian
Yiping Ke

Brain network analysis plays a crucial role in identifying distinctive patterns associated with neurological disorders. Functional magnetic resonance imaging (fMRI) enables the construction of brain networks by analyzing correlations in blood-oxygen-level-dependent (BOLD) signals across different brain regions, known as regions of interest (ROIs). These networks are typically constructed using atlases that parcellate the brain based on various hypotheses of functional and anatomical divisions. However, there is no standard atlas for brain network classification, leading to limitations in detecting abnormalities in disorders. Recent methods leveraging multiple atlases fail to ensure consistency across atlases and lack effective ROI-level information exchange, limiting their efficacy. To address these challenges, we propose the Atlas-Integrated Distillation and Fusion network (AIDFusion), a novel framework designed to enhance brain network classification using fMRI data. AIDFusion introduces a disentangle Transformer to filter out inconsistent atlas-specific information and distill meaningful cross-atlas connections. Additionally, it enforces subject- and population-level consistency constraints to improve cross-atlas coherence. To further enhance feature integration, AIDFusion incorporates an inter-atlas message-passing mechanism that facilitates the fusion of complementary information across brain regions. We evaluate AIDFusion on four resting-state fMRI datasets encompassing different neurological disorders. Experimental results demonstrate its superior classification performance and computational efficiency compared to state-of-the-art methods. Furthermore, a case study highlights AIDFusion’s ability to extract interpretable patterns that align with established neuroscience findings, reinforcing its potential as a robust tool for multi-atlas brain network analysis.

Details DOI

AAAI Conference 2026 Conference Paper

Recovering Coherent Affective Patterns: Addressing Modality Missing in Multimodal Sentiment Analysis

Huiting Huang
Tieliang Gong
Kai He
Wen Wen
Weizhan Zhang
Mengling Feng

Multimodal sentiment analysis (MSA) seeks to decode human emotions by integrating heterogeneous modalities. However, real-world scenarios often involve missing or misaligned data due to sensor failures or transmission errors, leading to disrupted temporal dynamics and degraded cross-modal correlations. To address these challenges, we propose RECAP (REcovery of Coherent Affective Patterns), a robust two-stage framework to restore temporal and structural emotional integrity under modality incompleteness. The first stage employs a causality-aware adversarial generator for multi-granularity temporal reconstruction, complemented by a contrastive mutual information factorization module that disentangles shared and modality-specific semantics. The second stage introduces a mutual information-guided attention fusion mechanism with a ranking-based objective, enabling adaptive integration of complementary signals for refined prediction. Extensive experiments on MOSI, MOSEI, and SIMS under various missing-modality conditions demonstrate that RECAP consistently outperforms state-of-the-art methods. Notably, it improves ACC-7 on MOSI by 2.71 percentage points and F1 on SIMS by 6.38 percentage points. These results verify the performance of RECAP in terms of capturing fine-grained emotional cues and robustness.

PDF Details DOI

NeurIPS Conference 2025 Conference Paper

GEM: Empowering MLLM for Grounded ECG Understanding with Time Series and Images

Xiang Lan
Feng Wu
Kai He
Qinghao Zhao
Shenda Hong
Mengling Feng

While recent multimodal large language models (MLLMs) have advanced automated ECG interpretation, they still face two key limitations: (1) insufficient multimodal synergy between ECG time series and ECG images, and (2) limited explainability in linking diagnoses to granular waveform evidence. We introduce GEM, the first MLLM unifying ECG time series, 12-lead ECG images and text for grounded and clinician-aligned ECG interpretation. GEM enables feature-grounded analysis, evidence-driven reasoning, and a clinician-like diagnostic process through three core innovations: a dual-encoder framework extracting complementary time series and image features, cross-modal alignment for effective multimodal understanding, and knowledge-guided instruction data generation for generating high-granularity grounding data (ECG-Grounding) linking diagnoses to measurable parameters ($e. g. $, QRS/PR Intervals). Additionally, we propose the Grounded ECG Understanding task, a clinically motivated benchmark designed to comprehensively assess the MLLM's capability in grounded ECG understanding. Experimental results on both existing and our proposed benchmarks show GEM significantly improves predictive performance (CSN $7. 4\%$ $\uparrow$), explainability ($22. 7\%$ $\uparrow$), and grounding ($25. 3\%$ $\uparrow$), making it a promising approach for real-world clinical applications. Codes, model, and data are available at https: //github. com/lanxiang1017/GEM.

PDF Details

NeurIPS Conference 2025 Conference Paper

LuxDiT: Lighting Estimation with Video Diffusion Transformer

Ruofan Liang
Kai He
Zan Gojcic
Igor Gilitschenski
Sanja Fidler
Nandita Vijaykumar
Zian Wang

Estimating scene lighting from a single image or video remains a longstanding challenge in computer vision and graphics. Learning-based approaches are constrained by the scarcity of ground-truth HDR environment maps, which are expensive to capture and limited in diversity. While recent generative models offer strong priors for image synthesis, lighting estimation remains difficult due to its reliance on indirect visual cues, the need to infer global (non-local) context, and the recovery of high-dynamic-range outputs. We propose LuxDiT, a novel data-driven approach that fine-tunes a video diffusion transformer to generate HDR environment maps conditioned on visual input. Trained on a large synthetic dataset with diverse lighting conditions, our model learns to infer illumination from indirect visual cues and generalizes effectively to real-world scenes. To improve semantic alignment between the input and the predicted environment map, we introduce a low-rank adaptation finetuning strategy using a collected dataset of HDR panoramas. Our method produces accurate lighting predictions with realistic angular high-frequency details, outperforming existing state-of-the-art techniques in both quantitative and qualitative evaluations.

PDF Details

JBHI Journal 2025 Journal Article

SecProGNN: Predicting Bronchoalveolar Lavage Fluid Secreted Protein Using Graph Neural Network

Dan Shao
Guangzhao Zhang
Lin Lin
Yucong Xiong
Kai He
Liyan Sun

Bronchoalveolar lavage fluid (BALF) is a liquid obtained from the alveoli and bronchi, often used to study pulmonary diseases. So far, proteomic analyses have identified over three thousand proteins in BALF. However, the comprehensive characterization of these proteins remains challenging due to their complexity and technological limitations. This paper presented a novel deep learning framework called SecProGNN, designed to predict secretory proteins in BALF. Firstly, SecProGNN represented proteins as graph-structured data, with amino acids connected based on their interactions. Then, these graphs were processed through graph neural networks (GNNs) model to extract graph features. Finally, the extracted feature vectors were fed into a multi-layer perceptron (MLP) module to predict BALF secreted proteins. Additionally, by utilizing SecProGNN, we investigated potential biomarkers for lung adenocarcinoma and identified 16 promising candidates that may be secreted into BALF.

Details DOI

JBHI Journal 2025 Journal Article

Towards Smarter Clinical Predictions: Foundation Models for Integrating Multi-Source Domain Knowledge in EHRs

Jialun Wu
Kai He
Zeyu Gao
Xuequn Shang
Mengling Feng

Medication recommendation (MR) is a crucial clinical prediction task that supports physicians in prescribing effectively for patients with complex conditions. Effective MR requires integrating diverse medical knowledge and intricate relationships within clinical data. Traditional methods, however, largely focus on modeling relationships among medical entities and learning patient representations, often constrained by their reliance on single-domain knowledge. These constraints limit their ability to address the complexities of real-world scenarios. Recent advancements in foundation models present a promising solution by unifying multi-source data and uncovering rich semantic and structural insights. Yet, their potential in MR remains underexplored. To overcome these challenges, we introduce MEDICS, a foundation-model-driven framework that integrates multi-source domain knowledge and unifies semantic and structural information for enhanced clinical predictions. MEDICS consists of three synergistic modules: KnowRetr for multi-source medical knowledge retrieval, MedChat for LLM-based semantic reasoning, and HyperMed for high-order structural modeling using sub-hypergraphs. MEDICS employs a systematic process to retrieve, organize, and summarize domain knowledge for each medical code, utilizing this knowledge through two specialized modules. The MedChat module leverages large language models (LLMs) to redefine MR as a conversational question-answering task, enabling advanced semantic analysis. Simultaneously, the HyperMed module refines a hypergraph-based model by incorporating textual and structural embeddings to model high-order interactions among medical codes. Additionally, our collaborative training strategy further synergistically harnesses information from both modules. Extensive experiments on real-world EHR datasets (MIMIC-III and MIMIC-IV) demonstrate that MEDICS significantly outperforms state-of-the-art methods, setting a new benchmark for clinical prediction tasks.

Details DOI

NeurIPS Conference 2025 Conference Paper

UniRelight: Learning Joint Decomposition and Synthesis for Video Relighting

Kai He
Ruofan Liang
Jacob Munkberg
Jon Hasselgren
Nandita Vijaykumar
Alexander Keller
Sanja Fidler
Igor Gilitschenski

We address the challenge of relighting a single image or video, a task that demands precise scene intrinsic understanding and high-quality light transport synthesis. Existing end-to-end relighting models are often limited by the scarcity of paired multi-illumination data, restricting their ability to generalize across diverse scenes. Conversely, two-stage pipelines that combine inverse and forward rendering can mitigate data requirements but are susceptible to error accumulation and often fail to produce realistic outputs under complex lighting conditions or with sophisticated materials. In this work, we introduce a general-purpose approach that jointly estimates albedo and synthesizes relit outputs in a single pass, harnessing the generative capabilities of video diffusion models. This joint formulation enhances implicit scene comprehension and facilitates the creation of realistic lighting effects and intricate material interactions, such as shadows, reflections, and transparency. Trained on synthetic multi-illumination data and extensive automatically labeled real-world videos, our model demonstrates strong generalization across diverse domains and surpasses previous methods in both visual fidelity and temporal consistency. Our project page is https: //research. nvidia. com/labs/toronto-ai/UniRelight/.

PDF Details

AAAI Conference 2023 Conference Paper

COCA: COllaborative CAusal Regularization for Audio-Visual Question Answering

Mingrui Lao
Nan Pu
Yu Liu
Kai He
Erwin M. Bakker
Michael S. Lew

Audio-Visual Question Answering (AVQA) is a sophisticated QA task, which aims at answering textual questions over given video-audio pairs with comprehensive multimodal reasoning. Through detailed causal-graph analyses and careful inspections of their learning processes, we reveal that AVQA models are not only prone to over-exploit prevalent language bias, but also suffer from additional joint-modal biases caused by the shortcut relations between textual-auditory/visual co-occurrences and dominated answers. In this paper, we propose a COllabrative CAusal (COCA) Regularization to remedy this more challenging issue of data biases. Specifically, a novel Bias-centered Causal Regularization (BCR) is proposed to alleviate specific shortcut biases by intervening bias-irrelevant causal effects, and further introspect the predictions of AVQA models in counterfactual and factual scenarios. Based on the fact that the dominated bias impairing model robustness for different samples tends to be different, we introduce a Multi-shortcut Collaborative Debiasing (MCD) to measure how each sample suffers from different biases, and dynamically adjust their debiasing concentration to different shortcut correlations. Extensive experiments demonstrate the effectiveness as well as backbone-agnostic ability of our COCA strategy, and it achieves state-of-the-art performance on the large-scale MUSIC-AVQA dataset.

PDF Details DOI

NeurIPS Conference 2021 Conference Paper

Learning Interpretable Decision Rule Sets: A Submodular Optimization Approach

Fan Yang
Kai He
Linxiao Yang
Hongxia Du
Jingbang Yang
Bo Yang
Liang Sun

Rule sets are highly interpretable logical models in which the predicates for decision are expressed in disjunctive normal form (DNF, OR-of-ANDs), or, equivalently, the overall model comprises an unordered collection of if-then decision rules. In this paper, we consider a submodular optimization based approach for learning rule sets. The learning problem is framed as a subset selection task in which a subset of all possible rules needs to be selected to form an accurate and interpretable rule set. We employ an objective function that exhibits submodularity and thus is amenable to submodular optimization techniques. To overcome the difficulty arose from dealing with the exponential-sized ground set of rules, the subproblem of searching a rule is casted as another subset selection task that asks for a subset of features. We show it is possible to write the induced objective function for the subproblem as a difference of two submodular (DS) functions to make it approximately solvable by DS optimization algorithms. Overall, the proposed approach is simple, scalable, and likely to be benefited from further research on submodular optimization. Experiments on real datasets demonstrate the effectiveness of our method.

PDF Details

TCS Journal 2016 Journal Article

On the security of two identity-based conditional proxy re-encryption schemes

Kai He
Jian Weng
Robert H. Deng
Joseph K. Liu

Proxy re-encryption allows a semi-trusted proxy with a re-encryption key to convert a delegator's ciphertext into a delegatee's ciphertext, and the semi-trusted proxy cannot learn anything about the underlying plaintext. If a proxy re-encryption scheme is indistinguishable against chosen-ciphertext attacks, its initialized ciphertext should be non-malleable. Otherwise, there might exist an adversary who can break the chosen-ciphertext security of the scheme. Recently, Liang et al. proposed two proxy re-encryption schemes. They claimed that their schemes were chosen-ciphertext secure in the standard model. However, we find that the original ciphertext in their schemes are malleable. Thus, we present some concrete attacks and indicate their schemes fail to achieve chosen-ciphertext security in the standard model.

Details DOI