Author name cluster

Xinyi Wang

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

24 papers

2 author rows

YNIMG Journal 2026 Journal Article

Alcohol impacts an fMRI marker of neural inhibition in humans and rodents

Monami Nishio
Xinyi Wang
Eli J. Cornblath
Sung-Ho Lee
Yen-Yu Ian Shih
Nicola Palomero-Gallagher
Michael J. Arcaro
David M. Lydon-Staley

Inhibitory neuronal activity is a key regulator of brain function and is implicated in numerous developmental and psychiatric disorders. However, measuring inhibition in vivo remains a challenge. The Hurst exponent of the fMRI signal has been shown to correlate spatially with cellular measures of neural inhibition, but there have been few causal tests of the relationship between the Hurst exponent and neural inhibition. Here, we used alcohol, a drug with known impacts on inhibition, as a way to evaluate the validity of the Hurst exponent as a marker of neural inhibition across rats and humans. In rats, acute alcohol administration significantly reduced the cortical Hurst exponent, with the spatial distribution of effects closely aligned with GABAA receptor expression. In humans, alcohol exposure similarly decreased the cortical Hurst exponent, particularly in regions with high GABAA receptor expression, demonstrating cross-species consistency. These results provide convergentin vivo evidence that the Hurst exponent is sensitive to pharmacologically induced changes in inhibitory neuronal activity.

Details DOI

AAAI Conference 2026 Conference Paper

Difficulty Is Not Enough: Curriculum Learning for LLMs Fine-tuning Must Consider Utility

Zishang Jiang
Jinyi Han
Tingyun Li
Xinyi Wang
Sihang Jiang
Xiaojun Meng
Jiansheng Wei
Jiaqing Liang

Fine-tuning plays an essential role in improving the performance of large language models (LLMs) on specific tasks. A central challenge lies in designing data-efficient strategy to achieve better fine-tuning performance. Curriculum learning, which organizes data from easy to hard, has become a widely adopted technique in LLMs training. However, existing methods for curriculum learning focus only on the difficulty of samples, while neglecting their contribution to improving model performance, making them vulnerable when applied to fine-tuning LLMs. To address this, we propose Difficulty-Utility Curriculum Learning (DUCL), a curriculum learning framework that jointly considers difficulty and utility. DUCL introduces a novel scoring method, Difficulty-Utility Evaluation (DUE), and a soft scheduling strategy called Window Ordering, which together promote efficient and effective fine-tuning. Our method not only improves convergence and final performance with negligible computational overhead, but is also broadly applicable across a wide range of tasks, making it a practical and scalable solution for LLMs fine-tuning.

PDF Details DOI

AAAI Conference 2026 Conference Paper

DPRM: A Dual Implicit Process Reward Model in Multi-Hop Question Answering

Xinyi Wang
Yiping Song
Zhiliang Tian
Bo Liu
Tingjin Luo
Minlie Huang

In multi-hop question answering (MHQA) tasks, Chain of Thought (CoT) improves the quality of generation by guiding large language models (LLMs) through multi-step reasoning, and Knowledge Graphs (KGs) reduce hallucinations via semantic matching. Outcome Reward Models (ORMs) provide feedback after generating the final answers but fail to evaluate the process for multi-step reasoning. Traditional Process Reward Models (PRMs) evaluate the reasoning process but require costly human annotations or rollout generation. While implicit PRM is trained only with outcome signals and derives step rewards through reward parameterization without explicit annotations, it is more suitable for multi-step reasoning in MHQA tasks. However, existing implicit PRM has only been explored for plain text scenarios. When adapting to MHQA tasks, it cannot handle the graph structure constraints in KGs and capture the potential inconsistency between CoT and KG paths. To address these limitations, we propose the DPRM (Dual Implicit Process Reward Model). It trains two implicit PRMs for CoT and KG reasoning in MHQA tasks. Both PRMs, namely KG-PRM and CoT-PRM, derive step-level rewards from outcome signals via reward parameterization without additional explicit annotations. Among them, KG-PRM uses preference pairs to learn structural constraints from KGs. DPRM further introduces a consistency constraint between CoT and KG reasoning steps, making the two PRMs mutually verify and collaboratively optimize the reasoning paths. We also provide a theoretical demonstration of the derivation of process rewards. Experimental results show that our method outperforms 13 baselines on multiple datasets with up to 16.6% improvement on Hit@1.

PDF Details DOI

AAAI Conference 2026 Conference Paper

HTTrack: Learning to Perceive Targets via Historical Trajectories in Satellite Video Tracking

Jiahao Wang
Fang Liu
Licheng Jiao
Hao Wang
Shuo Li
Xinyi Wang
Lingling Li
Puhua Chen

In recent years, the rapid progress of deep learning has driven notable advancements in satellite video tracking, a critical task for applications such as environmental monitoring, disaster management, and defense. Despite these strides, existing approaches remain constrained by their inability to handle dynamic challenges, such as target appearance variations, complex motion patterns, and occlusions. Traditional methods often suffer from static template matching or overly complex update mechanisms, compromising their robustness and practicality in real-world scenarios. To address these limitations, we propose a paradigm shift in satellite video tracking by integrating historical trajectory knowledge with visual features. This fusion enhances the tracker's perceptual understanding of targets over time, enabling more adaptive and resilient tracking. By aligning spatial, temporal, and cross-modal information, our approach effectively bridges the gap between fragmented observations and coherent tracking performance, even under challenging conditions like small target detection and cluttered backgrounds. Extensive experiments conducted on multiple satellite video tracking benchmarks demonstrate the superiority of our method, with HTTrack achieving success rates of 51.5% on SV248S, 52.9% on SatSOT, and 32.6% on VISO, significantly outperforming state-of-the-art trackers and marking a step forward in achieving robust, accurate, and scalable satellite video tracking.

PDF Details DOI

TMLR Journal 2026 Journal Article

KITTEN: A Knowledge-Integrated Evaluation of Image Generation on Visual Entities

Hsin-Ping Huang
Xinyi Wang
Yonatan Bitton
Hagai Taitelbaum
Gaurav Singh Tomar
Ming-Wei Chang
Xuhui Jia
Kelvin C.K. Chan

Recent advances in text-to-image generation have improved the quality of synthesized images, but evaluations mainly focus on aesthetics or alignment with text prompts. Thus, it remains unclear whether these models can accurately represent a wide variety of realistic visual entities. To bridge this gap, we propose KITTEN, a benchmark for Knowledge-InTegrated image generaTion on real-world ENtities. Using KITTEN, we conduct a systematic study of recent text-to-image models, retrieval-augmented models, and unified understanding and generation models, focusing on their ability to generate real-world visual entities such as landmarks and animals. Analyses using carefully designed human evaluations, automatic metrics, and MLLMs as judges show that even advanced text-to-image and unified models fail to generate accurate visual details of entities. While retrieval-augmented models improve entity fidelity by incorporating reference images, they tend to over-rely on them and struggle to create novel configurations of the entities in creative text prompts. The dataset and evaluation code are publicly available at https://kitten-project.github.io.

PDF Details

AAAI Conference 2026 Conference Paper

Semantic Feature Purification for Adversarially-Aware RGB-T Tracking

Jiahao Wang
Fang Liu
Hao Wang
Shuo Li
Xinyi Wang
Puhua Chen

RGB-T tracking is increasingly deployed in safety-critical applications such as autonomous driving, surveillance, and rescue robotics, where tracking reliability is essential under adverse conditions. Although the fusion of RGB and thermal infrared (TIR) modalities offers improved robustness in low-light and occluded scenes, recent findings show that RGB-T trackers remain highly susceptible to subtle input perturbations, human-imperceptible modifications that exploit cross-modal inconsistencies to mislead tracking outputs. In real-world scenarios, such perturbations can arise from sensor spoofing, infrared camouflage, or physical-world attacks, posing serious risks to operational safety. To address this, we propose SFPT, a Semantic Feature Purification framework that enhances RGB-T tracking at the representation level. Rather than filtering corrupted inputs at the pixel level, SFPT introduces task-specific semantic anchors into the feature space to reinforce perturbation-invariant cues. These anchors are derived from descriptive language, interact with visual features to purify representations. To further suppress modality-specific interference, we design an Adaptive Perturbation-Guided Cross-Modal Fusion (APG-CMF) module, which leverages language and visual signals to estimate reliability and dynamically reweight cross-modal features, ensuring robust fusion under perturbation conditions. Extensive experiments under diverse perturbation conditions validate the effectiveness of our approach. Notably, SFPT maintains performance comparable to clean settings even when subjected to perturbations of strength 1/255 and 4/255, demonstrating strong resilience to real-world interference.

PDF Details DOI

NeurIPS Conference 2025 Conference Paper

AffordBot: 3D Fine-grained Embodied Reasoning via Multimodal Large Language Models

Xinyi Wang
Xun Yang
Yanlong Xu
Yuchen Wu
Zhen Li
Na Zhao

Effective human-agent collaboration in physical environments requires understanding not only what to act upon, but also where the actionable elements are and how to interact with them. Existing approaches often operate at the object level or disjointedly handle fine-grained affordance reasoning, lacking coherent, instruction-driven grounding and reasoning. In this work, we introduce a new task: Fine-grained 3D Embodied Reasoning, which requires an agent to predict, for each referenced affordance element in a 3D scene, a structured triplet comprising its spatial location, motion type, and motion axis, based on a task instruction. To solve this task, we propose AffordBot, a novel framework that integrates Multimodal Large Language Models (MLLMs) with a tailored chain-of-thought (CoT) reasoning paradigm. To bridge the gap between 3D input and 2D-compatible MLLMs, we render surround-view images of the scene and project 3D element candidates into these views, forming a rich visual representation aligned with the scene geometry. Our CoT pipeline begins with an active perception stage, prompting the MLLM to select the most informative viewpoint based on the instruction, before proceeding with step-by-step reasoning to localize affordance elements and infer plausible interaction motions. Evaluated on the SceneFun3D dataset, AffordBot achieves state-of-the-art performance, demonstrating strong generalization and physically grounded reasoning with only 3D point cloud input and MLLMs. Our code is available at https: //github. com/hannahwxy/AffordBot.

PDF Details

ECAI Conference 2025 Conference Paper

ASMA-Tune: Unlocking LLMs' Assembly Code Comprehension via Structural-Semantic Instruction Tuning

Xinyi Wang
Jiashui Wang
Jinbo Su
Ke Wang
Peng Chen
Yanming Liu
Long Liu
Xiang Li

Assembly code analysis and comprehension play critical roles in applications like reverse engineering, yet they face substantial challenges due to low information density and a lack of explicit syntactic structures. While traditional masked language modeling (MLM) approaches do not explicitly focus on natural language interaction, emerging decoder-focused large language models (LLMs) demonstrate partial success in binary analysis yet remain underexplored for holistic comprehension. We present Assembly Augmented Tuning (ASMA-Tune), an end-to-end structural-semantic instruction tuning framework that synergizes encoder architecture with decoder-based LLMs through a projector module, where the assembly encoder extracts hardware-level structural features, the projector bridges representations with the semantic space, and the instruction-tuned LLM preserves natural language capabilities. Experimental results demonstrate three key advantages: (1) State-of-the-art performance in assembly comprehension with +39. 7% Recall@1 and +17. 8% MRR improvements over GPT-4-Turbo, (2) Consistent enhancements across base models (24. 6–107. 4% Recall@1 and 15. 2–106. 3% MRR on Qwen2. 5-Coder, Deepseek-Coder and CodeLlama variants), and (3) Superior instruction-following capabilities (41. 5%–118% improvements) with controlled code generation degradation (–8. 9% to –35% across architectures).

Details

AAAI Conference 2025 Conference Paper

AugRefer: Advancing 3D Visual Grounding via Cross-Modal Augmentation and Spatial Relation-based Referring

Xinyi Wang
Na Zhao
Zhiyuan Han
Dan Guo
Xun Yang

3D visual grounding (3DVG), which aims to correlate a natural language description with the target object within a 3D scene, is a significant yet challenging task. Despite recent advancements in this domain, existing approaches commonly encounter a shortage: a limited amount and diversity of text-3D pairs available for training. Moreover, they fall short in effectively leveraging different contextual clues (e.g., rich spatial relations within the 3D visual space) for grounding. To address these limitations, we propose AugRefer, a novel approach for advancing 3D visual grounding. AugRefer introduces cross-modal augmentation designed to extensively generate diverse text-3D pairs by placing objects into 3D scenes and creating accurate and semantically rich descriptions using foundation models. Notably, the resulting pairs can be utilized by any existing 3DVG methods for enriching their training data. Besides, AugRefer presents a language-spatial adaptive decoder that effectively adapts the potential referring objects based on the language description and various 3D spatial relations. Extensive experiments on three benchmark datasets clearly validate the effectiveness of AugRefer.

PDF Details DOI

EAAI Journal 2025 Journal Article

Compact-sparse prototype calibration network for few-shot continual fault diagnosis of rotating machinery

Shen Yan
Haidong Shao
Xinyi Wang
Haomiao Zhang
Yiming Xiao
Bin Liu

Rotating machinery inevitably generates only a few samples of new fault categories during long-term operation, which requires the fault diagnosis model to incrementally learn few new categories and retain the existing fault knowledge. Recent few-shot continual fault diagnosis (FSCFD) methods mainly rely on constructing prototype classifiers and generating virtual samples to address the challenges of catastrophic forgetting and overfitting. However, this would ignore the rich feature information in the base session, while the limited incremental data makes it difficult to accurately depict the new category feature information. Therefore, a compact-sparse prototype calibration network (CSPCN) is proposed to improve the diagnosis capacity for new category faults in the FSCFD scenario. First, a compact-sparse base loss (CSBL) is employed to reserve sufficient space for new fault categories by maximizing the variance distribution among base prototypes. Second, an incremental prototype calibration classifier (IPCC) is designed to improve the ability to distinguish new categories by integrating new prototypes with the weighted base prototypes in real time. Extensive experiments conducted on the subway train bogie and variable load gearbox dataset validate the proposed method's exceptional diagnostic performance. Through multidimensional comparisons with state-of-the-art FSCFD methods and rigorous ablation experiments, CSPCN demonstrates significant improvements in effectively identifying and distinguishing new fault categories.

Details DOI

JBHI Journal 2025 Journal Article

Video Object Segmentation with Optimal Frame Auto-selection Based on Prior Knowledge for Midbrain Assessment in Transcranial Ultrasound

Xinyi Wang
Sai Kit LAM
Hongyu KANG
Yu Sun
Chao HOU
Shuai Li
Xin Sun
Fangxian LI

Transcranial sonography (TCS) provides a non-invasive means of assessing movement disorders such as Parkinson's disease (PD). However, current TCS-based evaluations rely heavily on manual operation by experienced physicians, making the process time-consuming and physician-dependent. For the first time, we aimed to develop a hybrid pipeline for real-time video object segmentation (VOS) and automatic optimal frame selection. Eighty-three standardized TCS real-time data comprising 1, 992 midbrain frames from Beijing Tiantan Hospital were collected. We adopted three state-of-the-art VOS models (STCN, RDE-VOS, and XMEM) and incorporated anatomical priors to guide optimal frame selection. Specifically, we leveraged the anatomical trend of midbrain morphology to estimate the midbrain radius at the optimal frame and selected the frame where the VOS-segmented midbrain best matched this estimate. The XMEM-based pipeline achieved high segmentation performance (Jaccard: 0. 85, Boundary Accuracy: 0. 95, Dice: 0. 92) and optimal frame selection (Distance: 4. 87; Jaccard: 0. 92), with efficiency (51. 05 FPS, 0. 56 s/patient, 661. 55 MB). Subgroup analyses confirmed robustness across image quality and PD conditions. Assessment of a junior physician's selection suggests potential to reduce the expertise gap in optimal frame selection. The proposed hybrid pipeline offers an automated tool for midbrain assessment using TCS, which may help reduce physicians' workload and minimize subjectivity, particularly supporting junior physicians in mitigating the expertise-demanding nature of TCS. This approach may serve as a foundation for more promising TCS-based assessments in the future, contributing to broader adoption of non-invasive ultrasound techniques in PD evaluation.

Details DOI

TMLR Journal 2024 Journal Article

A Survey on Data Selection for Language Models

Alon Albalak
Yanai Elazar
Sang Michael Xie
Shayne Longpre
Nathan Lambert
Xinyi Wang
Niklas Muennighoff
Bairu Hou

A major factor in the recent success of large language models is the use of enormous and ever-growing text datasets for unsupervised pre-training. However, naively training a model on all available data may not be optimal (or feasible), as the quality of available text data can vary. Filtering out data can also decrease the carbon footprint and financial costs of training models by reducing the amount of training required. Data selection methods aim to determine which candidate data points to include in the training dataset and how to appropriately sample from the selected data points. The promise of improved data selection methods has caused the volume of research in the area to rapidly expand. However, because deep learning is mostly driven by empirical evidence and experimentation on large-scale data is expensive, few organizations have the resources for extensive data selection research. Consequently, knowledge of effective data selection practices has become concentrated within a few organizations, many of which do not openly share their findings and methodologies. To narrow this gap in knowledge, we present a comprehensive review of existing literature on data selection methods and related research areas, providing a taxonomy of existing approaches. By describing the current landscape of research, this work aims to accelerate progress in data selection by establishing an entry point for new and established researchers. Additionally, throughout this review we draw attention to noticeable holes in the literature and conclude the paper by proposing promising avenues for future research.

PDF Details

TCS Journal 2024 Journal Article

Complexity and enumeration in models of genome rearrangement

Lora Bailey
Heather Smith Blake
Garner Cochran
Nathan Fox
Michael Levet
Reem Mahmoud
Elizabeth Bailey Matson
Inne Singgih

In this paper, we examine the computational complexity of enumeration in certain genome rearrangement models. We first show that the Pairwise Rearrangement problem in the Single Cut-and-Join model (Bergeron et al. , 2010 [8]) is # P -complete under polynomial-time Turing reductions. Next, we show that in the Single Cut or Join model (Feijão and Meidanis, 2011 [21]), the problem of enumerating all medians ( Image 1 ) is logspace-computable ( FL ), improving upon the previous polynomial-time ( FP ) bound of Miklós & Smith [41].

Details DOI

AAAI Conference 2024 Conference Paper

MFABA: A More Faithful and Accelerated Boundary-Based Attribution Method for Deep Neural Networks

Zhiyu Zhu
Huaming Chen
Jiayu Zhang
Xinyi Wang
Zhibo Jin
Minhui Xue
Dongxiao Zhu
Kim-Kwang Raymond Choo

To better understand the output of deep neural networks (DNN), attribution based methods have been an important approach for model interpretability, which assign a score for each input dimension to indicate its importance towards the model outcome. Notably, the attribution methods use the ax- ioms of sensitivity and implementation invariance to ensure the validity and reliability of attribution results. Yet, the ex- isting attribution methods present challenges for effective in- terpretation and efficient computation. In this work, we in- troduce MFABA, an attribution algorithm that adheres to ax- ioms, as a novel method for interpreting DNN. Addition- ally, we provide the theoretical proof and in-depth analy- sis for MFABA algorithm, and conduct a large scale exper- iment. The results demonstrate its superiority by achieving over 101.5142 times faster speed than the state-of-the-art at- tribution algorithms. The effectiveness of MFABA is thor- oughly evaluated through the statistical analysis in compar- ison to other methods, and the full implementation package is open-source at: https://github.com/LMBTough/MFABA.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

T2V-Turbo: Breaking the Quality Bottleneck of Video Consistency Model with Mixed Reward Feedback

Jiachen Li
Weixi Feng
Tsu-Jui Fu
Xinyi Wang
Sugato Basu
Wenhu Chen
William Y. Wang

Diffusion-based text-to-video (T2V) models have achieved significant success but continue to be hampered by the slow sampling speed of their iterative sampling processes. To address the challenge, consistency models have been proposed to facilitate fast inference, albeit at the cost of sample quality. In this work, we aim to break the quality bottleneck of a video consistency model (VCM) to achieve both fast and high-quality video generation. We introduce T2V-Turbo, which integrates feedback from a mixture of differentiable reward models into the consistency distillation (CD) process of a pre-trained T2V model. Notably, we directly optimize rewards associated with single-step generations that arise naturally from computing the CD loss, effectively bypassing the memory constraints imposed by backpropagating gradients through an iterative sampling process. Remarkably, the 4-step generations from our T2V-Turbo achieve the highest total score on VBench, even surpassing Gen-2 and Pika. We further conduct human evaluations to corroborate the results, validating that the 4-step generations from our T2V-Turbo are preferred over the 50-step DDIM samples from their teacher models, representing more than a tenfold acceleration while improving video generation quality.

PDF Details DOI

ICML Conference 2024 Conference Paper

The Good, The Bad, and Why: Unveiling Emotions in Generative AI

Cheng Li
Jindong Wang 0001
Yixuan Zhang 0001
Kaijie Zhu
Xinyi Wang
Wenxin Hou
Jianxun Lian
Fang Luo

Emotion significantly impacts our daily behaviors and interactions. While recent generative AI models, such as large language models, have shown impressive performance in various tasks, it remains unclear whether they truly comprehend emotions and why. This paper aims to address this gap by incorporating psychological theories to gain a holistic understanding of emotions in generative AI models. Specifically, we propose three approaches: 1) EmotionPrompt to enhance AI model performance, 2) EmotionAttack to impair AI model performance, and 3) EmotionDecode to explain the effects of emotional stimuli, both benign and malignant. Through extensive experiments involving language and multi-modal models on semantic understanding, logical reasoning, and generation tasks, we demonstrate that both textual and visual EmotionPrompt can boost the performance of AI models while EmotionAttack can hinder it. More importantly, EmotionDecode reveals that AI models can comprehend emotional stimuli akin to the mechanism of dopamine in the human brain. Our work heralds a novel avenue for exploring psychology to enhance our understanding of generative AI models, thus boosting the research and development of human-AI collaboration and mitigating potential risks.

Details

NeurIPS Conference 2023 Conference Paper

Large Language Models Are Latent Variable Models: Explaining and Finding Good Demonstrations for In-Context Learning

Xinyi Wang
Wanrong Zhu
Michael Saxon
Mark Steyvers
William Yang Wang

In recent years, pre-trained large language models (LLMs) have demonstrated remarkable efficiency in achieving an inference-time few-shot learning capability known as in-context learning. However, existing literature has highlighted the sensitivity of this capability to the selection of few-shot demonstrations. Current understandings of the underlying mechanisms by which this capability arises from regular language model pretraining objectives remain disconnected from the real-world LLMs. This study aims to examine the in-context learning phenomenon through a Bayesian lens, viewing real-world LLMs as latent variable models. On this premise, we propose an algorithm to select optimal demonstrations from a set of annotated data with a small LM, and then directly generalize the selected demonstrations to larger LMs. We demonstrate significant improvement over baselines, averaged over eight GPT models on eight real-world text classification datasets. We also demonstrate the real-world usefulness of our algorithm on GSM8K, a math word problem dataset. Our empirical findings support our hypothesis that LLMs implicitly infer a latent variable containing task information.

PDF Details

ECAI Conference 2023 Conference Paper

Multiplicative Sparse Tensor Factorization for Multi-View Multi-Task Learning

Xinyi Wang
Lu Sun 0001
Canh Hao Nguyen
Hiroshi Mamitsuka

Multi-View Multi-Task Learning (MVMTL) aims to make predictions on dual-heterogeneous data. Such data contains features from multiple views, and multiple tasks in the data are related with each other through common views. Existing MVMTL methods usually face two major challenges: 1) to save the predictive information from full-order interactions between views efficiently. 2) to learn a parsimonious and highly interpretable model such that the target is related to the features through a subset of interactions. To deal with the challenges, we propose a novel MVMTL method based on multiplicative sparse tensor factorization. For 1), we represent full-order interactions between views as a tensor, that enables to capture the complex correlations in dual-heterogeneous data by a concise model. For 2), we decompose the interaction tensor into a product of two components: one being shared with all tasks and the other being specific to individual tasks. Moreover, tensor factorization is applied to control the model complexity and learn a consensus latent representation shared by multiple tasks. Theoretical analysis reveals the equivalence between our method and a family of models with a joint but more general form of regularizers. Experiments on both synthetic and real-world datasets prove its effectiveness.

Details

TMLR Journal 2023 Journal Article

Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks

Wenhu Chen
Xueguang Ma
Xinyi Wang
William W. Cohen

Recently, there has been significant progress in teaching language models to perform step-by-step reasoning to solve complex numerical reasoning tasks. Chain-of-thoughts prompting (CoT) is the state-of-art method for many of these tasks. CoT uses language models to produce text describing reasoning, and computation, and finally the answer to a question. Here we propose `Program of Thoughts' (PoT), which uses language models (mainly Codex) to generate text and programming language statements, and finally an answer. In PoT, the computation can be delegated to a program interpreter, which is used to execute the generated program, thus decoupling complex computation from reasoning and language understanding. We evaluate PoT on five math word problem datasets and three financial-QA datasets in both few-shot and zero-shot settings. We find that PoT has an average performance gain over CoT of around 12% across all datasets. By combining PoT with self-consistency decoding, we can achieve extremely strong performance on all the math datasets and financial datasets. All of our data and code will be released.

PDF Details

YNICL Journal 2023 Journal Article

Validation of deep learning techniques for quality augmentation in diffusion MRI for clinical studies

Santiago Aja-Fernández
Carmen Martín-Martín
Álvaro Planchuelo-Gómez
Abrar Faiyaz
Md Nasir Uddin
Giovanni Schifitto
Abhishek Tiwari
Saurabh J. Shigwan

The objective of this study is to evaluate the efficacy of deep learning (DL) techniques in improving the quality of diffusion MRI (dMRI) data in clinical applications. The study aims to determine whether the use of artificial intelligence (AI) methods in medical images may result in the loss of critical clinical information and/or the appearance of false information. To assess this, the focus was on the angular resolution of dMRI and a clinical trial was conducted on migraine, specifically between episodic and chronic migraine patients. The number of gradient directions had an impact on white matter analysis results, with statistically significant differences between groups being drastically reduced when using 21 gradient directions instead of the original 61. Fourteen teams from different institutions were tasked to use DL to enhance three diffusion metrics (FA, AD and MD) calculated from data acquired with 21 gradient directions and a b-value of 1000 s/mm2. The goal was to produce results that were comparable to those calculated from 61 gradient directions. The results were evaluated using both standard image quality metrics and Tract-Based Spatial Statistics (TBSS) to compare episodic and chronic migraine patients. The study results suggest that while most DL techniques improved the ability to detect statistical differences between groups, they also led to an increase in false positive. The results showed that there was a constant growth rate of false positives linearly proportional to the new true positives, which highlights the risk of generalization of AI-based tasks when assessing diverse clinical cohorts and training using data from a single group. The methods also showed divergent performance when replicating the original distribution of the data and some exhibited significant bias. In conclusion, extreme caution should be exercised when using AI methods for harmonization or synthesis in clinical studies when processing heterogeneous data in clinical studies, as important information may be altered, even when global metrics such as structural similarity or peak signal-to-noise ratio appear to suggest otherwise.

Details DOI

YNICL Journal 2022 Journal Article

Evaluating iron deposition in gray matter nuclei of patients with unilateral middle cerebral artery stenosis using quantitative susceptibility mapping

Huimin Mao
Weiqiang Dou
Kunjian Chen
Xinyu Wang
Xinyi Wang
Yu Guo
Chao Zhang

Iron mediated oxidative stress is involved in the process of brain injury after long-term ischemia. While increased iron deposition in the affected brain regions was observed in animal models of ischemic stroke, potential changes in the brain iron content in clinical patients with cerebral ischemia remain unclear. Quantitative susceptibility mapping (QSM), a non-invasive magnetic resonance imaging technique, can be used to evaluate iron content in the gray matter (GM) nuclei reliably. In this study, we aimed to quantitatively evaluate iron content changes in GM nuclei of patients with long-term unilateral middle cerebral artery (MCA) stenosis/occlusion-related cerebral ischemia using QSM. Forty-six unilateral MCA stenosis/occlusion patients and 38 age-, sex- and education-matched healthy controls underwent QSM. Clinical variables of history of hypertension, diabetes, hyperlipidemia, hyperhomocysteinemia, smoking, and drinking in all patients were evaluated. The iron-related susceptibility of GM nucleus subregions, including the bilateral caudate nucleus (CN), putamen (PU), globus pallidus (GP), thalamus, substantia nigra (SN), red nucleus, and dentate nucleus, was assessed. Susceptibility was compared between the bilateral GM nuclei in patients and controls. Receiver operating characteristic curve analysis was used to evaluate the efficacy of QSM susceptibility in distinguishing patients with unilateral MCA stenosis/occlusion from healthy controls. Multiple linear regression analysis was used to evaluate the relationship between ipsilateral susceptibility levels and clinical variables. Except for the CN, the susceptibility in most bilateral GM nucleus subregions was comparable in healthy controls, whereas for patients with unilateral MCA stenosis/occlusion, the ipsilateral PU, GP, and SN exhibited significantly higher susceptibility than the contralateral side (all P < 0.05). Compared with controls, susceptibility of the ipsilateral PU, GP, and SN and of contralateral PU in patients were significantly increased (all P < 0.05). The area under the curve (AUC) was greater for the ipsilateral PU than for the GP and SN (AUC = 0.773, 0.662 and 0.681; all P < 0.05). Multiple linear regression analysis showed that the increased susceptibility of the ipsilateral PU was significantly associated with hypertension, of the ipsilateral GP associated with smoking, and of the ipsilateral SN associated with diabetes (all P < 0.05). Our findings provide support for abnormal iron accumulation in the GM nuclei after chronic MCA stenosis/occlusion and its correlation with some cerebrovascular disease risk factors. Therefore, iron deposition in the GM nuclei, as measured by QSM, may be a potential biomarker for long-term cerebral ischemia.

Details DOI

JMLR Journal 2022 Journal Article

Innovations Autoencoder and its Application in One-class Anomalous Sequence Detection

Xinyi Wang
Lang Tong

An innovations sequence of a time series is a sequence of independent and identically distributed random variables with which the original time series has a causal representation. The innovation at a time is statistically independent of the history of the time series. As such, it represents the new information contained at present but not in the past. Because of its simple probability structure, the innovations sequence is the most efficient signature of the original. Unlike the principle or independent component representations, an innovations sequence preserves not only the complete statistical properties but also the temporal order of the original time series. An long-standing open problem is to find a computationally tractable way to extract an innovations sequence of non-Gaussian processes. This paper presents a deep learning approach, referred to as Innovations Autoencoder (IAE), that extracts innovations sequences using a causal convolutional neural network. An application of IAE to the one-class anomalous sequence detection problem with unknown anomaly and anomaly-free models is also presented. [abs] [ pdf ][ bib ] &copy JMLR 2022. ( edit, beta )

PDF Details

NeurIPS Conference 2021 Conference Paper

A Dataset for Answering Time-Sensitive Questions

Wenhu Chen
Xinyi Wang
William Yang Wang

Time is an important dimension in our physical world. Lots of facts can evolve with respect to time. For example, the U. S. President might change every four years. Therefore, it is important to consider the time dimension and empower the existing QA models to reason over time. However, the existing QA datasets contain rather few time-sensitive questions, hence not suitable for diagnosing or benchmarking the model's temporal reasoning capability. In order to promote research in this direction, we propose to construct a time-sensitive QA dataset. The dataset is constructed by 1) mining time-evolving facts from WikiData and aligning them to their corresponding Wikipedia page, 2) employing crowd workers to verify and calibrate these noisy facts, 3) generating question-answer pairs based on the annotated time-sensitive facts. Our dataset poses challenges in the aspect of both temporal understanding and temporal reasoning. We evaluate different SoTA long-document QA systems like BigBird and FiD on our dataset. The best-performing model FiD can only achieve 46\% accuracy, still far behind the human performance of 87\%. We demonstrate that these models are still lacking the ability to perform consistent temporal reasoning. Therefore, we believe that our dataset could serve as a benchmark to develop NLP models more sensitive to temporal shifts.

PDF Details

NeurIPS Conference 2021 Conference Paper

Counterfactual Maximum Likelihood Estimation for Training Deep Networks

Xinyi Wang
Wenhu Chen
Michael Saxon
William Yang Wang

Although deep learning models have driven state-of-the-art performance on a wide array of tasks, they are prone to spurious correlations that should not be learned as predictive clues. To mitigate this problem, we propose a causality-based training framework to reduce the spurious correlations caused by observed confounders. We give theoretical analysis on the underlying general Structural Causal Model (SCM) and propose to perform Maximum Likelihood Estimation (MLE) on the interventional distribution instead of the observational distribution, namely Counterfactual Maximum Likelihood Estimation (CMLE). As the interventional distribution, in general, is hidden from the observational data, we then derive two different upper bounds of the expected negative log-likelihood and propose two general algorithms, Implicit CMLE and Explicit CMLE, for causal predictions of deep learning models using observational data. We conduct experiments on both simulated data and two real-world tasks: Natural Language Inference (NLI) and Image Captioning. The results show that CMLE methods outperform the regular MLE method in terms of out-of-domain generalization performance and reducing spurious correlations, while maintaining comparable performance on the regular evaluations.

PDF Details