Author name cluster

Yiwei Wang

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

12 papers

2 author rows

AAAI Conference 2026 Conference Paper

OIDA-QA: A Multimodal Benchmark for Analyzing the Opioid Industry Documents Archive

Xuan Shen
Brian Wingenroth
Zichao Wang
Jason Kuen
Wanrong Zhu
Ruiyi Zhang
Yiwei Wang
Lichun Ma

The opioid crisis represents a significant moment in public health that reveals systemic shortcomings across regulatory systems, healthcare practices, corporate governance, and public policy. Analyzing how these interconnected systems simultaneously failed to protect public health requires innovative analytic approaches for exploring the vast amounts of data and documents disclosed in the UCSF-JHU Opioid Industry Documents Archive (OIDA). The complexity, multimodal nature, and specialized characteristics of these healthcare-related legal and corporate documents necessitate more advanced methods and models tailored to specific data types and detailed annotations, ensuring the precision and professionalism in the analysis. In this paper, we tackle this challenge by organizing the original dataset according to document attributes and constructing a benchmark with 400k training documents and 10k for testing. From each document, we extract rich multimodal information—including textual content, visual elements, and layout structures—to capture a comprehensive range of features. Using multiple AI models, we then generate a large-scale dataset comprising 360k training QA pairs and 10k testing QA pairs. Building on this foundation, we develop domain-specific multimodal Large Language Models (LLMs) and explore the impact of multimodal inputs on task performance. To further enhance response accuracy, we incorporate historical QA pairs as contextual grounding for answering current queries. Additionally, we incorporate page references within the answers and introduce an importance-based page classifier, further improving the precision and relevance of the information provided. Preliminary results indicate the improvements with our AI assistant in document information extraction and question-answering tasks.

PDF Details DOI

ICML Conference 2025 Conference Paper

Peripheral Memory for LLMs: Integration of Sequential Memory Banks with Adaptive Querying

Songlin Zhai
Yuan Meng
Yongrui Chen 0002
Yiwei Wang
Guilin Qi

Large Language Models (LLMs) have revolutionized various natural language processing tasks with their remarkable capabilities. However, challenges persist in effectively integrating new knowledge into LLMs without compromising their performance, particularly in the Large Language Models (LLMs) have revolutionized various natural language processing tasks with their remarkable capabilities. However, a challenge persists in effectively processing new information, particularly in the area of long-term knowledge updates without compromising model performance. To address this challenge, this paper introduces a novel memory augmentation framework that conceptualizes memory as a peripheral component (akin to physical RAM), with the LLM serving as the information processor (analogous to a CPU). Drawing inspiration from RAM architecture, we design memory as a sequence of memory banks, each modeled using Kolmogorov-Arnold Network (KAN) to ensure smooth state transitions. Memory read and write operations are dynamically controlled by query signals derived from the LLMs’ internal states, closely mimicking the interaction between a CPU and RAM. Furthermore, a dedicated memory bank is used to generate a mask value that indicates the relevance of the retrieved data, inspired by the sign bit in binary coding schemes. The retrieved memory feature is then integrated as a prefix to enhance the model prediction. Extensive experiments on knowledge-based model editing validate the effectiveness and efficiency of our peripheral memory.

Details

TMLR Journal 2025 Journal Article

UniTST: Effectively Modeling Inter-Series and Intra-Series Dependencies for Multivariate Time Series Forecasting

Juncheng Liu
Chenghao Liu
Gerald Woo
Yiwei Wang
Bryan Hooi
Caiming Xiong
Doyen Sahoo

Transformer-based models have emerged as powerful tools for multivariate time series forecasting (MTSF). However, existing Transformer models often fall short of capturing both intricate dependencies across variate and temporal dimensions in MTS data. Some recent models are proposed to separately capture variate and temporal dependencies through either two sequential or parallel attention mechanisms. However, these methods cannot directly and explicitly learn the intricate inter-series and intra-series dependencies. In this work, we first demonstrate that these dependencies are very important as they usually exist in real-world data. To directly model these dependencies, we propose a transformer-based model UniTST containing a unified attention mechanism on the flattened patch tokens. Additionally, we add a dispatcher module which reduces the complexity and makes the model feasible for a potentially large number of variates. Although our proposed model employs a simple architecture, it offers compelling performance as shown in our extensive experiments on several datasets for time series forecasting.

PDF Details

NeurIPS Conference 2024 Conference Paper

Fast Graph Sharpness-Aware Minimization for Enhancing and Accelerating Few-Shot Node Classification

Yihong Luo
Yuhan Chen
Siya Qiu
Yiwei Wang
Chen Zhang
Yan Zhou
Xiaochun Cao
Jing Tang

Graph Neural Networks (GNNs) have shown superior performance in node classification. However, GNNs perform poorly in the Few-Shot Node Classification (FSNC) task that requires robust generalization to make accurate predictions for unseen classes with limited labels. To tackle the challenge, we propose the integration of Sharpness-Aware Minimization (SAM)--a technique designed to enhance model generalization by finding a flat minimum of the loss landscape--into GNN training. The standard SAM approach, however, consists of two forward-backward steps in each training iteration, doubling the computational cost compared to the base optimizer (e. g. , Adam). To mitigate this drawback, we introduce a novel algorithm, Fast Graph Sharpness-Aware Minimization (FGSAM), that integrates the rapid training of Multi-Layer Perceptrons (MLPs) with the superior performance of GNNs. Specifically, we utilize GNNs for parameter perturbation while employing MLPs to minimize the perturbed loss so that we can find a flat minimum with good generalization more efficiently. Moreover, our method reutilizes the gradient from the perturbation phase to incorporate graph topology into the minimization process at almost zero additional cost. To further enhance training efficiency, we develop FGSAM+ that executes exact perturbations periodically. Extensive experiments demonstrate that our proposed algorithm outperforms the standard SAM with lower computational costs in FSNC tasks. In particular, our FGSAM+ as a SAM variant offers a faster optimization than the base optimizer in most cases. In addition to FSNC, our proposed methods also demonstrate competitive performance in the standard node classification task for heterophilic graphs, highlighting the broad applicability.

PDF Details DOI

AAAI Conference 2023 Conference Paper

AirFormer: Predicting Nationwide Air Quality in China with Transformers

Yuxuan Liang
Yutong Xia
Songyu Ke
Yiwei Wang
Qingsong Wen
Junbo Zhang
Yu Zheng
Roger Zimmermann

Air pollution is a crucial issue affecting human health and livelihoods, as well as one of the barriers to economic growth. Forecasting air quality has become an increasingly important endeavor with significant social impacts, especially in emerging countries. In this paper, we present a novel Transformer termed AirFormer to predict nationwide air quality in China, with an unprecedented fine spatial granularity covering thousands of locations. AirFormer decouples the learning process into two stages: 1) a bottom-up deterministic stage that contains two new types of self-attention mechanisms to efficiently learn spatio-temporal representations; 2) a top-down stochastic stage with latent variables to capture the intrinsic uncertainty of air quality data. We evaluate AirFormer with 4-year data from 1,085 stations in Chinese Mainland. Compared to prior models, AirFormer reduces prediction errors by 5%∼8% on 72-hour future predictions. Our source code is available at https://github.com/yoshall/airformer.

PDF Details DOI

JBHI Journal 2023 Journal Article

Laparoscopic Image-Based Critical Action Recognition and Anticipation With Explainable Features

Jie Zhang
Song Zhou
Yiwei Wang
Shenchao Shi
Chidan Wan
Huan Zhao
Xiong Cai
Han Ding

Surgical workflow analysis integrates perception, comprehension, and prediction of the surgical workflow, which helps real-time surgical support systems provide proper guidance and assistance for surgeons. This article promotes the idea of critical actions, which refer to the essential surgical actions that progress towards the fulfillment of the operation. Fine-grained workflow analysis involves recognizing current critical actions and previewing the moving tendency of instruments in the early stage of critical actions. Aiming at this, we propose a framework that incorporates operational experience to improve the robustness and interpretability of action recognition in in-vivo situations. High-dimensional images are mapped into an experience-based explainable feature space with low dimensions to achieve critical action recognition through a hierarchical classification structure. To forecast the instrument's motion tendency, we model the motion primitives in the polar coordinate system (PCS) to represent patterns of complex trajectories. Given the laparoscopy variance, the adaptive pattern recognition (APR) method, which adapts to uncertain trajectories by modifying model parameters, is designed to improve prediction accuracy. The in-vivo dataset validations show that our framework fulfilled the surgical awareness tasks with exceptional accuracy and real-time performance.

Details DOI

NeurIPS Conference 2023 Conference Paper

LargeST: A Benchmark Dataset for Large-Scale Traffic Forecasting

Xu Liu
Yutong Xia
Yuxuan Liang
Junfeng Hu
Yiwei Wang
Lei Bai
Chao Huang
Zhenguang Liu

Road traffic forecasting plays a critical role in smart city initiatives and has experienced significant advancements thanks to the power of deep learning in capturing non-linear patterns of traffic data. However, the promising results achieved on current public datasets may not be applicable to practical scenarios due to limitations within these datasets. First, the limited sizes of them may not reflect the real-world scale of traffic networks. Second, the temporal coverage of these datasets is typically short, posing hurdles in studying long-term patterns and acquiring sufficient samples for training deep models. Third, these datasets often lack adequate metadata for sensors, which compromises the reliability and interpretability of the data. To mitigate these limitations, we introduce the LargeST benchmark dataset. It encompasses a total number of 8, 600 sensors in California with a 5-year time coverage and includes comprehensive metadata. Using LargeST, we perform in-depth data analysis to extract data insights, benchmark well-known baselines in terms of their performance and efficiency, and identify challenges as well as opportunities for future research. We release the datasets and baseline implementations at: https: //github. com/liuxu77/LargeST.

PDF Details

EAAI Journal 2022 Journal Article

Life prediction of underground structure by sulfate corrosion using Harris hawks optimizing genetic programming

Yuan Xie
Wei Gao
Yiwei Wang
Xin Chen
Shuangshuang Ge
Sen Wang

A corrosive sulfate environment can cause strong deterioration and destruction of reinforced concrete (RC) underground structures and seriously reduce their service life. Thus, it is very important to predict the service life of RC underground structures in corrosive sulfate environments. However, the service life of underground structures is affected by numerous complicated engineering and environmental factors and cannot be determined by traditional theoretical and experimental investigations. Therefore, to solve this problem, a new data-driven method based on Harris hawks optimizing genetic programming (HHO-GP) is proposed. In this new method, to improve the traditional genetic programming (GP), a new global optimization algorithm called Harris hawks optimization (HHO) is adopted to optimize its main controlling parameters. Based on 25 groups of real engineering data, the life prediction model of underground structures in corrosive sulfate environments with 12 main engineering and environmental influence factors is established by the HHO-GP method. The results show that the average relative training error (5. 5%) and predicting error (6. 3%) of the new prediction model are small. Therefore, the proposed HHO-GP method can construct a suitable life prediction model based on only real engineering data, regardless of how many complicated influencing factors are considered. Moreover, our data-driven life prediction model is described by one explicit polynomial function based on 12 influencing factors. Thus, it can be applied in real engineering simply and easily. Finally, the influence of the main controlling parameters of the HHO-GP on its accuracy and efficiency is analyzed. The results reveal that considering the computing accuracy and efficiency and the model completeness, the small population size and maximum iterations of HHO are suitable, whose recommended values are all 15. The population size and maximum number of iterations of GP have little influence on the prediction accuracy. Their recommended values all can be 50.

Details DOI

NeurIPS Conference 2021 Conference Paper

Adaptive Data Augmentation on Temporal Graphs

Yiwei Wang
Yujun Cai
Yuxuan Liang
Henghui Ding
Changhu Wang
Siddharth Bhatia
Bryan Hooi

Temporal Graph Networks (TGNs) are powerful on modeling temporal graph data based on their increased complexity. Higher complexity carries with it a higher risk of overfitting, which makes TGNs capture random noise instead of essential semantic information. To address this issue, our idea is to transform the temporal graphs using data augmentation (DA) with adaptive magnitudes, so as to effectively augment the input features and preserve the essential semantic information. Based on this idea, we present the MeTA (Memory Tower Augmentation) module: a multi-level module that processes the augmented graphs of different magnitudes on separate levels, and performs message passing across levels to provide adaptively augmented inputs for every prediction. MeTA can be flexibly applied to the training of popular TGNs to improve their effectiveness without increasing their time complexity. To complement MeTA, we propose three DA strategies to realistically model noise by modifying both the temporal and topological features. Empirical results on standard datasets show that MeTA yields significant gains for the popular TGN models on edge prediction and node classification in an efficient manner.

PDF Details

ICRA Conference 2021 Conference Paper

Design of a 3-DOF Coupled Tendon-Driven Waist Joint

Yiwei Wang
Wenyang Li
Shunta Togo
Hiroshi Yokoi
Yinlai Jiang

This paper proposes a coupled tendon-driven waist joint for humanoid robots. The waist joint was designed as a 3 degrees of freedom (DOF) structure to simulate the motion of a human waist. The power transmission was designed by adopting a 3-motor 3-DOF (3M3D) coupled tendon-driven mechanism, so that the torque on the joints was multiplied. We derived the torque transmission formula and the rotation angle formula of the 3M3D tendon-driven structures and designed the waist joint by adopting an appropriate structure according to their features. To evaluate the accuracy and load capacity of the waist joint, we performed a rotational accuracy experiment and a maximum torque experiment. The experiment results showed that the maximum error of joint rotation was below 1°, and the maximum torque of the pitch, roll, and yaw rotations were 87[Nm], 53[Nm], and 22. 2[Nm], respectively.

Details

NeurIPS Conference 2021 Conference Paper

EIGNN: Efficient Infinite-Depth Graph Neural Networks

Juncheng Liu
Kenji Kawaguchi
Bryan Hooi
Yiwei Wang
Xiaokui Xiao

Graph neural networks (GNNs) are widely used for modelling graph-structured data in numerous applications. However, with their inherently finite aggregation layers, existing GNN models may not be able to effectively capture long-range dependencies in the underlying graphs. Motivated by this limitation, we propose a GNN model with infinite depth, which we call Efficient Infinite-Depth Graph Neural Networks (EIGNN), to efficiently capture very long-range dependencies. We theoretically derive a closed-form solution of EIGNN which makes training an infinite-depth GNN model tractable. We then further show that we can achieve more efficient computation for training EIGNN by using eigendecomposition. The empirical results of comprehensive experiments on synthetic and real-world datasets show that EIGNN has a better ability to capture long-range dependencies than recent baselines, and consistently achieves state-of-the-art performance. Furthermore, we show that our model is also more robust against both noise and adversarial perturbations on node features.

PDF Details

IJCAI Conference 2021 Conference Paper

Modeling Trajectories with Neural Ordinary Differential Equations

Yuxuan Liang
Kun Ouyang
Hanshu Yan
Yiwei Wang
Zekun Tong
Roger Zimmermann

Recent advances in location-acquisition techniques have generated massive spatial trajectory data. Recurrent Neural Networks (RNNs) are modern tools for modeling such trajectory data. After revisiting RNN-based methods for trajectory modeling, we expose two common critical drawbacks in the existing uses. First, RNNs are discrete-time models that only update the hidden states upon the arrival of new observations, which makes them an awkward fit for learning real-world trajectories with continuous-time dynamics. Second, real-world trajectories are never perfectly accurate due to unexpected sensor noise. Most RNN-based approaches are deterministic and thereby vulnerable to such noise. To tackle these challenges, we devise a novel method entitled TrajODE for more natural modeling of trajectories. It combines the continuous-time characteristic of Neural Ordinary Differential Equations (ODE) with the robustness of stochastic latent spaces. Extensive experiments on the task of trajectory classification demonstrate the superiority of our framework against the RNN counterparts.

PDF Details DOI