Siqi Liu

JBHI Journal 2026 Journal Article

CAM-Interacted Vision GNN for Multi-Label Medical Images

• Jingchao Wang
• Baoyao Yang
• Siqi Liu
• Xiaoqi Zheng
• Wenbin Yao
• Junxiang Chen

Vision Graph Neural Network (ViG) is designed to recognize different objects through graph-level processing. However, ViG constructs graphs with appearance-level neighbors and neglects the category semantic. The oversight results in the unintentional connection of patches that belong to different objects, thus affecting the distinctiveness of categories in multi-label medical image learning. Since the pixel-level annotations for images are not easily available, category-aware graphs can not be directly built. To solve this problem, we consider localizing category-specific regions using Class Activation Maps (CAMs), an effective way to highlight regions belonging to each category without requiring manual annotations. Specifically, we propose a CAM-interacted Vision GNN (CiV-GNN), in which category-aware graphs are formed to perform intra-category graph processing. CIV-GNN includes a Class-activated Patch Division (CAPD) module, which introduces CAMs as guidance for category-aware graph building. Furthermore, we develop a Multi-graph Interactive Processing (MIP) module to model the relations between category-aware graphs, promoting inter-category interaction learning. Experimental results show that CiV-GNN performs well in surgical tool localization and multi-label medical image classification. Specifically, for m2cai16-localization, CiV-GNN exhibits a 1. 43% and 7. 02% improvement in mAP50 and mAP50-95, respectively, compared to YOLOv8.

ICML Conference 2025 Conference Paper

From Black Boxes to Transparent Minds: Evaluating and Enhancing the Theory of Mind in Multimodal Large Language Models

• Xinyang Li
• Siqi Liu
• Bochao Zou
• Jiansheng Chen
• Huimin Ma 0001

As large language models evolve, there is growing anticipation that they will emulate human-like Theory of Mind (ToM) to assist with routine tasks. However, existing methods for evaluating machine ToM focus primarily on unimodal models and largely treat these models as black boxes, lacking an interpretative exploration of their internal mechanisms. In response, this study adopts an approach based on internal mechanisms to provide an interpretability-driven assessment of ToM in multimodal large language models (MLLMs). Specifically, we first construct a multimodal ToM test dataset, GridToM, which incorporates diverse belief testing tasks and perceptual information from multiple perspectives. Next, our analysis shows that attention heads in multimodal large models can distinguish cognitive information across perspectives, providing evidence of ToM capabilities. Furthermore, we present a lightweight, training-free approach that significantly enhances the model’s exhibited ToM by adjusting in the direction of the attention head.

AAAI Conference 2025 Conference Paper

Integrating Co-Training with Edge Discrimination to Enhance Graph Neural Networks Under Heterophily

• Siqi Liu
• Dongxiao He
• Zhizhi Yu
• Di Jin
• Zhiyong Feng
• Weixiong Zhang

Graph Neural Networks (GNNs) have recently achieved significant success in several graph-related tasks. However, traditional GNNs and their variants are constantly limited by the implicit homophily, assuming neighboring nodes belong to the same class. This results in weak performance on heterophilic graphs where most nodes are linked to neighbors of different classes. Despite the numerous attempts to adequately deal with heterophily, most methods still use the uniform propagation aggregation mechanism. In this paper, we argue that identifying neighbors with different class labels and exploiting them individually is crucial for heterophilic GNNs. We then propose a simple and efficient novel co-training approach, EG-GCN, which uses group aggregation to handle homophilic and heterophilic neighbors separately. In EG-GCN, we first use an edge discriminator to classify edges and split the neighborhood of every node into two parts. We then apply group graph convolution to the divided neighborhoods to obtain node representations. During training, we continuously optimize the edge discriminator to improve neighborhood partition and use the node classification results to identify highly confident unlabeled nodes to expand the edge training set. This co-training strategy enables both components to enhance each other mutually. Extensive experiments demonstrate that EG-GCN significantly outperforms the state-of-the-art approaches.

TMLR Journal 2024 Journal Article

DynaConF: Dynamic Forecasting of Non-Stationary Time Series

• Siqi Liu
• Andreas Lehrmann

Deep learning has shown impressive results in a variety of time series forecasting tasks, where modeling the conditional distribution of the future given the past is the essence. However, when this conditional distribution is non-stationary, it poses challenges for these models to learn consistently and to predict accurately. In this work, we propose a new method to model non-stationary conditional distributions over time by clearly decoupling stationary conditional distribution modeling from non-stationary dynamics modeling. Our method is based on a Bayesian dynamic model that can adapt to conditional distribution changes and a deep conditional distribution model that handles multivariate time series using a factorized output space. Our experimental results on synthetic and real-world datasets show that our model can adapt to non-stationary time series better than state-of-the-art deep learning solutions.

AAMAS Conference 2024 Conference Paper

Neural Population Learning beyond Symmetric Zero-Sum Games

• Siqi Liu
• Luke Marris
• Marc Lanctot
• Georgios Piliouras
• Joel Z. Leibo
• Nicolas Heess

We study computationally efficient methods for finding equilibria in n-player general-sum games, specifically ones that afford complex visuomotor skills. We show how existing methods would struggle in this setting, either computationally or in theory. We then introduce NeuPL-JPSRO, a neural population learning algorithm that benefits from transfer learning of skills and converges to a Coarse Correlated Equilibrium (CCE) of the game. We show empirical convergence in a suite of OpenSpiel games, validated rigorously by exact game solvers. We then deploy NeuPL-JPSRO to complex domains, where our approach enables adaptive coordination in a MuJoCo control domain and skill transfer in capture-the-flag. Our work shows that equilibrium convergent population learning can be implemented at scale and in generality, paving the way towards solving real-world games between heterogeneous players with mixed motives.

AAAI Conference 2024 Conference Paper

Primitive-Based 3D Human-Object Interaction Modelling and Programming

• Siqi Liu
• Yong-Lu Li
• Zhou Fang
• Xinpeng Liu
• Yang You
• Cewu Lu

Embedding Human and Articulated Object Interaction (HAOI) in 3D is an important direction for a deeper human activity understanding. Different from previous works that use parametric and CAD models to represent humans and objects, in this work, we propose a novel 3D geometric primitive-based language to encode both humans and objects. Given our new paradigm, humans and objects are all compositions of primitives instead of heterogeneous entities. Thus, mutual information learning may be achieved between the limited 3D data of humans and different object categories. Moreover, considering the simplicity of the expression and the richness of the information it contains, we choose the superquadric as the primitive representation. To explore an effective embedding of HAOI for the machine, we build a new benchmark on 3D HAOI consisting of primitives together with their images and propose a task requiring machines to recover 3D HAOI using primitives from images. Moreover, we propose a baseline of single-view 3D reconstruction on HAOI. We believe this primitive-based 3D HAOI representation would pave the way for 3D HAOI studies. Our code and data are available at https://mvig-rhos.com/p3haoi.

YNIMG Journal 2024 Journal Article

Same allocation proposed by an individual or a group elicits distinct responses: Evidence from event-related potentials and neural oscillation

• Peiqi Chen
• Siqi Liu
• Yinling Zhang
• Shaozheng Qin
• Xiaoqin Mai

JBHI Journal 2023 Journal Article

Automatic Calcification Morphology and Distribution Classification for Breast Mammograms With Multi-Task Graph Convolutional Neural Network

• Hao Du
• Melissa Min-Szu Yao
• Siqi Liu
• Liangyu Chen
• Wing P. Chan
• Mengling Feng

The morphology and distribution of microcalcifications are the most important descriptors for radiologists to diagnose breast cancer based on mammograms. However, it is very challenging and time-consuming for radiologists to characterize these descriptors manually, and there also lacks of effective and automatic solutions for this problem. We observed that the distribution and morphology descriptors are determined by the radiologists based on the spatial and visual relationships among calcifications. Thus, we hypothesize that this information can be effectively modelled by learning a relationship-aware representation using graph convolutional networks (GCNs). In this study, we propose a multi-task deep GCN method for automatic characterization of both the morphology and distribution of microcalcifications in mammograms. Our proposed method transforms morphology and distribution characterization into node and graph classification problem and learns the representations concurrently. We trained and validated the proposed method in an in-house dataset and public DDSM dataset with 195 and 583 cases, respectively. The proposed method reaches good and stable results with distribution AUC at 0. 812 $\pm$ 0. 043 and 0. 873 $\pm$ 0. 019, morphology AUC at 0. 663 $\pm$ 0. 016 and 0. 700 $\pm$ 0. 044 for both in-house and public datasets. In both datasets, our proposed method demonstrates statistically significant improvements compared to the baseline models. The performance improvements brought by our proposed multi-task mechanism can be attributed to the association between the distribution and morphology of calcifications in mammograms, which is interpretable using graphical visualizations and consistent with the definitions of descriptors in the standard BI-RADS guideline. In short, we explore, for the first time, the application of GCNs in microcalcification characterization that suggests the potential of using graph learning for more robust understanding of medical images.

IJCAI Conference 2022 Conference Paper

Learning Sparse Interpretable Features For NAS Scoring From Liver Biopsy Images

• Chong Yin
• Siqi Liu
• Vincent Wai-Sun Wong
• Pong C Yuen

Liver biopsy images play a key role in the diagnosis of global non-alcoholic fatty liver disease (NAFLD). The NAFLD activity score (NAS) on liver biopsy images grades the amount of histological findings that reflect the progression of NAFLD. However, liver biopsy image analysis remains a challenging task due to its complex tissue structures and sparse distribution of histological findings. In this paper, we propose a sparse interpretable feature learning method (SparseX) to efficiently estimate NAS scores. First, we introduce an interpretable spatial sampling strategy based on histological features to effectively select informative tissue regions containing tissue alterations. Then, SparseX formulates the feature learning as a low-rank decomposition problem. Non-negative matrix factorization (NMF)-based attributes learning is embedded into a deep network to compress and select sparse features for a small portion of tissue alterations contributing to diagnosis. Experiments conducted on the internal Liver-NAS and public SteatosisRaw datasets show the effectiveness of the proposed method in terms of classification performance and interpretability. regions containing tissue alterations. Then, SparseX formulates the feature learning as a low-rank decomposition problem. Non-negative matrix factorization (NMF)-based attributes learning is embedded into a deep network to compress and select sparse features for a small portion of tissue alterations contributing to diagnosis. Experiments conducted on the internal Liver-NAS and public SteatosisRaw datasets show the effectiveness of the proposed method in terms of classification performance and interpretability.

NeurIPS Conference 2022 Conference Paper

Turbocharging Solution Concepts: Solving NEs, CEs and CCEs with Neural Equilibrium Solvers

• Luke Marris
• Ian Gemp
• Thomas Anthony
• Andrea Tacchetti
• Siqi Liu
• Karl Tuyls

Solution concepts such as Nash Equilibria, Correlated Equilibria, and Coarse Correlated Equilibria are useful components for many multiagent machine learning algorithms. Unfortunately, solving a normal-form game could take prohibitive or non-deterministic time to converge, and could fail. We introduce the Neural Equilibrium Solver which utilizes a special equivariant neural network architecture to approximately solve the space of all games of fixed shape, buying speed and determinism. We define a flexible equilibrium selection framework, that is capable of uniquely selecting an equilibrium that minimizes relative entropy, or maximizes welfare. The network is trained without needing to generate any supervised training data. We show remarkable zero-shot generalization to larger games. We argue that such a network is a powerful component for many possible multiagent algorithms.

ICML Conference 2021 Conference Paper

Event Outlier Detection in Continuous Time

• Siqi Liu
• Milos Hauskrecht

Continuous-time event sequences represent discrete events occurring in continuous time. Such sequences arise frequently in real-life. Usually we expect the sequences to follow some regular pattern over time. However, sometimes these patterns may be interrupted by unexpected absence or occurrences of events. Identification of these unexpected cases can be very important as they may point to abnormal situations that need human attention. In this work, we study and develop methods for detecting outliers in continuous-time event sequences, including unexpected absence and unexpected occurrences of events. Since the patterns that event sequences tend to follow may change in different contexts, we develop outlier detection methods based on point processes that can take context information into account. Our methods are based on Bayesian decision theory and hypothesis testing with theoretical guarantees. To test the performance of the methods, we conduct experiments on both synthetic data and real-world clinical data and show the effectiveness of the proposed methods.

YNIMG Journal 2021 Journal Article

Human spatial navigation: Neural representations of spatial scales and reference frames obtained from an ALE meta-analysis

• Jinhui Li
• Ruibin Zhang
• Siqi Liu
• Qunjun Liang
• Senning Zheng
• Xianyou He
• Ruiwang Huang

AAMAS Conference 2021 Conference Paper

Pick Your Battles: Interaction Graphs as Population-Level Objectives for Strategic Diversity

• Marta Garnelo
• Wojciech Marian Czarnecki
• Siqi Liu
• Dhruva Tirumala
• Junhyuk Oh
• Gauthier Gidel
• Hado van Hasselt
• David Balduzzi

Strategic diversity is often essential in games: in multi-player games, for example, evaluating a player against a diverse set of strategies will yield a more accurate estimate of its performance. Furthermore, in games with non-transitivities diversity allows a player to cover several winning strategies. However, despite the significance of strategic diversity, training agents that exhibit diverse behaviour remains a challenge. In this paper we study how to construct diverse populations of agents by carefully structuring how individuals within a population interact. Our approach is based on interaction graphs, which control the flow of information between agents during training and can encourage agents to specialise on different strategies, leading to improved overall performance. We provide evidence for the importance of diversity in multi-agent training and analyse the effect of applying different interaction graphs on the training trajectories, diversity and performance of populations in a range of games.

AAAI Conference 2020 Conference Paper

DATA-GRU: Dual-Attention Time-Aware Gated Recurrent Unit for Irregular Multivariate Time Series

• Qingxiong Tan
• Mang Ye
• Baoyao Yang
• Siqi Liu
• Andy Jinhua Ma
• Terry Cheuk-Fung Yip
• Grace Lai-Hung Wong
• PongChi Yuen

Due to the discrepancy of diseases and symptoms, patients usually visit hospitals irregularly and different physiological variables are examined at each visit, producing large amounts of irregular multivariate time series (IMTS) data with missing values and varying intervals. Existing methods process IMTS into regular data so that standard machine learning models can be employed. However, time intervals are usually determined by the status of patients, while missing values are caused by changes in symptoms. Therefore, we propose a novel end-to-end Dual-Attention Time-Aware Gated Recurrent Unit (DATA-GRU) for IMTS to predict the mortality risk of patients. In particular, DATA-GRU is able to: 1) preserve the informative varying intervals by introducing a timeaware structure to directly adjust the influence of the previous status in coordination with the elapsed time, and 2) tackle missing values by proposing a novel dual-attention structure to jointly consider data-quality and medical-knowledge. A novel unreliability-aware attention mechanism is designed to handle the diversity in the reliability of different data, while a new symptom-aware attention mechanism is proposed to extract medical reasons from original clinical records. Extensive experimental results on two real-world datasets demonstrate that DATA-GRU can significantly outperform state-of-the-art methods and provide meaningful clinical interpretation.

NeurIPS Conference 2019 Conference Paper

Nonparametric Regressive Point Processes Based on Conditional Gaussian Processes

• Siqi Liu
• Milos Hauskrecht

Real-world event sequences consist of complex mixtures of different types of events occurring in time. An event may depend on past events of the same type, as well as, the other types. Point processes define a general class of models for event sequences. ``Regressive point processes'' refer to point processes that directly model the dependency between an event and any past event, an example of which is a Hawkes process. In this work, we propose and develop a new nonparametric regressive point process model based on Gaussian processes. We show that our model can represent better many commonly observed real-world event sequences and capture the dependencies between events that are difficult to model using existing nonparametric Hawkes process variants. We demonstrate the improved predictive performance of our model against state-of-the-art baselines on multiple synthetic and real-world datasets.

AAMAS Conference 2019 Conference Paper

Observational Learning by Reinforcement Learning

• Diana Borsa
• Nicolas Heess
• Bilal Piot
• Siqi Liu
• Leonard Hasenclever
• Remi Munos
• Olivier Pietquin

Observational learning is a type of learning that occurs as a function of observing, retaining and possibly imitating the behaviour of another agent. It is a core mechanism appearing in various instances of social learning and has been found to be employed in several intelligent species, including humans. In this paper, we investigate to what extent the explicit modelling of other agents is necessary to achieve observational learning through machine learning. Especially, we argue that observational learning can emerge from pure Reinforcement Learning (RL), potentially coupled with memory. Through simple scenarios, we demonstrate that an RL agent can leverage the information provided by the observations of an other agent performing a task in a shared environment. The other agent is only observed through the e�ect of its actions on the environment and never explicitly modeled. Two key aspects are borrowed from observational learning: i) the observer behaviour needs to change as a result of viewing a ’teacher’ (another agent) and ii) the observer needs to be motivated somehow to engage in making use of the other agent’s behaviour. The later is naturally modeled by RL, by correlating the learning agent’s reward with the teacher agent’s behaviour.

AAMAS Conference 2019 Conference Paper

The Body is Not a Given: Joint Agent Policy Learning and Morphology Evolution

• Dylan Banarse
• Yoram Bachrach
• Siqi Liu
• Guy Lever
• Nicolas Heess
• Chrisantha Fernando
• Pushmeet Kohli
• Thore Graepel

Reinforcement learning (RL) has proven to be a powerful paradigm for deriving complex behaviors from simple reward signals in a wide range of environments. When applying RL to continuous control agents in simulated physics environments, the body is usually considered to be part of the environment. However, during evolution the physical body of biological organisms and their controlling brains are co-evolved, thus exploring a much larger space of actuator/controller configurations. Put differently, the intelligence does not reside only in the agent’s mind, but also in the design of their body. We propose a method for uncovering strong agents, consisting of a good combination of a body and policy, based on combining RL with an evolutionary procedure. Given the resulting agent, we also propose an approach for identifying the body changes that contributed the most to the agent performance. We use the Shapley value from cooperative game theory to find the fair contribution of individual components, taking into account synergies between components. We evaluate our methods in an environment similar to the the recently proposed Robo-Sumo task, where agents in a software physics simulator compete in tipping over their opponent or pushing them out of the arena. Our results show that the proposed methods are indeed capable of generating strong agents, significantly outperforming baselines that focus on optimizing the agent policy alone. A video is available at: https: //youtu. be/CHlecRim9PI

AIIM Journal 2018 Journal Article

Change-point detection method for clinical decision support system rule monitoring

• Siqi Liu
• Adam Wright
• Milos Hauskrecht