Arrow Research search

Author name cluster

Lin Liu

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

38 papers
1 author row

Possible papers

38

AAAI Conference 2026 Conference Paper

Accelerating Controllable Generation via Hybrid-grained Cache

  • Lin Liu
  • Huixia Ben
  • Shuo Wang
  • Jinda Lu
  • Junxiang Qiu
  • Shengeng Tang
  • Yanbin Hao

Controllable generative models have been widely used to improve the realism of synthetic visual content. However, such models must handle control conditions and content generation computational requirements, resulting in generally low generation efficiency. To address this issue, we propose a Hybrid-Grained Cache (HGC) approach that reduces computational overhead by adopting cache strategies with different granularities at different computational stages. Specifically, (1) we use a coarse-grained cache (block-level) based on feature reuse to dynamically bypass redundant computations in encoder-decoder blocks between each step of model reasoning. (2) We design a fine-grained cache (prompt-level) that acts within a module, where the fine-grained cache reuses cross-attention maps within consecutive reasoning steps and extends them to the corresponding module computations of adjacent steps. These caches of different granularities can be seamlessly integrated into each computational link of the controllable generation process. We verify the effectiveness of HGC on four benchmark datasets, especially its advantages in balancing generation efficiency and visual quality. For example, on the COCO-Stuff segmentation benchmark, our HGC significantly reduces the computational cost (MACs) by 63% (from 18.22T → 6.70T↓), while keeping the loss of semantic fidelity (quantized performance degradation) within 1.5%.

PDF Details DOI

JBHI Journal 2026 Journal Article

Camera-Based Dual-Wavelength Defocused Speckle Imaging for Multi-Point Seismocardiographic Motion Measurement

  • Lin Liu
  • Qiang Wei
  • Haozhe Li
  • Caifeng Shan
  • Wenjin Wang

Continuous monitoring of cardiac activity is crucial for detecting anomalies such as heart failure and coronary artery disease, and it can alleviate the burden of cardiovascular disease on healthcare systems. This study introduces a novel concept for contactless monitoring of multi-point cardiac motion using dual-wavelength defocused speckle imaging (DW-DSI). A prototype system was developed to measure multi-point seismocardiography (MP-SCG) signals from the atrial and ventricular regions. In addition, blood pressure (BP) monitoring was demonstrated as a proof of concept using the time delay between atrial and ventricular motion signals. An experiment involving 19 subjects with ice water stimulation protocol demonstrated that the performance of BP estimation using time delay features of MP-SCG is comparable to BP estimated from ECG-PPG derived pulse arrival time. The results showed that the best performance was achieved using the correlation features extracted from MP-SCG, such as time delay information and heart rate, in combination with an artificial neural network model. The mean absolute error for systolic/diastolic/mean BP are 6. 954 mmHg, 5. 368 mmHg and 5. 415 mmHg, with Pearson correlation coefficient of 0. 639, 0. 559, and 0. 517. This demonstrates the potential of the camera-based DW-DSI system for measuring MP-SCG and the feasibility towards continuous BP monitoring.

Details DOI

AAAI Conference 2026 Conference Paper

Learning Fair Graph Representations via Probability of Necessity and Sufficiency

  • Chuxun Liu
  • Qingfeng Chen
  • Debo Cheng
  • Jiangzhang Gan
  • Jiuyong Li
  • Lin Liu

Graph Neural Networks (GNNs) excel at modeling graph data but often amplify biases tied to sensitive attributes like gender and race. Existing causality-based methods use isolated interventions on graph topology or features but struggle to produce representations that balance predictive power with fairness. This leads to two issues: (1) weak predictive power, where representations miss critical task-relevant features, and (2) bias amplification, where representations encode sensitive attributes, causing unfair outcomes. To address these issues, we introduce the Probability of Necessity and Sufficiency (PNS), where necessity ensures representations capture only essential features for predictions, and sufficiency guarantees these features are adequate without relying on sensitive attributes. We propose FairSNR, a fairness-aware graph representation learning framework that introduces constraints based on the PNS. This leverages PNS to guide the learning of fair representations from graph data. In particular, FairSNR employs an encoder to learn node representations with high PNS for downstream tasks. To compute and optimize PNS, FairSNR introduces an intervenor to generate the most challenging counterfactual interventions on the representations, thereby enhancing the model’s causal stability even under worst-case scenarios. Further, a discriminator is trained to detect and mitigate sensitive information leakage in the learned representations, effectively disentangling sensitive biases from task-relevant features. Experiments on real-world graph datasets demonstrate that FairSNR outperforms existing state-of-the-art (SOTA) methods in both fairness and utility.

PDF Details DOI

AAAI Conference 2026 Conference Paper

O-DisCo-Edit: Object Distortion Control for Unified Realistic Video Editing

  • Yuqing Chen
  • Junjie Wang
  • Lin Liu
  • Ruihang Chu
  • Xiaopeng Zhang
  • Qi Tian
  • Yujiu Yang

Diffusion models have recently advanced video editing, yet controllable editing remains challenging due to the need for precise manipulation of diverse object properties. Current methods require different control signal for diverse editing tasks, which complicates model design and demands significant training resources. To address this, we propose O-DisCo-Edit, a unified framework that incorporates a novel object distortion control (O-DisCo). This signal, based on random and adaptive noise, flexibly encapsulates a wide range of editing cues within a single representation. Paired with a “copy-form” preservation module for preserving non-edited regions, O-DisCo-Edit enables efficient, high-fidelity editing through an effective training paradigm. Extensive experiments and comprehensive human evaluations consistently demonstrate that O-DisCo-Edit surpasses both specialized and multitask state-of-the-art methods across various video editing tasks.

PDF Details DOI

TIST Journal 2026 Journal Article

Towards Fair Graph Representation Learning by Overcoming Social Homophily

  • Guixian Zhang
  • Guan Yuan
  • Debo Cheng
  • Lin Liu
  • Jiuyong Li
  • Shichao Zhang

With the widespread use of Graph Neural Networks (GNNs) for representation learning from network data, the fairness of GNN models has raised great attention lately. Fair GNNs aim to ensure that node representations can be accurately classified, but not easily associated with a specific group. Existing advanced approaches essentially enhance the generalisation of node representation in combination with data augmentation strategy and do not directly impose constraints on the fairness of GNNs. In this work, we identify that a fundamental reason for the unfairness of GNNs is the phenomenon of social homophily, i.e., users in the same group are more inclined to congregate. The message-passing mechanism of GNNs can cause users in the same group to have similar representations due to social homophily, leading model predictions to establish spurious correlations with sensitive attributes. Inspired by this reason, we propose a method called Equity-Aware GNN (EAGNN) towards fair graph representation learning. Specifically, to ensure that model predictions are independent of sensitive attributes while maintaining prediction performance, we introduce constraints for fair representation learning based on three principles: sufficiency, independence and separation. We theoretically demonstrate that our EAGNN method can effectively achieve group fairness. Extensive experiments on three datasets with varying levels of social homophily illustrate that our EAGNN method achieves the state-of-the-art performance across two fairness metrics and offers competitive effectiveness.

Details DOI

JBHI Journal 2025 Journal Article

Cardiac 3D Motion Reconstruction Using Dual-Camera Defocused Speckle Imaging With Multi-Scale Amplification

  • Lin Liu
  • Haimiao Mo
  • Guowei Wang
  • Yingen Zhu
  • Yifei Da
  • Caifeng Shan
  • Wenjin Wang

Cardiovascular diseases are one of the leading causes of death worldwide. Accurately capturing and analyzing the multidimensional dynamics of cardiac motion is crucial for early diagnosis and rehabilitation assessment. This study introduces a novel concept for non-contact cardiac linear vibration (SCG) and rotational components (GCGx and GCGy) decoupling and reconstruction by integrating speckle motion signals captured from two cameras with different defocus levels. The intention is to overcome the motion coupling issues inherent in single-camera imaging and improve the accuracy in characterizing the cardiac complex 3D mechanical behavior. Using a sternum-mounted inertial sensor as the reference, experiments were conducted on 42 subjects in laboratory and intensive care unit settings. The results show that the reconstructed cardiac 3D motion signals exhibit greater waveform similarity to the reference signal than the raw speckle motion signal from a single camera, with similarity indices above 87. 471%. In addition, with an 8 ms tolerance error, the localization accuracy of 6 key biomarkers (aortic valve opening/closing (AO/AC), mitral valve opening/closing (MO/MC), the biomarkers corresponding to the AO event in GCGy and the MC event in GCGx) are 73. 080%, 99. 998%, 85. 587%, 86. 617%, 99. 683% and 77. 301%, respectively. These results also outperform those obtained from the raw speckle motion signal. These findings validate the rationale and effectiveness of using dual-camera imaging with different defocus levels to reconstruct SCG, GCGx, and GCGy, offering a promising approach for accurately capturing complex cardiac 3D motion and improving cardiac function assessment.

Details DOI

IJCAI Conference 2025 Conference Paper

Deconfounding Multi-Cause Latent Confounders: A Factor-Model Approach to Climate Model Bias Correction

  • Wentao Gao
  • Jiuyong Li
  • Debo Cheng
  • Lin Liu
  • Jixue Liu
  • Thuc Le
  • Xiaojing Du
  • Xiongren Chen

Global Climate Models (GCMs) are crucial for predicting future climate changes by simulating the Earth systems. However, GCM outputs exhibit systematic biases due to model uncertainties, parameterization simplifications, and inadequate representation of complex climate phenomena. Traditional bias correction methods, which rely on historical observation data and statistical techniques, often neglect unobserved confounders, leading to biased results. This paper proposes a novel bias correction approach to utilize both GCM and observational data to learn a factor model that captures multi-cause latent confounders. Inspired by recent advances in causality based time series deconfounding, our method first constructs a factor model to learn latent confounders from historical data and then applies them to enhance the bias correction process using advanced time series forecasting models. The experimental results demonstrate significant improvements in the accuracy of precipitation outputs. By addressing unobserved confounders, our approach offers a robust and theoretically grounded solution for climate model bias correction.

PDF Details DOI

AAAI Conference 2025 Conference Paper

Diffusion Models for Attribution

  • Xiongren Chen
  • Jiuyong Li
  • Jixue Liu
  • Lin Liu
  • Stefan Peters
  • Thuc Duy Le
  • Wentao Gao
  • Xiaojing Du

In high-stakes domains such as healthcare, finance, and law, the need for explainable AI is critical. Traditional methods for generating attribution maps, including white-box approaches relying on gradients and black-box techniques that perturb inputs, face challenges like gradient vanishing, blurred attributions, and computational inefficiencies. To overcome these limitations, we introduce a novel approach that leverages diffusion models within the framework of Information Bottleneck (IB) theory. By utilizing the Gaussian noise from diffusion models, we connect the information bottleneck with the Minimum Mean Squared Error (MMSE) from classical information theory, enabling precise calculation of mutual information. This connection leads to a new loss function that minimizes the Signal-to-Noise Ratio (SNR), facilitating efficient optimization and producing high-resolution, pixel-level attribution maps. Our method achieves greater clarity and accuracy in attributions than existing techniques, requiring significantly fewer pixel values to reach the necessary predictive confidence. This work demonstrates the power of diffusion models in advancing explainable AI, particularly in identifying critical input features with high precision.

PDF Details DOI

AAAI Conference 2025 Conference Paper

GenAuction: A Generative Auction for Online Advertising

  • Yuchao Ma
  • Ruohan Qian
  • Bingzhe Wang
  • Qi Qi
  • Wenqiang Liu
  • Qian Tang
  • Zhao Shen
  • Wei Zhong

Previous ad auctions predominantly relied on rule-based mechanisms, which selected winning advertisements (ads) at the ad-level and subsequently combined them into page views (PVs), leading to suboptimal allocations in multi-round auctions. This limitation stems from the significant computational burden required to design ranking score rules and select winning ad sets, as well as the inability to fully capture contextual information within PVs during ad-level selection. In this paper, we propose a key-performance-indicator (KPI) based auction mechanism that selects winning PVs at the PV-level, modeling the ad allocation as a constrained optimization problem. This approach enables us to address both short-term and long-term KPIs while leveraging the comprehensive contextual information available within PVs. Based on this framework, we design GenAuction, a generative auction mechanism utilizing a Generator-Evaluator architecture powered by Transformer algorithms. The Generator swiftly generates multiple candidate PVs, while the Evaluator selects the optimal PVs based on contextual information, adhering to the objectives and KPIs of multi-round auctions. We conduct extensive experiments using real-world data and online A/B tests to validate that GenAuction efficiently handles multi-objective allocation tasks, demonstrating its efficacy and potential for real-world application.

PDF Details DOI

IJCAI Conference 2025 Conference Paper

Interaction-Data-guided Conditional Instrumental Variables for Debiasing Recommender Systems

  • Zhirong Huang
  • Debo Cheng
  • Lin Liu
  • Jiuyong Li
  • Guangquan Lu
  • Shichao Zhang

It is often challenging to identify a valid instrumental variable (IV), although the IV methods have been regarded as effective tools of addressing the confounding bias introduced by latent variables. To deal with this issue, an Interaction-Data-guided Conditional IV (IDCIV) debiasing method is proposed for Recommender Systems, called IDCIV-RS. The IDCIV-RS automatically generates the representations of valid CIVs and their corresponding conditioning sets directly from interaction data, significantly reducing the complexity of IV selection while effectively mitigating the confounding bias caused by latent variables in recommender systems. Specifically, the IDCIV-RS leverages a variational autoencoder (VAE) to learn both the CIV representations and their conditioning sets from interaction data, followed by the application of least squares to derive causal representations for click prediction. Extensive experiments on two real-world datasets, Movielens-10M and Douban-Movie, demonstrate that IDCIV-RS successfully learns the representations of valid CIVs, effectively reduces bias, and consequently improves recommendation accuracy.

PDF Details DOI

JBHI Journal 2025 Journal Article

Leveraging Channel Coherence in Long-Term iEEG Data for Seizure Prediction

  • Sha Lu
  • Lin Liu
  • Jiuyong Li
  • Jordan Chambers
  • Mark J. Cook
  • David B. Grayden

Epilepsy affects millions worldwide, posing significant challenges due to the erratic and unexpected nature of seizures. Despite advancements, existing seizure prediction techniques remain limited in their ability to forecast seizures with high accuracy, impacting the quality of life for those with epilepsy. This research introduces the Coherence-based Seizure Prediction (CoSP) method, which integrates coherence analysis with deep learning to enhance seizure prediction efficacy. In CoSP, electroencephalography (EEG) recordings are divided into 10-second segments to extract channel pairwise coherence. This coherence data is then used to train a four-layer convolutional neural network to predict the probability of being in a preictal state. The predicted probabilities are then processed to issue seizure warnings. CoSP was evaluated in a pseudo-prospective setting using long-term iEEG data from ten patients in the NeuroVista seizure advisory system. CoSP demonstrated promising predictive performance across a range of preictal intervals (4 to 180 minutes). CoSP achieved a median Seizure Sensitivity (SS) of 0. 79, a median false alarm rate of 0. 15 per hour, and a median Time in Warning (TiW) of 27%, highlighting its potential for accurate and reliable seizure prediction. Statistical analysis confirmed that CoSP significantly outperformed chance (p = 0. 001) and other baseline methods (p <0. 05) under similar evaluation configurations.

Details DOI

JBHI Journal 2025 Journal Article

scIDPMs: Single-Cell RNA-Seq Imputation Using Diffusion Probabilistic Models

  • Zhiqiang Zhang
  • Lin Liu

Single-cell RNA sequencing (scRNA-seq) technology has revolutionized biological research by enabling the sequencing of mRNA in individual cells, thereby providing valuable insights into cellular gene expression and functions. However, scRNA-seq data often contain false zero values known as dropout events, which can obscure true gene expression levels and compromise downstream analysis accuracy. To address this issue, several computational approaches have been proposed for imputing missing gene expression values. Nevertheless, these methods struggle to capture dropout value distributions due to the sparsity of scRNA-seq data and complex gene expression patterns. In this study, we present a novel method called scIDPMs that utilizes conditional diffusion probabilistic models to impute scRNA-seq data. Firstly, scIDPMs identifies dropout sites based on gene expression characteristics and subsequently infers the missing values by considering available gene expression information. To effectively capture global gene expression features, scIDPMs employs a deep neural network with an attention mechanism to optimize the imputation process. We evaluated the performance of scIDPMs using simulated and real scRNA-seq datasets and compared it with ten other imputation methods. The results indicate that scIDPMs outperform other methods in restoring biologically meaningful gene expression values and improving downstream analysis.

Details DOI

JBHI Journal 2024 Journal Article

Camera-Based Seismocardiogram for Heart Rate Variability Monitoring

  • Lin Liu
  • Dongfang Yu
  • Hongzhou Lu
  • Caifeng Shan
  • Wenjin Wang

Heart rate variability (HRV) is a crucial metric that quantifies the variation between consecutive heartbeats, serving as a significant indicator of autonomic nervous system (ANS) activity. It has found widespread applications in clinical diagnosis, treatment, and prevention of cardiovascular diseases. In this study, we proposed an optical model for defocused speckle imaging, to simultaneously incorporate out-of-plane translation and rotation-induced motion for highly-sensitive non-contact seismocardiogram (SCG) measurement. Using electrocardiogram (ECG) signals as the gold standard, we evaluated the performance of photoplethysmogram (PPG) signals and speckle-based SCG signals in assessing HRV. The results indicated that the HRV parameters measured from SCG signals extracted from laser speckle videos showed higher consistency with the results obtained from the ECG signals compared to PPG signals. Additionally, we confirmed that even when clothing obstructed the measurement site, the efficacy of SCG signals extracted from the motion of laser speckle patterns persisted in assessing the HRV levels. This demonstrates the robustness of camera-based non-contact SCG in monitoring HRV, highlighting its potential as a reliable, non-contact alternative to traditional contact-PPG sensors.

Details DOI

AAAI Conference 2024 Conference Paper

FedCSL: A Scalable and Accurate Approach to Federated Causal Structure Learning

  • Xianjie Guo
  • Kui Yu
  • Lin Liu
  • Jiuyong Li

As an emerging research direction, federated causal structure learning (CSL) aims at learning causal relationships from decentralized data across multiple clients while preserving data privacy. Existing federated CSL algorithms suffer from scalability and accuracy issues, since they require computationally expensive CSL algorithms to be executed at each client. Furthermore, in real-world scenarios, the number of samples held by each client varies significantly, and existing methods still assign equal weights to the learned structural information from each client, which severely harms the learning accuracy of those methods. To address these two limitations, we propose FedCSL, a scalable and accurate method for federated CSL. Specifically, FedCSL consists of two novel strategies: (1) a federated local-to-global learning strategy that enables FedCSL to scale to high-dimensional data for tackling the scalability issue, and (2) a novel weighted aggregation strategy that does not rely on any complex encryption techniques while preserving data privacy for tackling the accuracy issue. Extensive experiments on benchmark datasets, high-dimensional synthetic datasets and a real-world dataset verify the efficacy of the proposed FedCSL method. The source code is available at https://github.com/Xianjie-Guo/FedCSL.

PDF Details DOI

AAAI Conference 2024 Conference Paper

Instrumental Variable Estimation for Causal Inference in Longitudinal Data with Time-Dependent Latent Confounders

  • Debo Cheng
  • Ziqi Xu
  • Jiuyong Li
  • Lin Liu
  • Jixue Liu
  • Wentao Gao
  • Thuc Duy Le

Causal inference from longitudinal observational data is a challenging problem due to the difficulty in correctly identifying the time-dependent confounders, especially in the presence of latent time-dependent confounders. Instrumental variable (IV) is a powerful tool for addressing the latent confounders issue, but the traditional IV technique cannot deal with latent time-dependent confounders in longitudinal studies. In this work, we propose a novel Time-dependent Instrumental Factor Model (TIFM) for time-varying causal effect estimation from data with latent time-dependent confounders. At each time-step, the proposed TIFM method employs the Recurrent Neural Network (RNN) architecture to infer latent IV, and then uses the inferred latent IV factor for addressing the confounding bias caused by the latent time-dependent confounders. We provide a theoretical analysis for the proposed TIFM method regarding causal effect estimation in longitudinal data. Extensive evaluation with synthetic datasets demonstrates the effectiveness of TIFM in addressing causal effect estimation over time. We further apply TIFM to a climate dataset to showcase the potential of the proposed method in tackling real-world problems.

PDF Details DOI

AAAI Conference 2024 Conference Paper

PREFER: Prompt Ensemble Learning via Feedback-Reflect-Refine

  • Chenrui Zhang
  • Lin Liu
  • Chuyuan Wang
  • Xiao Sun
  • Hongyu Wang
  • Jinpeng Wang
  • Mingchen Cai

As an effective tool for eliciting the power of Large Language Models (LLMs), prompting has recently demonstrated unprecedented abilities across a variety of complex tasks. To further improve the performance, prompt ensemble has attracted substantial interest for tackling the hallucination and instability of LLMs. However, existing methods usually adopt a two-stage paradigm, which requires a pre-prepared set of prompts with substantial manual effort, and is unable to perform directed optimization for different weak learners. In this paper, we propose a simple, universal, and automatic method named PREFER (Prompt Ensemble learning via Feedback-Reflect-Refine) to address the stated limitations. Specifically, given the fact that weak learners are supposed to focus on hard examples during boosting, PREFER builds a feedback mechanism for reflecting on the inadequacies of existing weak learners. Based on this, the LLM is required to automatically synthesize new prompts for iterative refinement. Moreover, to enhance stability of the prompt effect evaluation, we propose a novel prompt bagging method involving forward and backward thinking, which is superior to majority voting and is beneficial for both feedback and weight calculation in boosting. Extensive experiments demonstrate that our PREFER achieves state-of-the-art performance in multiple types of tasks by a significant margin. We have made our code publicly available.

PDF Details DOI

IJCAI Conference 2024 Conference Paper

RoboFusion: Towards Robust Multi-Modal 3D Object Detection via SAM

  • Ziying Song
  • Guoxing Zhang
  • Lin Liu
  • Lei Yang
  • Shaoqing Xu
  • Caiyan Jia
  • Feiyang Jia
  • Li Wang

Multi-modal 3D object detectors are dedicated to exploring secure and reliable perception systems for autonomous driving (AD). Although achieving state-of-the-art (SOTA) performance on clean benchmark datasets, they tend to overlook the complexity and harsh conditions of real-world environments. With the emergence of visual foundation models (VFMs), opportunities and challenges are presented for improving the robustness and generalization of multi-modal 3D object detection in AD. Therefore, we propose RoboFusion, a robust framework that leverages VFMs like SAM to tackle out-of-distribution (OOD) noise scenarios. We first adapt the original SAM for AD scenarios named SAM-AD. To align SAM or SAM-AD with multi-modal methods, we then introduce AD-FPN for upsampling the image features extracted by SAM. We employ wavelet decomposition to denoise the depth-guided images for further noise reduction and weather interference. At last, we employ self-attention mechanisms to adaptively reweight the fused features, enhancing informative features while suppressing excess noise. In summary, RoboFusion significantly reduces noise by leveraging the generalization and robustness of VFMs, thereby enhancing the resilience of multi-modal 3D object detection. Consequently, RoboFusion achieves SOTA performance in noisy scenarios, as demonstrated by the KITTI-C and nuScenes-C benchmarks. Code is available at https: //github. com/adept-thu/RoboFusion.

PDF Details DOI

AAAI Conference 2023 Conference Paper

Causal Inference with Conditional Instruments Using Deep Generative Models

  • Debo Cheng
  • Ziqi Xu
  • Jiuyong Li
  • Lin Liu
  • Jixue Liu
  • Thuc Duy Le

The instrumental variable (IV) approach is a widely used way to estimate the causal effects of a treatment on an outcome of interest from observational data with latent confounders. A standard IV is expected to be related to the treatment variable and independent of all other variables in the system. However, it is challenging to search for a standard IV from data directly due to the strict conditions. The conditional IV (CIV) method has been proposed to allow a variable to be an instrument conditioning on a set of variables, allowing a wider choice of possible IVs and enabling broader practical applications of the IV approach. Nevertheless, there is not a data-driven method to discover a CIV and its conditioning set directly from data. To fill this gap, in this paper, we propose to learn the representations of the information of a CIV and its conditioning set from data with latent confounders for average causal effect estimation. By taking advantage of deep generative models, we develop a novel data-driven approach for simultaneously learning the representation of a CIV from measured variables and generating the representation of its conditioning set given measured variables. Extensive experiments on synthetic and real-world datasets show that our method outperforms the existing IV methods.

PDF Details DOI

AAAI Conference 2023 Conference Paper

Disentangled Representation for Causal Mediation Analysis

  • Ziqi Xu
  • Debo Cheng
  • Jiuyong Li
  • Jixue Liu
  • Lin Liu
  • Ke Wang

Estimating direct and indirect causal effects from observational data is crucial to understanding the causal mechanisms and predicting the behaviour under different interventions. Causal mediation analysis is a method that is often used to reveal direct and indirect effects. Deep learning shows promise in mediation analysis, but the current methods only assume latent confounders that affect treatment, mediator and outcome simultaneously, and fail to identify different types of latent confounders (e.g., confounders that only affect the mediator or outcome). Furthermore, current methods are based on the sequential ignorability assumption, which is not feasible for dealing with multiple types of latent confounders. This work aims to circumvent the sequential ignorability assumption and applies the piecemeal deconfounding assumption as an alternative. We propose the Disentangled Mediation Analysis Variational AutoEncoder (DMAVAE), which disentangles the representations of latent confounders into three types to accurately estimate the natural direct effect, natural indirect effect and total effect. Experimental results show that the proposed method outperforms existing methods and has strong generalisation ability. We further apply the method to a real-world dataset to show its potential application.

PDF Details DOI

AAAI Conference 2023 Conference Paper

Low-Light Video Enhancement with Synthetic Event Guidance

  • Lin Liu
  • Junfeng An
  • Jianzhuang Liu
  • Shanxin Yuan
  • Xiangyu Chen
  • Wengang Zhou
  • Houqiang Li
  • Yan Feng Wang

Low-light video enhancement (LLVE) is an important yet challenging task with many applications such as photographing and autonomous driving. Unlike single image low-light enhancement, most LLVE methods utilize temporal information from adjacent frames to restore the color and remove the noise of the target frame. However, these algorithms, based on the framework of multi-frame alignment and enhancement, may produce multi-frame fusion artifacts when encountering extreme low light or fast motion. In this paper, inspired by the low latency and high dynamic range of events, we use synthetic events from multiple frames to guide the enhancement and restoration of low-light videos. Our method contains three stages: 1) event synthesis and enhancement, 2) event and image fusion, and 3) low-light enhancement. In this framework, we design two novel modules (event-image fusion transform and event-guided dual branch) for the second and third stages, respectively. Extensive experiments show that our method outperforms existing low-light video or single image enhancement approaches on both synthetic and real LLVE datasets. Our code will be available at https://gitee.com/mindspore/models/tree/master/research/cv/LLVE-SEG.

PDF Details DOI

NeurIPS Conference 2023 Conference Paper

SAMRS: Scaling-up Remote Sensing Segmentation Dataset with Segment Anything Model

  • Di Wang
  • Jing Zhang
  • Bo Du
  • Minqiang Xu
  • Lin Liu
  • Dacheng Tao
  • Liangpei Zhang

The success of the Segment Anything Model (SAM) demonstrates the significance of data-centric machine learning. However, due to the difficulties and high costs associated with annotating Remote Sensing (RS) images, a large amount of valuable RS data remains unlabeled, particularly at the pixel level. In this study, we leverage SAM and existing RS object detection datasets to develop an efficient pipeline for generating a large-scale RS segmentation dataset, dubbed SAMRS. SAMRS totally possesses 105, 090 images and 1, 668, 241 instances, surpassing existing high-resolution RS segmentation datasets in size by several orders of magnitude. It provides object category, location, and instance information that can be used for semantic segmentation, instance segmentation, and object detection, either individually or in combination. We also provide a comprehensive analysis of SAMRS from various aspects. Moreover, preliminary experiments highlight the importance of conducting segmentation pre-training with SAMRS to address task discrepancies and alleviate the limitations posed by limited training data during fine-tuning. The code and dataset will be available at https: //github. com/ViTAE-Transformer/SAMRS

PDF Details

IJCAI Conference 2022 Conference Paper

Ancestral Instrument Method for Causal Inference without Complete Knowledge

  • Debo Cheng
  • Jiuyong Li
  • Lin Liu
  • Jiji Zhang
  • Thuc Duy Le
  • Jixue Liu

Unobserved confounding is the main obstacle to causal effect estimation from observational data. Instrumental variables (IVs) are widely used for causal effect estimation when there exist latent confounders. With the standard IV method, when a given IV is valid, unbiased estimation can be obtained, but the validity requirement on a standard IV is strict and untestable. Conditional IVs have been proposed to relax the requirement of standard IVs by conditioning on a set of observed variables (known as a conditioning set for a conditional IV). However, the criterion for finding a conditioning set for a conditional IV needs a directed acyclic graph (DAG) representing the causal relationships of both observed and unobserved variables. This makes it challenging to discover a conditioning set directly from data. In this paper, by leveraging maximal ancestral graphs (MAGs) for causal inference with latent variables, we study the graphical properties of ancestral IVs, a type of conditional IVs using MAGs, and develop the theory to support data-driven discovery of the conditioning set for a given ancestral IV in data under the pretreatment variable assumption. Based on the theory, we develop an algorithm for unbiased causal effect estimation with a given ancestral IV and observational data. Extensive experiments on synthetic and real-world datasets demonstrate the performance of the algorithm in comparison with existing IV methods.

PDF Details DOI

NeurIPS Conference 2022 Conference Paper

DeepMed: Semiparametric Causal Mediation Analysis with Debiased Deep Learning

  • Siqi Xu
  • Lin Liu
  • Zhonghua Liu

Causal mediation analysis can unpack the black box of causality and is therefore a powerful tool for disentangling causal pathways in biomedical and social sciences, and also for evaluating machine learning fairness. To reduce bias for estimating Natural Direct and Indirect Effects in mediation analysis, we propose a new method called DeepMed that uses deep neural networks (DNNs) to cross-fit the infinite-dimensional nuisance functions in the efficient influence functions. We obtain novel theoretical results that our DeepMed method (1) can achieve semiparametric efficiency bound without imposing sparsity constraints on the DNN architecture and (2) can adapt to certain low dimensional structures of the nuisance functions, significantly advancing the existing literature on DNN-based semiparametric causal inference. Extensive synthetic experiments are conducted to support our findings and also expose the gap between theory and practice. As a proof of concept, we apply DeepMed to analyze two real datasets on machine learning fairness and reach conclusions consistent with previous findings.

PDF Details

JBHI Journal 2022 Journal Article

SDoF-Net: Super Depth of Field Network for Cell Detection in Leucorrhea Micrograph

  • Xiaohui Du
  • Xiangzhou Wang
  • Guangming Ni
  • Jing Zhang
  • Ruqian Hao
  • Jiaxi Zhao
  • Xudong Wang
  • Juanxiu Liu

Accompanied with the rapid increase of the demand for routine examination of leucorrhea, efficiency and accuracy become the primary task. However, in super depth of field (SDoF) system, the problem of automatic detection and localization of cells in leucorrhea micro-images is still a big challenge. The changing of the relative position between the cell center and focus plane of microscope lead to variable cell morphological structure in the two-dimensional image, which is an important reason for the low accuracy of current deep learning target detection algorithms. In this paper, an object detection method based on Retinanet in state of super depth of field is proposed, which can achieve high precision detecting of leucorrhea components by the SDoF feature aggregation module. Compared with the current mainstream algorithms, the mean average accuracy (mAP) index has been improved significantly, the mAP index is 82. 7% for SDoF module and 83. 0% for SDoF+ module, with an average increase of more than 10%. These improved features can significantly improve the efficiency and accuracy of the algorithm. The algorithm proposed in this paper can be integrated into the leucorrhea automatic detection system.

Details DOI

AAAI Conference 2022 Conference Paper

SiamTrans: Zero-Shot Multi-Frame Image Restoration with Pre-trained Siamese Transformers

  • Lin Liu
  • Shanxin Yuan
  • Jianzhuang Liu
  • Xin Guo
  • Youliang Yan
  • Qi Tian

We propose a novel zero-shot multi-frame image restoration method for removing unwanted obstruction elements (such as rains, snow, and moiré patterns) that vary in successive frames. It has three stages: transformer pre-training, zero-shot restoration, and hard patch refinement. Using the pre-trained transformers, our model is able to tell the motion difference between the true image information and the obstructing elements. For zero-shot image restoration, we design a novel model, termed SiamTrans, which is constructed by Siamese transformers, encoders, and decoders. Each transformer has a temporal attention layer and several self-attention layers, to capture both temporal and spatial information of multiple frames. Only pre-trained (self-supervised) on the denoising task, SiamTrans is tested on three different low-level vision tasks (deraining, demoiréing, and desnowing). Compared with related methods, ours achieves the best performances, even outperforming those with supervised learning.

PDF Details

JMLR Journal 2021 Journal Article

Adaptive estimation of nonparametric functionals

  • Lin Liu
  • Rajarshi Mukherjee
  • James M. Robins
  • Eric Tchetgen Tchetgen

We provide general adaptive upper bounds for estimating nonparametric functionals based on second-order U-statistics arising from finite-dimensional approximation of the infinite-dimensional models. We then provide examples of functionals for which the theory produces rate optimally matching adaptive upper and lower bounds. Our results are automatically adaptive in both parametric and nonparametric regimes of estimation and are automatically adaptive and semiparametric efficient in the regime of parametric convergence rate. [abs] [ pdf ][ bib ] &copy JMLR 2021. ( edit, beta )

PDF Details

AAAI Conference 2021 Conference Paper

Treatment Effect Estimation with Disentangled Latent Factors

  • Weijia Zhang
  • Lin Liu
  • Jiuyong Li

Much research has been devoted to the problem of estimating treatment effects from observational data; however, most methods assume that the observed variables only contain confounders, i. e. , variables that affect both the treatment and the outcome. Unfortunately, this assumption is frequently violated in real-world applications, since some variables only affect the treatment but not the outcome, and vice versa. Moreover, in many cases only the proxy variables of the underlying confounding factors can be observed. In this work, we first show the importance of differentiating confounding factors from instrumental and risk factors for both average and conditional average treatment effect estimation, and then we propose a variational inference approach to simultaneously infer latent factors from the observed variables, disentangle the factors into three disjoint sets corresponding to the instrumental, confounding, and risk factors, and use the disentangled factors for treatment effect estimation. Experimental results demonstrate the effectiveness of the proposed method on a wide range of synthetic, benchmark, and real-world datasets.

PDF Details

NeurIPS Conference 2020 Conference Paper

Self-Adaptively Learning to Demoiré from Focused and Defocused Image Pairs

  • Lin Liu
  • Shanxin Yuan
  • Jianzhuang Liu
  • Liping Bao
  • Gregory Slabaugh
  • Qi Tian

Moiré artifacts are common in digital photography, resulting from the interference between high-frequency scene content and the color filter array of the camera. Existing deep learning-based demoiréing methods trained on large scale datasets are limited in handling various complex moiré patterns, and mainly focus on demoiréing of photos taken of digital displays. Moreover, obtaining moiré-free ground-truth in natural scenes is difficult but needed for training. In this paper, we propose a self-adaptive learning method for demoiréing a high-frequency image, with the help of an additional defocused moiré-free blur image. Given an image degraded with moiré artifacts and a moiré-free blur image, our network predicts a moiré-free clean image and a blur kernel with a self-adaptive strategy that does not require an explicit training stage, instead performing test-time adaptation. Our model has two sub-networks and works iteratively. During each iteration, one sub-network takes the moiré image as input, removing moiré patterns and restoring image details, and the other sub-network estimates the blur kernel from the blur image. The two sub-networks are jointly optimized. Extensive experiments demonstrate that our method outperforms state-of-the-art methods and can produce high-quality demoiréd results. It can generalize well to the task of removing moiré artifacts caused by display screens. In addition, we build a new moiré dataset, including images with screen and texture moiré artifacts. As far as we know, this is the first dataset with real texture moiré patterns.

PDF Details

TIST Journal 2019 Journal Article

BAMB

  • Zhaolong Ling
  • Kui Yu
  • Hao Wang
  • Lin Liu
  • Wei Ding
  • Xindong Wu

The discovery of Markov blanket (MB) for feature selection has attracted much attention in recent years, since the MB of the class attribute is the optimal feature subset for feature selection. However, almost all existing MB discovery algorithms focus on either improving computational efficiency or boosting learning accuracy, instead of both. In this article, we propose a novel MB discovery algorithm for balancing efficiency and accuracy, called <underline>BA</underline>lanced <underline>M</underline>arkov <underline>B</underline>lanket (BAMB) discovery. To achieve this goal, given a class attribute of interest, BAMB finds candidate PC (parents and children) and spouses and removes false positives from the candidate MB set in one go. Specifically, once a feature is successfully added to the current PC set, BAMB finds the spouses with regard to this feature, then uses the updated PC and the spouse set to remove false positives from the current MB set. This makes the PC and spouses of the target as small as possible and thus achieves a trade-off between computational efficiency and learning accuracy. In the experiments, we first compare BAMB with 8 state-of-the-art MB discovery algorithms on 7 benchmark Bayesian networks, then we use 10 real-world datasets and compare BAMB with 12 feature selection algorithms, including 8 state-of-the-art MB discovery algorithms and 4 other well-established feature selection methods. On prediction accuracy, BAMB outperforms 12 feature selection algorithms compared. On computational efficiency, BAMB is close to the IAMB algorithm while it is much faster than the remaining seven MB discovery algorithms.

Details DOI

IJCAI Conference 2017 Conference Paper

A Structural Representation Learning for Multi-relational Networks

  • Lin Liu
  • Xin Li
  • William K. Cheung
  • Chengcheng Xu

Most of the existing multi-relational network embedding methods, e. g. , TransE, are formulated to preserve pair-wise connectivity structures in the networks. With the observations that significant triangular connectivity structures and parallelogram connectivity structures found in many real multi-relational networks are often ignored and that a hard-constraint commonly adopted by most of the network embedding methods is inaccurate by design, we propose a novel representation learning model for multi-relational networks which can alleviate both fundamental limitations. Scalable learning algorithms are derived using the stochastic gradient descent algorithm and negative sampling. Extensive experiments on real multi-relational network datasets of WordNet and Freebase demonstrate the efficacy of the proposed model when compared with the state-of-the-art embedding methods.

PDF Details

TIST Journal 2015 Journal Article

From Observational Studies to Causal Rule Mining

  • Jiuyong Li
  • Thuc Duy Le
  • Lin Liu
  • Jixue Liu
  • Zhou Jin
  • Bingyu Sun
  • Saisai Ma

Randomised controlled trials (RCTs) are the most effective approach to causal discovery, but in many circumstances it is impossible to conduct RCTs. Therefore, observational studies based on passively observed data are widely accepted as an alternative to RCTs. However, in observational studies, prior knowledge is required to generate the hypotheses about the cause-effect relationships to be tested, and hence they can only be applied to problems with available domain knowledge and a handful of variables. In practice, many datasets are of high dimensionality, which leaves observational studies out of the opportunities for causal discovery from such a wealth of data sources. In another direction, many efficient data mining methods have been developed to identify associations among variables in large datasets. The problem is that causal relationships imply associations, but the reverse is not always true. However, we can see the synergy between the two paradigms here. Specifically, association rule mining can be used to deal with the high-dimensionality problem, whereas observational studies can be utilised to eliminate noncausal associations. In this article, we propose the concept of causal rules (CRs) and develop an algorithm for mining CRs in large datasets. We use the idea of retrospective cohort studies to detect CRs based on the results of association rule mining. Experiments with both synthetic and real-world datasets have demonstrated the effectiveness and efficiency of CR mining. In comparison with the commonly used causal discovery methods, the proposed approach generally is faster and has better or competitive performance in finding correct or sensible causes. It is also capable of finding a cause consisting of multiple variables—a feature that other causal discovery methods do not possess.

Details DOI