Arrow Research search

Author name cluster

Dong Liu

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

34 papers
2 author rows

Possible papers

34

YNIMG Journal 2026 Journal Article

Alteration of water exchange rates following focused ultrasound-mediated BBB opening in the dorsal striatum of non-human primates: A diffusion-prepared pCASL study

  • Dong Liu
  • Xingfeng Shao
  • Fabian Munoz Silva
  • Soroosh Sanatkhani
  • Ray Lee
  • Elisa E Konofagou
  • Danny JJ Wang
  • Vincent P Ferrera

This study applied diffusion-prepared pseudo-continuous arterial spin labeling (DP-pCASL) to quantify cerebral blood flow (CBF), arterial transit time (ATT), and blood-brain barrier (BBB) water exchange rate ( K w ) before and after focused ultrasound (FUS)-mediated blood-brain barrier opening (BBBO) in the dorsal striatum of four non-human primates. Six baseline and seven BBBO sessions were performed. DP-pCASL was acquired approximately 45 min after FUS sonication combined with intravenous microbubbles, and contrast-enhanced T1-weighted imaging was subsequently used to confirm the BBBO region. Whole-brain analyses revealed no significant changes in CBF or ATT following BBBO (permutation p > 0. 05). Region-of-interest analysis within the sonicated caudate demonstrated a significant localized decrease in K w, with median (IQR) values of 45. 0 (40. 6 - 55. 6) min⁻¹ at the BBBO site versus 61. 6 (58. 3 - 70. 4) min⁻¹ in the contralateral control region (p < 0. 05), confirming spatially specific suppression of transendothelial water flux. In contrast, whole-brain K w increased significantly following BBBO, with median (IQR) values of 49. 8 (46. 3 - 55. 9) min⁻¹ in non-BBBO sessions versus 59. 4 (56. 6 - 66. 3) min⁻¹ in BBBO sessions (p < 0. 01), indicating a diffuse enhancement of water exchange across the brain. These findings establish DP-pCASL-derived K w as a sensitive, non-contrast biomarker for both local and global BBB permeability changes induced by focused ultrasound, supporting its potential for longitudinal monitoring in preclinical and clinical neurotherapeutic applications.

Details DOI

AAAI Conference 2026 Conference Paper

Neural Video Compression with Reference Hierarchy

  • Chuanbo Tang
  • Zhuoyuan Li
  • Li Li
  • Dong Liu
  • Feng Wu

Efficient reference structures are essential in video compression, enabling the exploitation of temporal dependencies across frames to reduce redundancy. In this paper, we delve into the inter-frame reference management mechanism in neural video codecs (NVCs). Previous schemes have inherited the reference propagation mechanism with the guidance of predefined reference structure, but the reference modeling across diverse reference sources remains underexplored. Moreover, the mismatch between the reference structure used for motion estimation and motion compensation limits the effectiveness of inter-frame prediction. To address the above limitations, we propose the unified reference hierarchy that integrates a learned hierarchical reference structure into the existing inherent reference propagation mechanism. Specifically, we first propose the hierarchical reference structure (HRS) to manage the multiple temporal contexts in the propagated reference feature, where a hierarchy-aware reference modulation module is integrated to select the most relevant reference features across different quality levels under the guidance of the reference balance loss. In addition, we propose the HRS-guided feature-wise inter-frame prediction that learns the low-rank approximation of the selected reference feature for ensuring the consistency and improving the inter-frame prediction performance. We conduct experiments on a state-of-the-art NVC, DCVC-DC. Experimental results show that our codec achieves an average 26% bitrate saving over H.266/VVC, and a 28.2% bitrate reduction compared to DCVC-DC without increasing the decoding complexity.

PDF Details DOI

JBHI Journal 2026 Journal Article

SpineVLM: A Markdown-Guided Structured Fine-Tuning Framework for Spine X-ray Report Generation

  • Dong Liu
  • Wenhui Li
  • Ning Xu
  • Guoge Han
  • Rui Hao
  • Xianzhu Liu
  • An-An Liu

Automated medical report generation in specialized fields like spine radiography is constrained by data scarcity and high annotation costs. Consequently, existing multimodal large language models (MLLMs) struggle in these settings, often missing minute, scattered spinal abnormalities. We introduce SpineVLM, a data-efficient framework for structured spine X-ray report generation. The framework is built upon the newly constructed SXRG dataset, comprising 10, 468 image-report pairs developed via a hierarchical AI-assisted annotation pipeline. To optimize learning under limited data, we propose Markdown-Guided Structured Learning (MGSL), which reformulates unconstrained free-text synthesis into a structured completion task, acting as a strong regularizer. Furthermore, an unsupervised Region-Focused Inference (RFI) module powered by foundation models (DINOv2) isolates the vertebral column to enhance the perception of subtle lesions without requiring manual spatial annotations. Evaluated on a 7B-parameter vision-language backbone, SpineVLM achieves strong performance against ten baseline multimodal models across standard linguistic metrics. In a double-blind reader study, the system achieved a diagnostic F1-score of 0. 866, comparable to specialist performance, while reducing clinical reporting time by over 41%. By open-sourcing the dataset and codebase, we provide, to our knowledge, the first quantitative benchmark for automated spine radiography report generation, together with a structured framework for this data-limited setting. All data and code will be publicly released at https://github.com/LiuDongDaniel/SpineVLM.

Details DOI

AAAI Conference 2026 Conference Paper

Transform-Free Feature Coding via Entropy-Constrained Vector Quantization

  • Qiaoxi Chen
  • Changsheng Gao
  • Li Li
  • Dong Liu

Feature coding has recently emerged as a key technique for efficient transmission of intermediate representations in distributed AI systems. Existing approaches largely follow a transform-based pipeline inherited from image and video coding, where the transform module is used to remove spatial structural redundancies in visual signals. However, our analysis indicates that such redundancies have already been largely removed during feature extraction, which reduces the necessity of the transform module. Building on this insight, we propose a new transform-free pipeline that directly encodes the extracted features via a vector quantization module and an entropy model. The proposed transform‑free framework jointly learns the quantization codebook and entropy model, enabling end‑to‑end optimization tailored to the inherent feature characteristics. Furthermore, the proposed method inherently avoids the computational complexity of the transform module. Experiments on features from diverse architectures and tasks demonstrate that our method achieves superior rate-distortion performance compared to transform-based baselines, while significantly reducing the encoding and decoding complexity.

PDF Details DOI

EAAI Journal 2025 Journal Article

A cross-domain multi-scale feature fusion network based on graph convolution for intelligent fault diagnosis

  • Quanyu Zhong
  • Qiang Li
  • Junxiao Ren
  • Xin Chen
  • Dong Liu
  • Qiang Yang

Fault diagnosis of mechanical equipment plays a critical role in enhancing system stability and ensuring operational safety. Multi-modal monitoring data provides a more comprehensive view of the equipment’s condition, enabling more precise state diagnosis through the processing of such data. A major challenge in contemporary multi-modal information fusion lies in the effective integration of cross-modal information. To overcome the inherent limitations of single-view approaches often found in existing multi-modal methods, this paper presents a multi-view, multi-modal information fusion approach based on graph convolutional networks. This method not only extracts key features from signals efficiently but also uncovers the interrelationships between different modal signals through a multi-view interaction mechanism, achieving more robust information fusion and enhanced fault diagnosis performance. This method consists of three core modules: feature extraction, information fusion, and classification. The feature extraction module utilizes a multi-level residual architecture, Mini-Long Short Term Memory, and self-attention mechanisms to capture both local and global signal features, model temporal dependencies, and refine feature selection. The information fusion module combines data and feature-level signal correlations using graph convolutional networks for cross-modal interaction. The classification module employs a two-layer fully connected network for fault diagnosis. This method quantitatively assesses the contribution of each modality, providing a theoretical basis for understanding their physical significance. Experimental results and ablation studies demonstrate its superior performance and enhanced accuracy in fault diagnosis, offering a novel approach for condition monitoring of complex equipment.

Details DOI

EAAI Journal 2025 Journal Article

A fruit fly-inspired path planning algorithm for unmanned aerial vehicle in underground environments based on low-discrepancy sequences

  • Huageng Zhong
  • Yu Du
  • Dong Liu
  • Minghao Wang
  • Ming Cong
  • Xiaojing Tian

Sampling planning algorithms are crucial in high-dimensional path planning for unmanned aerial vehicle (UAV), particularly in underground environments where Global Positioning System (GPS) signals are absent. The Rapidly-exploring Random Tree (RRT) algorithm, however, faces challenges due to the use of pseudo-random sequences, resulting in issues such as under-sampling, over-sampling, and they also suffer from high computational costs and redundant paths. To address these limitations, the (Halton-based Clustering) HBC-RRT algorithm is proposed. This algorithm utilizes the Halton sequence, replacing the pseudo-random sequence, which fundamentally resolves the problems of under-sampling and over-sampling in the RRT method. Additionally, a novel sampler is introduced, inspired by the fruit fly guidance mechanism, which optimizes the dual-tree sampling process by selecting either the optimal sampling candidate strategy or virtual sub-goal points. The optimal sampling candidate strategy conducts global sampling in three-dimensional space, effectively reducing sampling time. The virtual sub-goal points accelerate the dual-tree connection process, enhancing the directional guidance. The fruit fly mechanism reconstructs the path, leading to a reduction in UAV path costs. The performance of the HBC-RRT algorithm is evaluated through simulation experiments, demonstrating superior results in comparison to four baseline algorithms, including reductions in path length, convergence time, and memory usage. The practical applicability of the algorithm is further validated through real-world experiments involving a UAV and radar in an underground environment.

Details DOI

EAAI Journal 2025 Journal Article

A human-robot interaction system for automated chemical experiments based on vision and natural language processing semantics

  • Zhuang Yang
  • Yu Du
  • Dong Liu
  • Kesong Zhao
  • Ming Cong

Using collaborative robots to replace researchers in performing repetitive and hazardous chemical experiments can effectively enhance experimental efficiency. However, this technology still faces several challenges, including understanding researchers' natural language instructions, autonomously generating action sequences, and more. Therefore, we developed a general control framework for robots in automated chemical experiments based on visual and natural language semantic information. Firstly, starting with the recognition of keywords within Chinese language instructions, we established a domain dictionary for chemical experiment operations and proposed an instruction understanding model based on the bidirectional long-short-term memory and conditional random field(BiLSTM-CRF), enhancing the robot's cognitive ability towards user instructions. Then, a rule matching method for chemical experimental information and a multimodal information feature matching mechanism were established for command content verification and the automatic generation of multiple types of structured language. At the same time, a robot feedback mechanism was added, enabling human-computer interaction and establishing closed-loop control of the system. Finally, propose a robot action sequence generation mechanism based on hierarchical finite state machines(HFSM), transforming structured language into operational strategies for chemical experiments required by the robot. Experimental results show that on the instruction task comprehension dataset created in this paper, the proposed method improves the F1 score by up to 4. 44% in the instruction keyword extraction task compared to other models. In addition, compared to traditional manual teaching control, this method significantly reduces time costs. This verifies that the method effectively enhances the robot's ability to comprehend Chinese instructions and generates reliable executable action sequences.

Details DOI

EAAI Journal 2025 Journal Article

A small object detection algorithm for mine environment

  • Dong Liu
  • Xin Zhao
  • Weiqiang Fan

The detection of protective equipment carried by underground mine operators is a crucial measure for preventing safety accidents and safeguarding personal life and property. However, current challenges include low object detection accuracy and difficulty detecting small objects, we propose a small object detection algorithm based on the improved You Only Look Once Version 8 (YOLOv8) for the mine environment. To minimize the semantic gap between features at different levels and enhance the feature fusion effect, the Asymptotic Feature Pyramid Network-Four (AFPN-F) has been designed to replace the Neck component of YOLOv8, enabling the detection model to better adapt to semantic information across varying levels. To enhance the model's sensitivity to small objects in the mine environment, a superficial feature output layer has been added to the model. This addition helps to prevent the loss of small-sized objects, which may contain limited feature information, during successive convolution operations. To address the significant differences in the scales of various objects in the mine, the More Focused Intersection over Union Loss (Focaler-IoU) is introduced as a loss function. This modification is intended to improve the handling of different types of regression samples, enhance training accuracy, and ensure that the model is more focused on small objects in the mine environment. The experimental results show that the proposed model outperforms other mainstream models. Compared to the baseline model YOLOv8, achieving an improvement of 3. 7 percent in mean Average Precision (mAP), the number of parameters has been reduced by 30 percent, resulting in a model size of only 5 Megabytes. This study provides an effective solution for detecting small objects in underground mines.

Details DOI

EAAI Journal 2025 Journal Article

A survey on closed-loop intelligent frameworks for parallel training of deep neural networks

  • Zhiyuan Ren
  • shijie zhou
  • Dong Liu
  • Qihe Liu

This paper proposes a novel closed-loop intelligent framework to overcome the critical challenges of computational inefficiency, memory limitations, and communication overhead in large-scale deep neural network training. Unlike conventional static or decoupled approaches, our methodology integrates environmental perception, policy decision-making, and execution optimization into a dynamic, cohesive system capable of self-adaptation. The core of our research involves a dynamic collaboration kernel for real-time hardware monitoring, a unified multi-objective cost model for Pareto-optimal resource allocation, and a generalized architecture for cross-domain deployment. Our major findings demonstrate that this framework reduces communication latency by 40% and improves training throughput by 1. 45 × in Generative Pre-trained Transformer 3 (GPT-3)-like models. It achieves a 58% reduction in end-to-end training time for 175-billion-parameter models and maintains robust performance under significant resource fluctuations, such as 30% bandwidth variations. Validation across diverse scenarios shows 81% weak scaling efficiency in computational fluid dynamics simulations and a 9. 3 × speedup for generalized linear model training on edge devices. We conclude that the framework establishes a new paradigm for scalable, adaptive, and efficient deep learning, effectively bridging the gap between system-level parallelism and algorithmic intelligence. Its generalizability offers significant potential for applications in scientific computing, edge intelligence, and next-generation model architectures.

Details DOI

AAAI Conference 2025 Conference Paper

Conditional Latent Coding with Learnable Synthesized Reference for Deep Image Compression

  • Siqi Wu
  • Yinda Chen
  • Dong Liu
  • Zhihai He

In this paper, we study how to synthesize a dynamic reference from an external dictionary to perform conditional coding of the input image in the latent domain and how to learn the conditional latent synthesis and coding modules in an end-to-end manner. Our approach begins by constructing a universal image feature dictionary using a multi-stage approach involving modified spatial pyramid pooling, dimension reduction, and multi-scale feature clustering. For each input image, we learn to synthesize a conditioning latent by selecting and synthesizing relevant features from the dictionary, which significantly enhances the model's capability in capturing and exploring image source correlation. This conditional latent synthesis involves a correlation-based feature matching and alignment strategy, comprising a Conditional Latent Matching (CLM) module and a Conditional Latent Synthesis (CLS) module. The synthesized latent is then used to guide the encoding process, allowing for more efficient compression by exploiting the correlation between the input image and the reference dictionary. According to our theoretical analysis, the proposed conditional latent coding (CLC) method is robust to perturbations in the external dictionary samples and the selected conditioning latent, with an error bound that scales logarithmically with the dictionary size, ensuring stability even with large and diverse dictionaries. Experimental results on benchmark datasets show that our new method improves the coding performance by a large margin (up to 1.2 dB) with a very small overhead of approximately 0.5% bits per pixel.

PDF Details DOI

JAIR Journal 2025 Journal Article

EPINN: Enhanced Physics-Informed Neural Network for Solving Continuous Integral Equations

  • Zhiyuan Ren
  • shijie zhou
  • Dong Liu
  • Qihe Liu

Background: Integral equations play a crucial role in modeling complex systems across various scientific disciplines. However, traditional numerical methods and existing physics-informed neural networks (PINNs) face substantial challenges, including the curse of dimensionality, uncontrolled error propagation, and limited generalization capabilities. Objectives: This paper aims to overcome these limitations by developing a robust and scalable solver for high-dimensional and nonlinear integral equations. The primary goal is to achieve higher accuracy and efficiency compared to traditional methods and existing deep learning approaches. Methods: We present the enhanced physics-informed neural network (EPINN), a novel framework that incorporates three key innovations: 1) a variable-order operator decomposition theory that transforms integral equations into well-posed differential systems, thereby mitigating error accumulation, 2) a differentiable primal function projection layer that ensures physical consistency within the Sobolev spaces, and 3) a boundary-aware multi-objective training paradigm that improves generalization. Results: Experimental validation across five benchmark cases spanning two to four dimensions, including linear/nonlinear Volterra/Fredholm and hybrid Volterra-Fredholm integral equations, demonstrates the superior performance of EPINN. Compared with traditional methods, EPINN reduces relative errors by 1 to 2 orders of magnitude, while achieving over 92% accuracy with limited training data. When compared with existing deep learning solvers, EPINN provides significant improvements in computational efficiency (with a speedup factor of 3 to 6 times) and accuracy (error reduction of 23% to 85%). Conclusions: These advancements establish EPINN as a robust and scalable solver for high-dimensional and nonlinear integral equations, with wide-ranging applications in computational physics and engineering. The success of EPINN suggests that integrating physical principles with neural networks can lead to substantial improvements in solving complex mathematical problems.

PDF Details DOI

AAAI Conference 2025 Conference Paper

Exploiting Diffusion Prior for Real-World Image Dehazing with Unpaired Training

  • Yunwei Lan
  • Zhigao Cui
  • Chang Liu
  • Jialun Peng
  • Nian Wang
  • Xin Luo
  • Dong Liu

Unpaired training has been verified as one of the most effective paradigms for real scene dehazing by learning from unpaired real-world hazy and clear images. Although numerous studies have been proposed, current methods demonstrate limited generalization for various real scenes due to limited feature representation and insufficient use of real-world prior. Inspired by the strong generative capabilities of diffusion models in producing both hazy and clear images, we exploit diffusion prior for real-world image dehazing, and propose an unpaired framework named Diff-Dehazer. Specifically, we leverage diffusion prior as bijective mapping learners within the CycleGAN, a classic unpaired learning framework. Considering that physical priors contain pivotal statistics information of real-world data, we further excavate real-world knowledge by integrating physical priors into our framework. Furthermore, we introduce a new perspective for adequately leveraging the representation ability of diffusion models by removing degradation in image and text modalities, so as to improve the dehazing effect. Extensive experiments on multiple real-world datasets demonstrate the superior performance of our method.

PDF Details DOI

AAAI Conference 2025 Conference Paper

Few-Shot Domain Adaptation for Learned Image Compression

  • Tianyu Zhang
  • Haotian Zhang
  • Yuqi Li
  • Li Li
  • Dong Liu

Learned image compression (LIC) has achieved state-of-the-art rate-distortion performance, deemed promising for next-generation image compression techniques. However, pre-trained LIC models usually suffer from significant performance degradation when applied to out-of-training-domain images, implying their poor generalization capabilities. To tackle this problem, we propose a few-shot domain adaptation method for LIC by integrating plug-and-play adapters into pre-trained models. Drawing inspiration from the analogy between latent channels and frequency components, we examine domain gaps in LIC and observe that out-of-training-domain images disrupt pre-trained channel-wise decomposition. Consequently, we introduce a method for channel-wise re-allocation using convolution-based adapters and low-rank adapters, which are lightweight and compatible to mainstream LIC schemes. Extensive experiments across multiple domains and multiple representative LIC schemes demonstrate that our method significantly enhances pre-trained models, achieving comparable performance to H.266/VVC intra coding with merely 25 target-domain samples. Additionally, our method matches the performance of full-model finetune while transmitting fewer than 2% of the parameters.

PDF Details DOI

IROS Conference 2025 Conference Paper

MK-Pose: Category-Level Object Pose Estimation via Multimodal-Based Keypoint Learning

  • Yifan Yang
  • Peili Song
  • Enfan Lan
  • Dong Liu
  • Jingtai Liu

Category-level object pose estimation, which predicts the pose of objects within a known category without prior knowledge of individual instances, is essential in applications like warehouse automation and manufacturing. Existing methods relying on RGB images or point cloud data often struggle with object occlusion and generalization across different instances and categories. This paper proposes a multimodal-based keypoint learning framework (MK-Pose) that integrates RGB images, point clouds, and category-level textual descriptions. The model uses a self-supervised keypoint detection module enhanced with attention- based query generation, soft heatmap matching and graph-based relational modeling. Additionally, a graph-enhanced feature fusion module is designed to integrate local geometric information and global context. MK-Pose is evaluated on CAMERA25 and REAL275 dataset, and is further tested for cross-dataset capability on HouseCat6D dataset. The results demonstrate that MK-Pose outperforms existing state-of-the-art methods in both IoU and average precision without shape priors. Codes will be released at https://github.com/yangyifanYYF/MK-Pose.

Details

AAMAS Conference 2025 Conference Paper

OGS-SLAM: Hybrid ORB-Gaussian Splatting SLAM

  • Xiaohan Li
  • Wenxiang Shen
  • Dong Liu
  • Jun Wu

Traditional visual SLAM systems (dense and sparse) focus on building metric maps, but the internal representations are misaligned with human vision, making it insufficient for assisting robots in scene perception and interpretation. Conversely, aligning robot scene representation with human vision enables more intuitive human-to-robot commands and improves the generalization capability of deployed neural networks trained on natural images. Neural scene representation-based visual SLAM system, with its consistent and high-fidelity mapping, provides a novel way to assist robots in detailed scene depiction and comprehensive perception. However, end-to-end methods suffer from low accuracy in robot localization, which inevitably degrades mapping quality and limits their practical applications. In this paper, we propose a robust hybrid SLAM system, named OGS-SLAM, which integrates traditional visual SLAM with 3D Gaussian Splatting (3D GS) mapping. This system inherits the high localization accuracy of traditional SLAM while providing a scene model that aligns with human cognition, thereby offering a reliable foundation for downstream human-robot interaction tasks. Experiments demonstrate that our method outperforms state-of-the-art (SOTA) end-to-end SLAM systems in localization, mapping, and map semantic segmentation. Code will be available at: https: //github. com/realXiaohan/OGS-SLAM.

PDF

YNIMG Journal 2025 Journal Article

Potential separation of multiple system atrophy and Parkinson’s disease by susceptibility-derived components

  • Su Yan
  • Jun Lu
  • Bingfang Duan
  • Shun Zhang
  • Dong Liu
  • Yuanyuan Qin
  • Alexey V. Dimov
  • Junghun Cho

BACKGROUND: Substantial evidence emphasizes the dysregulation of iron homeostasis, demyelination and oxidative stress in the neurodegenerative process of multiple system atrophy (MSA) and Parkinson's disease (PD), although its clinical implications remain unclear. Recent MRI post-processing techniques leveraging magnetic susceptibility properties provide a noninvasive means to characterize iron, myelin content and oxygen metabolism alterations. This study aims to investigate subcortical alterations of susceptibility-derived metrics in these two synucleinopathies. METHODS: A cohort comprising 180 patients (122 with PD and 58 with MSA) and 77 healthy controls (HCs) underwent clinical evaluation and multi-echo gradient echo MRI scans. Susceptibility source separation, susceptibility-based oxygen extraction fraction (OEF) mapping and semiautomatic subcortical nuclei segmentation were utilized to derive parametric values of deep gray matter in all subjects. RESULTS: MSA patients showed markedly elevated paramagnetic susceptibility values in the putamen, globus pallidus (GP) and thalamus; increased diamagnetic susceptibility values in the putamen and dentate nucleus; and reduced OEF values across all nuclei compared with PD patients and HCs. Whereas PD exhibited increased positive susceptibility values in the substantia nigra and enhancing negative values in the GP, similar to MSA. Notably, age-related reductions in OEF were evident in HCs, which was altered by the MSA pathology. Paramagnetic susceptibility was correlated with disease severity. Moreover, the susceptibility-derived metrics of striatum and midbrain nuclei proved to be effective predictors to distinguish PD from MSA (AUC = 0.833). CONCLUSION: Susceptibility-derived metrics could detect pathological involvement distinct to each disease, offering significant potential for differentiating between MSA and PD in clinical settings.

Details DOI

EAAI Journal 2025 Journal Article

Research, application, and challenges of causal inference in industrial fault diagnosis: A survey

  • Bo Li
  • Qiang Li
  • Tingfeng Du
  • Dong Liu
  • Qiang Yang
  • Tianxiang Chen
  • Jing Xiong
  • Bo Peng

Industrial fault diagnosis technologies leveraging convolutional neural networks and other advanced neural network architectures are pivotal for ensuring stable equipment operation, enhancing production efficiency, and minimizing maintenance costs. Nevertheless, these methods encounter inherent challenges due to data constraints and the complexity of production environments, particularly in identifying fault root causes and ensuring the interpretability of models. The integration of causal inference into industrial fault diagnosis offers significant promise for elucidating fault propagation pathways, revealing causal interrelations within complex systems, and advancing model interpretability. This survey presents a holistic review of research trajectories, pivotal technologies, and methodological advancements in causal inference for industrial fault diagnosis while systematically delineating the advantages and prospective challenges in this domain. First, this paper examines the limitations of conventional machine-learning approaches in fault diagnosis and traces the evolutionary trajectory of causal inference development in this context. Subsequently, the core theories and foundational technologies underpinning causal inference in industrial fault diagnosis are comprehensively discussed. Following this, the survey categorizes the existing literature according to different causal inferences to solve specific problems in industrial fault diagnosis and delves into detailed case studies, underscoring their utility in addressing distinct challenges. Finally, this survey synthesizes insights from existing literature to encapsulate the merits of causal inference in industrial fault diagnosis and to elucidate the prospective challenges it may encounter.

Details DOI

ICML Conference 2025 Conference Paper

Widening the Network Mitigates the Impact of Data Heterogeneity on FedAvg

  • Like Jian
  • Dong Liu

Federated learning (FL) enables decentralized clients to train a model collaboratively without sharing local data. A key distinction between FL and centralized learning is that clients’ data are non-independent and identically distributed, which poses significant challenges in training a global model that generalizes well across heterogeneous local data distributions. In this paper, we analyze the convergence of overparameterized FedAvg with gradient descent (GD). We prove that the impact of data heterogeneity diminishes as the width of neural networks increases, ultimately vanishing when the width approaches infinity. In the infinite-width regime, we further prove that both the global and local models in FedAvg behave as linear models, and that FedAvg achieves the same generalization performance as centralized learning with the same number of GD iterations. Extensive experiments validate our theoretical findings across various network architectures, loss functions, and optimization methods.

Details

AAAI Conference 2024 Conference Paper

Arbitrary-Scale Video Super-resolution Guided by Dynamic Context

  • Cong Huang
  • Jiahao Li
  • Lei Chu
  • Dong Liu
  • Yan Lu

We propose a Dynamic Context-Guided Upsampling (DCGU) module for video super-resolution (VSR) that leverages temporal context guidance to achieve efficient and effective arbitrary-scale VSR. While most VSR research focuses on backbone design, the importance of the upsampling part is often overlooked. Existing methods rely on pixelshuffle-based upsampling, which has limited capabilities in handling arbitrary upsampling scales. Recent attempts to replace pixelshuffle-based modules with implicit neural function-based and filter-based approaches suffer from slow inference speeds and limited representation capacity, respectively. To overcome these limitations, our DCGU module predicts non-local sampling locations and content-dependent filter weights, enabling efficient and effective arbitrary-scale VSR. Our proposed multi-granularity location search module efficiently identifies non-local sampling locations across the entire low-resolution grid, and the temporal bilateral filter modulation module integrates content information with the filter weight to enhance textual details. Extensive experiments demonstrate the superiority of our method in terms of performance and speed on arbitrary-scale VSR.

PDF Details DOI

AAAI Conference 2024 Conference Paper

CTO-SLAM: Contour Tracking for Object-Level Robust 4D SLAM

  • Xiaohan Li
  • Dong Liu
  • Jun Wu

The demand for 4D ( 3D+time ) SLAM system is increasingly urgent, especially for decision-making and scene understanding. However, most of the existing simultaneous localization and mapping ( SLAM ) systems primarily assume static environments. They fail to represent dynamic scenarios due to the challenge of establishing robust long-term spatiotemporal associations in dynamic object tracking. We address this limitation and propose CTO-SLAM, a monocular and RGB-D object-level 4D SLAM system to track moving objects and estimate their motion simultaneously. In this paper, we propose contour tracking, which introduces contour features to enhance the keypoint representation of dynamic objects and coupled with pixel tracking to achieve long-term robust object tracking. Based on contour tracking, we propose a novel sampling-based object pose initialization algorithm and the following adapted bundle adjustment ( BA ) optimization algorithm to estimate dynamic object poses with high accuracy. The CTO-SLAM system is verified on both KITTI and VKITTI datasets. The experimental results demonstrate that our system effectively addresses cumulative errors in long-term spatiotemporal association and hence obtains substantial improvements over the state-of-the-art systems. The source code is available at https://github.com/realXiaohan/CTO-SLAM.

PDF Details DOI

AAAI Conference 2024 Conference Paper

Generalizable Fourier Augmentation for Unsupervised Video Object Segmentation

  • Huihui Song
  • Tiankang Su
  • Yuhui Zheng
  • Kaihua Zhang
  • Bo Liu
  • Dong Liu

The performance of existing unsupervised video object segmentation methods typically suffers from severe performance degradation on test videos when tested in out-of-distribution scenarios. The primary reason is that the test data in real- world may not follow the independent and identically distribution (i.i.d.) assumption, leading to domain shift. In this paper, we propose a generalizable fourier augmentation method during training to improve the generalization ability of the model. To achieve this, we perform Fast Fourier Transform (FFT) over the intermediate spatial domain features in each layer to yield corresponding frequency representations, including amplitude components (encoding scene-aware styles such as texture, color, contrast of the scene) and phase components (encoding rich semantics). We produce a variety of style features via Gaussian sampling to augment the training data, thereby improving the generalization capability of the model. To further improve the cross-domain generalization performance of the model, we design a phase feature update strategy via exponential moving average using phase features from past frames in an online update manner, which could help the model to learn cross-domain-invariant features. Extensive experiments show that our proposed method achieves the state-of-the-art performance on popular benchmarks.

PDF Details DOI

EAAI Journal 2024 Journal Article

HB-RRT:A path planning algorithm for mobile robots using Halton sequence-based rapidly-exploring random tree

  • Huageng Zhong
  • Ming Cong
  • Minghao Wang
  • Yu Du
  • Dong Liu

Path planning remains crucial for efficient robot operation. A Halton Biased Rapidly-exploring Random Tree (HB-RRT) path planning algorithm is introduced in this study. The Halton sequence, known for its uniform distribution and low discrepancy, is employed for sampling. Issues arising from the pseudo-random sequence in the standard RRT algorithm, leading to uneven distribution of sampling points, are addressed. A mouse-inspired goal-oriented strategy and a candidate sampling pool strategy are incorporated to enhance the sampling point quality, thereby addressing the challenge of insufficient memory during node expansion. Path optimization is further achieved through a multi-level planning approach, which aims to minimize redundancy. A subsequent smoothing of the path is conducted using a cubic B-spline method. Comparisons with the RRT, Bionic Target Bias-RRT, and Informed-RRT* algorithms, through both numerical simulations and real-world testing, confirm the superiority of the HB-RRT algorithm in terms of planning time, path length, and overall path quality.

Details DOI

ICRA Conference 2024 Conference Paper

Language-Conditioned Robotic Manipulation with Fast and Slow Thinking

  • Minjie Zhu
  • Yichen Zhu 0001
  • Jinming Li
  • Junjie Wen
  • Zhiyuan Xu
  • Zhengping Che
  • Chaomin Shen 0001
  • Yaxin Peng

The language-conditioned robotic manipulation aims to transfer natural language instructions into executable actions, from simple "pick-and-place" to tasks requiring intent recognition and visual reasoning. Inspired by the dual-process theory in cognitive science—which suggests two parallel systems of fast and slow thinking in human decision-making—we introduce Robotics with Fast and Slow Thinking (RFST), a framework that mimics human cognitive architecture to classify tasks and makes decisions on two systems based on instruction types. Our RFST consists of two key components: 1) an instruction discriminator to determine which system should be activated based on the current user’s instruction, and 2) a slow-thinking system that is comprised of a fine-tuned vision-language model aligned with the policy networks, which allow the robot to recognize user’s intention or perform reasoning tasks. To assess our methodology, we built a dataset featuring real-world trajectories, capturing actions ranging from spontaneous impulses to tasks requiring deliberate contemplation. Our results, both in simulation and real-world scenarios, confirm that our approach adeptly manages intricate tasks that demand intent recognition and reasoning.

Details

EAAI Journal 2024 Journal Article

Leveraging Contrastive Language–Image Pre-Training and Bidirectional Cross-attention for Multimodal Keyword Spotting

  • Dong Liu
  • Qirong Mao
  • Lijian Gao
  • Gang Wang

In resource-limited keyword spotting scenarios, the scarcity of annotated corpora hinders deep learning’s ability to develop robust models for representing acoustic features. Recent studies focus on contrastive learning, using paired examples for self-supervision, indicating a growing research interest in keyword spotting. Typically, models are trained on segmented spoken words. However, the absence of distinct word boundaries in lengthy audio poses significant challenges. This study presents a novel approach that integrates Contrastive Language–Image Pre-Training with Cross-Attention for Self-Supervised Alignment in the field of Multimodal Keyword Spotting. The proposed method introduces a cross-modal process for matching word pairs, enhancing collaborative effectiveness between audio and text embeddings, achieving word-level congruence across different modalities. Employing a self-supervised learning approach, we use a restricted number of annotated audio–text pairs to discern semantic congruities and divergences, resulting in improved multimodal feature representation. Using this strategy, we input into subsequent keyword detection tasks, adopting a Bidirectional Cross-Attention block for detailed coordination across modalities, bridging the semantic gap between diverse audio and text representations, consequently improving keyword localization precision. Comprehensive experiments conducted on the Aishell-2 and Librispeech datasets demonstrate that the proposed method markedly surpasses existing techniques, especially regarding the comprehensive performance metric F1 for keyword spotting. It attains state-of-the-art outcomes for both pristine and noisy speech, showing respective enhancements of 5. 2% and 5. 4% on Aishell-2, as well as 5. 4% and 5. 6% on Librispeech.

Details DOI

ICRA Conference 2024 Conference Paper

Object-Centric Instruction Augmentation for Robotic Manipulation

  • Junjie Wen
  • Yichen Zhu 0001
  • Minjie Zhu
  • Jinming Li
  • Zhiyuan Xu
  • Zhengping Che
  • Chaomin Shen 0001
  • Yaxin Peng

Humans interpret scenes by recognizing both the identities and positions of objects in their observations. For a robot to perform tasks such as "pick and place", understanding both what the objects are and where they are located is crucial. While the former has been extensively discussed in the literature that uses the large language model to enrich the text descriptions, the latter remains underexplored. In this work, we introduce the Object-Centric Instruction Augmentation (OCI) framework to augment highly semantic and information-dense language instruction with position cues. We utilize a Multi-modal Large Language Model (MLLM) to weave knowledge of object locations into natural language instruction, thus aiding the policy network in mastering actions for versatile manipulation. Additionally, we present a feature reuse mechanism to integrate the vision-language features from off-the-shelf pre-trained MLLM into policy networks. Through a series of simulated and real-world robotic tasks, we demonstrate that robotic manipulator imitation policies trained with our enhanced instructions outperform those relying solely on traditional language instructions.

Details

AAAI Conference 2024 Conference Paper

Offline and Online Optical Flow Enhancement for Deep Video Compression

  • Chuanbo Tang
  • Xihua Sheng
  • Zhuoyuan Li
  • Haotian Zhang
  • Li Li
  • Dong Liu

Video compression relies heavily on exploiting the temporal redundancy between video frames, which is usually achieved by estimating and using the motion information. The motion information is represented as optical flows in most of the existing deep video compression networks. Indeed, these networks often adopt pre-trained optical flow estimation networks for motion estimation. The optical flows, however, may be less suitable for video compression due to the following two factors. First, the optical flow estimation networks were trained to perform inter-frame prediction as accurately as possible, but the optical flows themselves may cost too many bits to encode. Second, the optical flow estimation networks were trained on synthetic data, and may not generalize well enough to real-world videos. We address the twofold limitations by enhancing the optical flows in two stages: offline and online. In the offline stage, we fine-tune a trained optical flow estimation network with the motion information provided by a traditional (non-deep) video compression scheme, e.g. H.266/VVC, as we believe the motion information of H.266/VVC achieves a better rate-distortion trade-off. In the online stage, we further optimize the latent features of the optical flows with a gradient descent-based algorithm for the video to be compressed, so as to enhance the adaptivity of the optical flows. We conduct experiments on two state-of-the-art deep video compression schemes, DCVC and DCVC-DC. Experimental results demonstrate that the proposed offline and online enhancement together achieves on average 13.4% bitrate saving for DCVC and 4.1% bitrate saving for DCVC-DC on the tested videos, without increasing the model or computational complexity of the decoder side.

PDF Details DOI

EAAI Journal 2023 Journal Article

Self-supervised temporal autoencoder for egocentric action segmentation

  • Mingming Zhang
  • Dong Liu
  • Shizhe Hu
  • Xiaoqiang Yan
  • Zhongchuan Sun
  • Yangdong Ye

Given an egocentric video, action temporal segmentation aims to temporally segment the video into basic units, each depicting an action. As the camera is constantly moving, some important objects may disappear in some consecutive frames and cause an abrupt change in the visual content. Recently works fail to deal with this condition in the absence of manually annotating abundant frames. In this study, we propose a temporal-aware clustering method for egocentric action temporal segmentation: a self-supervised temporal autoencoder (SSTAE). Instead of directly learning visual features, the SSTAE is implemented by encoding the preceding target frame and predicting subsequent frames in the temporal relationship domain, which takes into account the local temporal consistency. Our proposed algorithm is guided by the reconstruction and predicted losses. Consequently, local temporal contexts are naturally integrated into the feature representation, and a clustering step is performed. Experiments on three egocentric datasets demonstrate the our proposed approach outperforms the state-of-the-art methods by clustering Accuracy(ACC) 7. 57%, Normalized Mutual Information(NMI) 8. 17%, Adjusted Rand Index(ARI) 8. 6%.

Details DOI

TCS Journal 2022 Journal Article

Component edge connectivity of hypercube-like networks

  • Dong Liu
  • Pingshan Li
  • Bicheng Zhang

As a generalization of the traditional connectivity, the g-component edge connectivity c λ g ( G ) of a non-complete graph G is the minimum number of edges to be deleted from the graph G such that the resulting graph has at least g components. Hypercube-like networks (HL-networks for short) are obtained by manipulating some pairs of edges in hypercubes, which contain several famous interconnection networks such as twisted cubes, Möbius cubes, crossed cubes, locally twisted cubes. In this paper, we determine the ( g + 1 ) -component edge connectivity of the n-dimensional HL-networks for g ≤ 2 ⌈ n 2 ⌉, n ≥ 8.

Details DOI

IROS Conference 2022 Conference Paper

Extrinsic Calibration of a 2D Laser Rangefinder and a Depth-camera Using an Orthogonal Trihedron

  • Zhengbin Li
  • Haiqing Dong
  • Dong Liu
  • Yabin Ding

2D laser range-finders and depth-cameras are usually equipped on service robots. But there are rarely calibration methods of them. This paper proposes an extrinsic calibration method of a 2D laser range-finder and a depth-camera using an orthogonal trihedron. The trihedron with orthogonal assumptions is taken as a reference frame to roughly estimate the relative pose between the sensors by solving a perspective-three-point (P3P) problem and basis-to-basis correspondence. Then, the estimated relative pose is refined via non-linear optimization based on line-to-plane constraints. Unlike other works which require enough motion, only one-shot observation is required, and it is insensitive to sensor ranging noise and the manufacturing errors of calibration targets. Verified by simulation and real experiments, the proposed method is simple, effective and accurate.

Details

JBHI Journal 2020 Journal Article

Deep Learning-Based Classification of Liver Cancer Histopathology Images Using Only Global Labels

  • Chunli Sun
  • Ao Xu
  • Dong Liu
  • Zhiwei Xiong
  • Feng Zhao
  • Weiping Ding

Liver cancer is a leading cause of cancer deaths worldwide due to its high morbidity and mortality. Histopathological image analysis (HIA) is a crucial step in the early diagnosis of liver cancer and is routinely performed manually. However, this process is time-consuming, error-prone, and easily affected by the expertise of pathologists. Recently, computer-aided methods have been widely applied to medical image analysis; however, the current medical image analysis studies have not yet focused on the histopathological morphology of liver cancer due to its complex features and the insufficiency of training images with detailed annotations. This paper proposes a deep learning method for liver cancer histopathological image classification using only global labels. To compensate for the lack of detailed cancer region annotations in those images, patch features are extracted and fully utilized. Transfer learning is used to obtain the patch-level features and then combined with multiple-instance learning to acquire the image-level features for classification. The method proposed here solves the processing of large-scale images and training sample insufficiency in liver cancer histopathological images for image classification. The proposed method can distinguish and classify liver histopathological images as abnormal or normal with high accuracy, thus providing support for the early diagnosis of liver cancer.

Details DOI

AAAI Conference 2020 Conference Paper

Multi-Question Learning for Visual Question Answering

  • Chenyi Lei
  • Lei Wu
  • Dong Liu
  • Zhao Li
  • Guoxin Wang
  • Haihong Tang
  • Houqiang Li

Visual Question Answering (VQA) raises a great challenge for computer vision and natural language processing communities. Most of the existing approaches consider videoquestion pairs individually during training. However, we observe that there are usually multiple (either sequentially generated or not) questions for the target video in a VQA task, and the questions themselves have abundant semantic relations. To explore these relations, we propose a new paradigm for VQA termed Multi-Question Learning (MQL). Inspired by the multi-task learning, MQL learns from multiple questions jointly together with their corresponding answers for a target video sequence. The learned representations of videoquestion pairs are then more general to be transferred for new questions. We further propose an effective VQA framework and design a training procedure for MQL, where the specifically designed attention network models the relation between input video and corresponding questions, enabling multiple video-question pairs to be co-trained. Experimental results on public datasets show the favorable performance of the proposed MQL-VQA framework compared to state-of-the-arts.

PDF Details

JBHI Journal 2019 Journal Article

Automated Layer Segmentation of Retinal Optical Coherence Tomography Images Using a Deep Feature Enhanced Structured Random Forests Classifier

  • Xiaoming Liu
  • Tianyu Fu
  • Zhifang Pan
  • Dong Liu
  • Wei Hu
  • Jun Liu
  • Kai Zhang

Optical coherence tomography (OCT) is a high-resolution and noninvasive imaging modality that has become one of the most prevalent techniques for ophthalmic diagnosis. Retinal layer segmentation is very crucial for doctors to diagnose and study retinal diseases. However, manual segmentation is often a time-consuming and subjective process. In this work, we propose a new method for automatically segmenting retinal OCT images, which integrates deep features and hand-designed features to train a structured random forests classifier. The deep convolutional features are learned from deep residual network. With the trained classifier, we can get the contour probability graph of each layer; finally, the shortest path is employed to achieve the final layer segmentation. The experimental results show that our method achieves good results with the mean layer contour error of 1. 215 pixels, whereas that of the state of the art was 1. 464 pixels, and achieves an F1-score of 0. 885, which is also better than 0. 863 that is obtained by the state of the art method.

Details DOI

NeurIPS Conference 2019 Conference Paper

On The Classification-Distortion-Perception Tradeoff

  • Dong Liu
  • Haochen Zhang
  • Zhiwei Xiong

Signal degradation is ubiquitous, and computational restoration of degraded signal has been investigated for many years. Recently, it is reported that the capability of signal restoration is fundamentally limited by the so-called perception-distortion tradeoff, i. e. the distortion and the perceptual difference between the restored signal and the ideal "original" signal cannot be made both minimal simultaneously. Distortion corresponds to signal fidelity and perceptual difference corresponds to perceptual naturalness, both of which are important metrics in practice. Besides, there is another dimension worthy of consideration--the semantic quality of the restored signal, i. e. the utility of the signal for recognition purpose. In this paper, we extend the previous perception-distortion tradeoff to the case of classification-distortion-perception (CDP) tradeoff, where we introduced the classification error rate of the restored signal in addition to distortion and perceptual difference. In particular, we consider the classification error rate achieved on the restored signal using a predefined classifier as a representative metric for semantic quality. We rigorously prove the existence of the CDP tradeoff, i. e. the distortion, perceptual difference, and classification error rate cannot be made all minimal simultaneously. We also provide both simulation and experimental results to showcase the CDP tradeoff. Our findings can be useful especially for computer vision research where some low-level vision tasks (signal restoration) serve for high-level vision tasks (visual understanding). Our code and models have been published.

PDF Details

AAAI Conference 2019 Conference Paper

Robust Deep Co-Saliency Detection with Group Semantic

  • Chong Wang
  • Zheng-Jun Zha
  • Dong Liu
  • Hongtao Xie

High-level semantic knowledge in addition to low-level visual cues is essentially crucial for co-saliency detection. This paper proposes a novel end-to-end deep learning approach for robust co-saliency detection by simultaneously learning highlevel group-wise semantic representation as well as deep visual features of a given image group. The inter-image interaction at semantic-level as well as the complementarity between group semantics and visual features are exploited to boost the inferring of co-salient regions. Specifically, the proposed approach consists of a co-category learning branch and a co-saliency detection branch. While the former is proposed to learn group-wise semantic vector using co-category association of an image group as supervision, the latter is to infer precise co-salient maps based on the ensemble of group semantic knowledge and deep visual cues. The group semantic vector is broadcasted to each spatial location of multi-scale visual feature maps and is used as a top-down semantic guidance for boosting the bottom-up inferring of co-saliency. The co-category learning and co-saliency detection branches are jointly optimized in a multi-task learning manner, further improving the robustness of the approach. Moreover, we construct a new large-scale co-saliency dataset COCO-SEG to facilitate research of co-saliency detection. Extensive experimental results on COCO-SEG and a widely used benchmark Cosal2015 have demonstrated the superiority of the proposed approach as compared to the state-of-the-art methods.

PDF Details
v2026.09.13