Arrow Research search

Author name cluster

Bo Liu

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

100 papers
2 author rows

Possible papers

100

AAAI Conference 2026 Conference Paper

CausalCLIP: Causally-Informed Feature Disentanglement and Filtering for Generalizable Detection of Generated Images

  • Bo Liu
  • Qiao Qin
  • Qinghui He

The rapid advancement of generative models has increased the demand for generated image detectors capable of generalizing across diverse and evolving generation techniques. However, existing methods, including those leveraging pre-trained vision-language models, often produce highly entangled representations, mixing task-relevant forensic cues (causal features) with spurious or irrelevant patterns (non-causal features), thus limiting generalization. To address this issue, we propose CausalCLIP, a framework that explicitly disentangles causal from non-causal features and employs targeted filtering guided by causal inference principles to retain only the most transferable and discriminative forensic cues. By modeling the generation process with a structural causal model and enforcing statistical independence through Gumbel-Softmax-based feature masking and Hilbert-Schmidt Independence Criterion (HSIC) constraints, CausalCLIP isolates stable causal features robust to distribution shifts. When tested on unseen generative models from different series, CausalCLIP demonstrates strong generalization ability, achieving improvements of 6.83% in accuracy and 4.06% in average precision over state-of-the-art methods.

PDF Details DOI

AAAI Conference 2026 Conference Paper

DPRM: A Dual Implicit Process Reward Model in Multi-Hop Question Answering

  • Xinyi Wang
  • Yiping Song
  • Zhiliang Tian
  • Bo Liu
  • Tingjin Luo
  • Minlie Huang

In multi-hop question answering (MHQA) tasks, Chain of Thought (CoT) improves the quality of generation by guiding large language models (LLMs) through multi-step reasoning, and Knowledge Graphs (KGs) reduce hallucinations via semantic matching. Outcome Reward Models (ORMs) provide feedback after generating the final answers but fail to evaluate the process for multi-step reasoning. Traditional Process Reward Models (PRMs) evaluate the reasoning process but require costly human annotations or rollout generation. While implicit PRM is trained only with outcome signals and derives step rewards through reward parameterization without explicit annotations, it is more suitable for multi-step reasoning in MHQA tasks. However, existing implicit PRM has only been explored for plain text scenarios. When adapting to MHQA tasks, it cannot handle the graph structure constraints in KGs and capture the potential inconsistency between CoT and KG paths. To address these limitations, we propose the DPRM (Dual Implicit Process Reward Model). It trains two implicit PRMs for CoT and KG reasoning in MHQA tasks. Both PRMs, namely KG-PRM and CoT-PRM, derive step-level rewards from outcome signals via reward parameterization without additional explicit annotations. Among them, KG-PRM uses preference pairs to learn structural constraints from KGs. DPRM further introduces a consistency constraint between CoT and KG reasoning steps, making the two PRMs mutually verify and collaboratively optimize the reasoning paths. We also provide a theoretical demonstration of the derivation of process rewards. Experimental results show that our method outperforms 13 baselines on multiple datasets with up to 16.6% improvement on Hit@1.

PDF Details DOI

JBHI Journal 2026 Journal Article

DTQFL: A Digital Twin-Assisted Quantum Federated Learning Algorithm for Intelligent Diagnosis in 5G Mobile Network

  • Zhiguo Qu
  • Yang Li
  • Bo Liu
  • Deepak Gupta
  • Prayag Tiwari

Smart healthcare aims to revolutionize medical services by integrating artificial intelligence (AI). The limitations of classical machine learning include privacy concerns that prevent direct data sharing among medical institutions, untimely updates, and long training times. To address these issues, this study proposes a digital twin-assisted quantum federated learning algorithm (DTQFL). By leveraging the 5G mobile network, digital twins (DT) of patients can be created instantly using data from various Internet of Medical Things (IoMT) devices and simultaneously reduce communication time in federated learning (FL) at the same time. DTQFL generates DT for patients with specific diseases, allowing for synchronous training and updating of the variational quantum neural network (VQNN) without disrupting the VQNN in the real world. This study utilized DTQFL to train its own personalized VQNN for each hospital, considering privacy security and training speed. Simultaneously, the personalized VQNN of each hospital was obtained through further local iterations of the final global parameters. The results indicate that DTQFL can train a good VQNN without collecting local data while achieving accuracy comparable to that of data-centralized algorithms. In addition, after personalized training, the VQNN can achieve higher accuracy than that without personalized training.

Details DOI

EAAI Journal 2026 Journal Article

Ensemble modeling via entropy weight method and technique for order preference by similarity to an ideal solution for powder factor optimization toward targeted blast fragmentation

  • Weizhong Chen
  • Bo Liu
  • Xianyang Qiu
  • Wenbo Shen
  • Hongjie Qiu
  • Xiuzhi Shi

Accurate prediction of the powder factor (Pf) is crucial for optimizing blasting efficiency and cost in open-pit mining. To overcome the limitations of single-model approaches—such as poor stability and low interpretability—this study, utilizing 161 field datasets from the Mirador Copper Mine in Ecuador, innovatively integrates the Entropy Weight Method with the Technique for Order Preference by Similarity to an Ideal Solution (TOPSIS) to construct an objective weighted fusion framework for seven machine learning (ML) models. Furthermore, it creatively combines SHapley Additive exPlanations (SHAP) with three-dimensional (3D) partial dependence plots (PDP) to decode the complex nonlinear interaction mechanisms among key features influencing the powder factor. The results show that the proposed model achieved superior performance with a coefficient of determination (R2) of 0. 921, a mean squared error (MSE) of 0. 003, and a mean absolute error (MAE) of 0. 046. SHAP analysis identified the 80% passing fragment size (D80), the burden-to-diameter ratio (B/D), rock density (R0), uniaxial compressive strength (UCS), and elastic modulus (E) as the most influential features, collectively accounting for 78. 65% of the total contribution. Three-dimensional PDP further revealed key nonlinear interactions: such as Pf exceeds 0. 65 when D80 > 0. 6 and B/D > 35, whereas it stabilizes between 0. 35 and 0. 45 when D80 < 0. 4. Field trials confirmed the system's practical applicability, with relative errors of only 3. 1%–5. 24% between target and measured fragmentation. This study offers a transparent, data-driven artificial intelligence (AI)methodology for Pf prediction applicable to geologically complex open-pit mines, enhancing both economic and safety outcomes.

Details DOI

EAAI Journal 2026 Journal Article

Fault prototype weighted guided multi-level dynamic alignment network for partial set domain adaptation fault diagnosis

  • Bo Liu
  • Guofa Li
  • Jialong He
  • Tianzhe Wang
  • Rundong Shi

Partial set domain adaptation for fault diagnosis has garnered significant attention due to its alignment with practical engineering applications. However, current research heavily relies on classifier predictions to evaluate sample transferability, where misclassification may hinder the model's ability to distinguish shared and outlier samples. Moreover, the joint alignment at both instance and class level is often neglected, in complex dynamic application scenarios, the contributions of marginal and conditional probability distributions to positive model transfer differ significantly. To address this, a fault prototype weighted guided multi-level dynamic alignment network (FPW-MDAN) is proposed. To mitigate the biasing effect of classifier predictions on cross-domain transferability evaluation, an inter-domain fault prototype correlation guided weighting strategy is developed. This strategy re-weights both instance and class-level probability weights to enhance the robustness of transferability evaluation and correct the model's identification of shared and outlier samples. Building on this weighting strategy, a multi-level weighted dynamic alignment network is constructed, aiming to align the joint probability distribution of cross-domain shared samples from both global and local perspectives, thereby reducing the influence of outlier samples on the transfer process. In addition, the domain classifier alignment strategy is introduced to ensure the consistency of sub-classifiers' predictions for the same samples from target domain and to improve the prediction accuracy of class boundary samples. Finally, comparative experiments and ablation studies were conducted on both public planetary gearbox and self-made parallel gearbox dataset. The experimental results validate the effectiveness and superiority of the proposed method in addressing partial-set domain adaptation for fault diagnosis.

Details DOI

AAMAS Conference 2026 Conference Paper

From Knowledge to Causality: Self-Supervised Representation Learning for Granger Causal Discovery in Groups of Time Series

  • Bo Liu
  • Hongyan Li
  • Shenda Hong

Causal inference among groups of time series is crucial for understanding complex systems like brain networks and climate dynamics. Existing methods often rely on simple aggregation to represent groups, leading to significant information loss and suboptimal causal discovery. We propose CausalKGR, a novel framework that learns expressive latent representations for Granger causal discovery. By introducing a Knowledge-Conditional Attention mechanism, CausalKGR distills temporal knowledge into a compact latent space, capturing both intra-group dynamics and inter-group interactions. Extensive experiments on synthetic and real-world datasets demonstrate that CausalKGR significantly outperforms state-of-the-art baselines in accuracy and interpretability.

PDF

JBHI Journal 2026 Journal Article

KidMesh: Computational Mesh Reconstruction for Pediatric Congenital Hydronephrosis Using Deep Neural Networks

  • Haoran Sun
  • Zhanpeng Zhu
  • Anguo Zhang
  • Bo Liu
  • Zhaohua Lin
  • Liqin Huang
  • Mingjing Yang
  • Lei Liu

Pediatric congenital hydronephrosis (CH) is a common urinary tract disorder, primarily caused by obstruction at the renal pelvis-ureter junction. Magnetic resonance urography (MRU) can visualize hydronephrosis, including renal pelvis and calyces, by utilizing the natural contrast provided by water. Existing voxel-based segmentation approaches can extract CH regions from MRU, facilitating disease diagnosis and prognosis. However, these segmentation methods predominantly focus on morphological features, such as size, shape, and structure. To enable functional assessments, such as urodynamic simulations, external complex post-processing steps are required to convert these results into mesh-level representations. To address this limitation, we propose an end-to-end method based on deep neural networks, namely KidMesh, which could automatically reconstruct CH meshes directly from MRU. Generally, KidMesh extracts feature maps from MRU images and converts them into feature vertices through grid sampling. It then deforms a template mesh according to these feature vertices to generate the specific CH meshes of MRU images. Meanwhile, we develop a novel schema to train KidMesh without relying on accurate mesh-level annotations, which are difficult to obtain due to the sparsely sampled MRU slices. Experimental results show that KidMesh reconstructs CH meshes in an average of 0. 4 seconds, and achieve comparable performance to conventional methods without requiring post-processing. The reconstructed meshes exhibited no self-intersections, with only 3. 7% and 0. 2% of the vertices having error distances exceeding 3. 2mm and 6. 4mm, respectively. After rasterization, these meshes achieved a Dice score of 0. 86 against manually delineated CH masks. Furthermore, these meshes could be used in renal urine flow simulations, providing valuable urodynamic information for clinical practice.

Details DOI

JBHI Journal 2026 Journal Article

MGTP: Multi-Granularity Textual Prompts for Low-Dose Brain PET Image Denoising via Adversarial Diffusion Model

  • Jiaqi Cui
  • Xinyi Zeng
  • Pinxian Zeng
  • Bo Liu
  • Xi Wu
  • Deng Xiong
  • Jiliu Zhou
  • Yan Wang

Positron emission tomography (PET) is an advanced nuclear imaging technique and has been widely applied in clinic. However, radiation risks associated with standard-dose PET imaging raise health concerns, whereas the quality of low-dose PET images fails to meet clinical requirements. To reduce the tracer dose while maintaining image quality, it is of great interest to estimate high-quality PET images from low-dose images. However, existing low-dose PET image denoising methods primarily focus on image data, overlooking crucial information in non-image textual data such as patients’ clinical tabular and textual descriptions of general image quality. This neglect can lead to subpar denoising quality with inaccurate contexts and poor details. To address these problems, in this paper, we propose Multi-Granularity Textual Prompts, namely MGTP, to denoise low-dose PET images via an adversarial diffusion model. Different from prior methods that rely solely on image conditioning, our MGTP innovatively introduces textual prompts spanning diverse granularities to capture both high-level semantic-related contexts and low-level degradation-related details. To harmonize multi-granularity textual prompts with low-dose PET images, we design a Cross-Modality Selective Conditioning (CMSC) module, which prioritizes semantic- and detail-relevant information while eliminating irrelevant components. The resulting features are fed into diffusion model as conditions, enforcing a more controlled diffusion process. In addition, we develop a Masked Prompt Reconstruction Network (MPR-Net) to enhance the preservation of semantics and details in denoised images, mitigating distortions brought by the random noise in the diffusion process. Experiments on clinical PET data show that our method achieves the state-of-the-art performance.

Details DOI

AAAI Conference 2026 Conference Paper

PromptEmo: Learning Emotion with Bilateral Textual Prompts in Multi-Domain Open-set Scenarios

  • Xinyi Zeng
  • Yuxiang Yang
  • Pinxian Zeng
  • Wenxia Yin
  • Bo Liu
  • Xi Wu
  • Yan Wang

Facial Expression Recognition (FER) is crucial to human-computer interaction. Existing cross-domain FER (CD-FER) methods mainly focus on single-source closed-set scenarios, transferring knowledge from a single source domain to a target domain with identical class sets. However, CD-FER faces two real-world challenges: 1) the need to leverage information from multiple sources, leading to multi-domain shift, and 2) the necessity to recognize unseen target classes, resulting in class shift. These issues give rise to a novel and challenging task, which we define as Multi-domain Open-set FER (MO-FER). In this paper, we propose PromptEmo, a novel CLIP-based framework that leverages bilateral textual prompts to address both shifts in the MO-FER task. Leveraging the generalizability of LLM, PromptEmo constructs trainable positive prompts with LLM-generated emotion descriptions for seen classes, as well as template-derived negative prompts to enhance the reasoning for unseen classes. Then, we introduce a modal-task optimization paradigm organized from two perspectives: textual semantics and visual domains, yielding Intra-modal Space-specific Optimization (ISO) and Cross-modal Emotion-aware Interaction (CEI) strategies. ISO refines the CLIP-based textual space to ensure semantic separation between bilateral prompts and improves the latent visual space by promoting inter-domain alignment. Founded on ISO, CEI facilitates effective vision-language interactions, resulting in four joint loss terms that improve emotion recognition by shaping a domain-invariant, discriminative feature space. PromptEmo surpasses the current SOTA method by 7.7% AUC on unseen classes across four FER datasets, serving as a strong baseline for the MO-FER task.

PDF Details DOI

AAAI Conference 2026 Conference Paper

ProRec-Video: Guiding Hierarchical Interest Transitions for Proactive Short Video Recommendation with Dynamic Feedback Adaptation

  • Weizhi Chen
  • Baoyun Peng
  • Bo Liu
  • Xingkong Ma
  • Houjie Qiu

Traditional short video recommendations primarily enhance user retention by reinforcing existing user preferences, potentially leading to information cocoons. Conversely, proactive recommendations aim to diversify user interests by exposing users to content beyond their historical preferences. However, current proactive approaches face three limitations: (1) homogeneous receptivity assumption, neglecting individual differences in users' openness to new interests; (2) short-term item exposure without interest anchoring, focusing on item-level shifts rather than interest evolution; and (3) static feedback utilization, failing to incorporate dynamic user feedback during the recommendation adequately. To address these challenges, we propose ProRec-Video, a proactive framework that guides hierarchical interest transitions through three innovations. First, User Receptivity Profiling assesses individual openness for new interests, ensuring personalized transition pacing. Second, Hierarchical Interest Transition Planning decomposes complex interest shifts into intermediate steps to generate smooth interest transition paths and semantically coherent video sequences, addressing overemphasis on item exposure. Third, Dynamic Feedback Adaptation integrates agent-based simulation and Reflexion mechanisms to refine interest transition paths and video sequences based on real-time user feedback, enhancing adaptability and satisfaction. Extensive experiments on two datasets demonstrate that ProRec-Video achieves a significant improvement in proactive recommendation performance, with an interest transition success rate of 85% and a user satisfaction rate of 78.3%.

PDF Details DOI

AAMAS Conference 2026 Conference Paper

SAT: Sequential Agent Tuning for Coordinator-Free Plug-and-Play Multi-LLM Training with Monotonic Improvement Guarantees

  • Yi Xie
  • Yangyang Xu
  • Yi Fan
  • Bo Liu

Large language models (LLMs) with a large number of parameters achieve strong performance but are often prohibitively expensive to deploy. Recent work explores using teams of smaller, more efficient LLMs that collectively match or even outperform a single large model. However, jointly updating multiple agents introduces compounding distribution shifts, making coordination and stability during training difficult. We address this by introducing Sequential Agent Tuning (SAT), a coordinator-free training paradigm. SAT represents the team as a factorized policy and employs blockcoordinate updates over agents, enabling scalable, decentralized training without a central controller. Specifically, we develop a sequence-aware, on-policy advantage estimator that conditions on the evolving team policy, coupled with per-agent KL trust regions that isolate occupancy drift. Theoretically, this framework provides two critical guarantees. First, it ensures monotonic improvement, stabilizing the training process. Second, it establishes provable plugand-playinvariance: anyagentcanbeupgradedtoastrongermodel without retraining the rest of the team, with a formal guarantee that the performance bound improves. Empirically, a team of three 4B agents (12B total) trained with SAT surpasses the much larger Qwen3-32B on AIME24/25 benchmarks by 3. 9% on average. We validate our plug-and-play theory by swapping in two 8B agents, which boosts the composite score by 10. 4%. We provide code and appendix of proof at https: //github. com/Yydc/SAT-AAMAS

PDF

AAAI Conference 2026 Conference Paper

Sparse-Scale Transformer with Bidirectional Awareness for Time Series Forecasting

  • Ying Liu
  • Bo Liu
  • Sheng Huang
  • Gang Luo
  • Wenbo Hu
  • Meng Wang
  • Richang Hong

Time series forecasting (TSF) plays a crucial role in many real-world applications, such as weather prediction and economic planning. While Transformer-based models have shown strong capabilities in modeling long-range dependencies, effectively capturing the multi-scale temporal dynamics inherent in time series remains a major challenge. Existing methods often adopt time-windows of varying sizes, which may introduce noisy or irrelevant representations when mismatched with the underlying temporal patterns, potentially leading to overfitting. In this paper, we propose Sparse-Scale Transformer (SSformer) with Bidirectional Awareness for Time Series Forecasting to enhance the multi-scale modeling for time series. Specifically, we propose a novel Sparse-Scale Convolution (SSC) block that imposes sparsity on scales to obtain the informative representations by evaluating the intra-scale segment similarity of time series, and utilizes scale-specific convolutions to extract local patterns. Furthermore, we design a Bidirectional-Scale Interaction (BSI) block to explicitly model scale correlations in both coarse-to-fine and fine-to-coarse directions. Finally, scale predictions are ensembled to fully exploit the complementary forecasting capabilities across scales. Extensive experiments on various real-world datasets demonstrate that SSformer achieves state-of-the-art performance with superior efficiency.

PDF Details DOI

EAAI Journal 2026 Journal Article

Spatial Frequency Guidance Network for real-time lung computed tomography image segmentation

  • Mohammed A.M. Elhassan
  • JiaZhou Xiao
  • Bo Liu
  • Xu Li
  • Chenxi Huang
  • Jinbao Xie
  • Minglian Qiu

With the increasing demand for real-time lung tumor segmentation in clinical settings, there is a pressing need for efficient and accurate segmentation models. However, current state-of-the art methods are often burdened by high computational complexity and struggle to simultaneously capture fine-grained local details and global contextual cues essential for precise tumor delineation. To address these challenges, we present the Spatial Frequency Guidance Network (SFGNet), a lightweight encoder–decoder architecture tailored for real-time lung tumor segmentation. The proposed framework incorporates three key components: (1) a Residual Downsampling Block that reduces spatial resolution while preserving high-frequency structural cues; (2) a Spatial Frequency Guidance Module that integrates spatial and frequency-domain information to enhance multi-scale contextual representation; and (3) a Multi-Scale Feature Interaction Module that adaptively fuses features across resolutions to improve pixel-level classification accuracy. Experiments on two public lung computed tomography datasets demonstrate that the proposed architecture achieves a favorable balance between accuracy and efficiency. For example, the base variant attains best average across all metrics 97. 32% at real-time inference speed, while also producing best performance on a benchmark dataset of coronavirus disease (COVID-19) computed tomography scans. The source code, will be released publicly at: https: //github. com/mohamedac29/SFGNet.

Details DOI

AAAI Conference 2026 Conference Paper

TGDD: Trajectory Guided Dataset Distillation with Balanced Distribution

  • Fengli Ran
  • Xiao Pu
  • Bo Liu
  • Xiuli Bi
  • Bin Xiao

Dataset distillation compresses large datasets into compact synthetic ones to reduce storage and computational costs. Among various approaches, distribution matching (DM)-based methods have attracted attention for their high efficiency. However, they often overlook the evolution of feature representations during training, which limits the expressiveness of synthetic data and weakens downstream performance. To address this issue, we propose Trajectory Guided Dataset Distillation (TGDD), which reformulates distribution matching as a dynamic alignment process along the model’s training trajectory. At each training stage, TGDD captures evolving semantics by aligning the feature distribution between the synthetic and original dataset. Meanwhile, it introduces a distribution constraint regularization to reduce class overlap. This design helps synthetic data preserve both semantic diversity and representativeness, improving performance in downstream tasks. Without additional optimization overhead, TGDD achieves a favorable balance between performance and efficiency. Experiments on ten datasets demonstrate that TGDD achieves state-of-the-art performance, notably a 5.0% accuracy gain on high-resolution benchmarks.

PDF Details DOI

EAAI Journal 2025 Journal Article

A novel multi-fidelity sequential optimization method based on multi-level Gaussian process

  • Zecong Liu
  • Liang Yan
  • Xiaojun Duan
  • Yike Xiao
  • Bo Liu
  • Jiangtao Chen

Multi-fidelity optimization algorithms can efficiently find the optimum of the high-fidelity system with assistance of lower-fidelity and cheaper simulation(s). However, existing methods do not utilize this “assistance” sufficiently, where they generally tend to query high-fidelity systems during the optimization procedure. In this paper, we propose a novel sequential criterion based on multi-level Gaussian process (MLGP) and name it the multi-level expected improvement criterion (LEI). LEI is an extended version of the expected improvement criterion (EI) with a closed form, which integrates the correlation index, cost ratio, and constraint handling terms, hence determining both location and fidelity level of the next sample. Specifically, the correlation index is a function of the prediction error for each fidelity, which can reflect the metamodeling accuracy of low-fidelity and the desirable sampling site for high-fidelity system. We have proved that low-fidelity samples can improve the accuracy of MLGP modeling while the LEI criterion can improve the robustness and accuracy of optimization theoretically. Furthermore, the LEI criterion can easily be extended to multiple fidelity scenarios. The numerical results show that the proposed algorithm saves the sample size for high fidelity with higher optimization efficiency and stronger robustness.

Details DOI

IROS Conference 2025 Conference Paper

Advancing Depth Anything Model for Unsupervised Monocular Depth Estimation in Endoscopy

  • Bojian Li
  • Bo Liu
  • Xinning Yao
  • Jinghua Yue
  • Fugen Zhou

Depth estimation is a cornerstone of 3D reconstruction and plays a vital role in minimally invasive endoscopic surgeries. However, most current depth estimation networks rely on traditional convolutional neural networks, which are limited in their ability to capture global information. Foundation models offer a promising approach to enhance depth estimation, but those models currently available are primarily trained on natural images, leading to suboptimal performance when applied to endoscopic images. In this work, we introduce a novel fine-tuning strategy for the Depth Anything Model and integrate it with an intrinsic-based unsupervised monocular depth estimation framework. Our approach includes a low-rank adaptation technique based on random vectors, which improves the model’s adaptability to different scales. Additionally, we propose a residual block built on depthwise separable convolution to compensate for the transformer’s limited ability to capture local features. Our experimental results on the SCARED dataset and Hamlyn dataset show that our method achieves state-of-the-art performance while minimizing the number of trainable parameters. Applying this method in minimally invasive endoscopic surgery can enhance surgeons’ spatial awareness, thereby improving the precision and safety of the procedures.

Details

EAAI Journal 2025 Journal Article

An adaptive appliance identification method leveraging topological data analysis

  • Yang Han
  • Keke Li
  • Hanju Cai
  • Wenpeng Luan
  • Bochao Zhao
  • Bo Liu

Non-intrusive load monitoring (NILM) is a technique that involves analyzing changes in voltage and current flowing through the main feeder to determine which appliances are in operation and their energy consumption. With the increasing amount and diversity of electric loads nowadays, it is becoming increasingly important to extract unique load signatures and build robust classification models for NILM. Topological data analysis (TDA) studies the properties of space that are preserved under continuous deformations, which can reveal the structure and relationships within complex datasets, such as networks, graphs, and manifolds. In this paper, we use TDA as feature extractor to mine vast of non-linear shape features from Voltage–Current (V-I) trajectory for appliance identification. Then, an adaptive feature selection method based on mutual information (MI) is proposed, as a result, the selected subset of features is more discriminative for the adopted dataset. Further, using selected features, the strategy of Frienemy Indecision Region Dynamic Ensemble Selection (FIRE-DES), which adaptively selects combination of classifiers according to data-samples by considering the improvement by indecision region, is employed for non-intrusive appliance classification. By such fusion of TDA and technique of ensemble learning, the experimental results on two public datasets prove efficacy of proposed method in both identification accuracy and computation time.

Details DOI

RLC Conference 2025 Conference Paper

Benchmarking Massively Parallelized Multi-Task Reinforcement Learning for Robotics Tasks

  • Viraj Joshi
  • Zifan Xu
  • Bo Liu
  • Peter Stone
  • Amy Zhang

Multi-task Reinforcement Learning (MTRL) has emerged as a critical training paradigm for applying reinforcement learning (RL) to a set of complex real-world robotic tasks, which demands a generalizable and robust policy. At the same time, \emph{massively parallelized training} has gained popularity, not only for significantly accelerating data collection through GPU-accelerated simulation but also for enabling diverse data collection across multiple tasks by simulating heterogeneous scenes in parallel. However, existing MTRL research has largely been limited to off-policy methods like SAC in the low-parallelization regime. MTRL could capitalize on the higher asymptotic performance of on-policy algorithms, whose batches require data from the current policy, and as a result, take advantage of massive parallelization offered by GPU-accelerated simulation. To bridge this gap, we introduce a massively parallelized $\textbf{M}$ulti-$\textbf{T}$ask $\textbf{Bench}$mark for robotics (MTBench), an open-sourced benchmark featuring a broad distribution of 50 manipulation tasks and 20 locomotion tasks, implemented using the GPU-accelerated simulator IsaacGym. MTBench also includes four base RL algorithms combined with seven state-of-the-art MTRL algorithms and architectures, providing a unified framework for evaluating their performance. Our extensive experiments highlight the superior speed of evaluating MTRL approaches using MTBench, while also uncovering unique challenges that arise from combining massive parallelism with MTRL. Code is available at $\href{https: //github. com/Viraj-Joshi/MTBench}{ https: //github. com/Viraj-Joshi/MTBench}$

PDF Details

RLJ Journal 2025 Journal Article

Benchmarking Massively Parallelized Multi-Task Reinforcement Learning for Robotics Tasks

  • Viraj Joshi
  • Zifan Xu
  • Bo Liu
  • Peter Stone
  • Amy Zhang

Multi-task Reinforcement Learning (MTRL) has emerged as a critical training paradigm for applying reinforcement learning (RL) to a set of complex real-world robotic tasks, which demands a generalizable and robust policy. At the same time, \emph{massively parallelized training} has gained popularity, not only for significantly accelerating data collection through GPU-accelerated simulation but also for enabling diverse data collection across multiple tasks by simulating heterogeneous scenes in parallel. However, existing MTRL research has largely been limited to off-policy methods like SAC in the low-parallelization regime. MTRL could capitalize on the higher asymptotic performance of on-policy algorithms, whose batches require data from the current policy, and as a result, take advantage of massive parallelization offered by GPU-accelerated simulation. To bridge this gap, we introduce a massively parallelized $\textbf{M}$ulti-$\textbf{T}$ask $\textbf{Bench}$mark for robotics (MTBench), an open-sourced benchmark featuring a broad distribution of 50 manipulation tasks and 20 locomotion tasks, implemented using the GPU-accelerated simulator IsaacGym. MTBench also includes four base RL algorithms combined with seven state-of-the-art MTRL algorithms and architectures, providing a unified framework for evaluating their performance. Our extensive experiments highlight the superior speed of evaluating MTRL approaches using MTBench, while also uncovering unique challenges that arise from combining massive parallelism with MTRL. Code is available at $\href{https://github.com/Viraj-Joshi/MTBench}{ https://github.com/Viraj-Joshi/MTBench}$

PDF Details

AAAI Conference 2025 Conference Paper

CustomTTT: Motion and Appearance Customized Video Generation via Test-Time Training

  • Xiuli Bi
  • Jian Lu
  • Bo Liu
  • Xiaodong Cun
  • Yong Zhang
  • Weisheng Li
  • Bin Xiao

Benefiting from large-scale pre-training of text-video pairs, current text-to-video (T2V) diffusion models can generate high-quality videos from the text description. Besides, given some reference images or videos, the parameter-efficient fine-tuning method, i.e. LoRA, can generate high-quality customized concepts, e.g., the specific subject or the motions from a reference video. However, combining the trained multiple concepts from different references into a single network shows obvious artifacts. To this end, we propose CustomTTT, where we can joint custom the appearance and the motion of the given video easily. In detail, we first analyze the prompt influence in the current video diffusion model and find the LoRAs are only needed for the specific layers for appearance and motion customization. Besides, since each LoRA is trained individually, we propose a novel test-time training technique to update parameters after combination utilizing the trained customized models. We conduct detailed experiments to verify the effectiveness of the proposed methods. Our method outperforms several state-of-the-art works in both qualitative and quantitative evaluations.

PDF Details DOI

EAAI Journal 2025 Journal Article

Data-driven analysis of the process, organization and properties of large-size complex thin-walled die-casting aluminium alloys

  • Jian Yang
  • Bo Liu
  • Dongwei Shu
  • Qin Yang
  • Yunbo Zeng
  • Jun Huang

High pressure die casting technology, as an advanced precision forming process, is widely used in the manufacture of different size parts. Whereas in the production of large-size parts, the spatial heterogeneity of mechanical properties caused by local thermal field inhomogeneity becomes a key technical bottleneck. This paper presents a data-driven framework that effectively leverages the thermal history throughout the casting process, enabling precise predictions of part mechanical properties at various spatial locations. A physical simulation model is used to simulate the casting and molding process of front nacelle, with sensors embedded to provide temperature data at different locations. Subsequently, the effects of process and location on solidification rate, microstructure, and mechanical properties are discussed. Relying solely on the solidification rate to predict mechanical properties is inadequate, necessitating the consideration of additional factors and material characteristics. Finally, the thermal history data of the whole casting process is provided to the one-dimensional convolutional neural network model, which is used to extract global features, not just local information, to predict the mechanical properties at unknown locations. To gain deeper insight into the intricate relationship between the variables, each convolutional layer is visualized to elucidate the correlation between thermal history and mechanical properties. This research is an extension of die-casting technology in the field of artificial intelligence and a new application in the field of automotive lightweighting.

Details DOI

NeurIPS Conference 2025 Conference Paper

DataSIR: A Benchmark Dataset for Sensitive Information Recognition

  • Fan Mo
  • Bo Liu
  • Yuan Fan
  • Kun Qin
  • Yizhou Zhao
  • Jinhe Zhou
  • Jia Sun
  • Jinfei Liu

With the rapid development of artificial intelligence technologies, the demand for training data has surged, exacerbating risks of data leakage. Despite increasing incidents and costs associated with such leaks, data leakage prevention (DLP) technologies lag behind evolving evasion techniques that bypass existing sensitive information recognition (SIR) models. Current datasets lack comprehensive coverage of these adversarial transformations, limiting the evaluation of robust SIR systems. To address this gap, we introduce DataSIR, a benchmark dataset specifically designed to evaluate SIR models on sensitive data subjected to diverse format transformations. We curate 26 sensitive data categories based on multiple international regulations, and collect 131, 890 original samples correspondingly. Through empirical analysis of real-world evasion tactics, we implement 21 format transformation methods, which are applied to the original samples, expanding the dataset to 1, 647, 501 samples to simulate adversarial scenarios. We evaluated DataSIR using four traditional NLP models and four large language models (LLMs). For LLMs, we design structured prompts with varying degrees of contextual hints to assess the impact of prior knowledge on recognition accuracy. These evaluations demonstrate that our dataset effectively differentiates the performance of various SIR algorithms. Combined with its rich category and format diversity, the dataset can serve as a benchmark for evaluating related models and help develop future more advanced SIR models. Our dataset and experimental code are publicly available at https: //www. kaggle. com/datasets/fanmo1/datasir and https: //github. com/Fan-Mo-ZJU/DataSIR.

PDF Details

ICLR Conference 2025 Conference Paper

DeepSeek-Prover-V1. 5: Harnessing Proof Assistant Feedback for Reinforcement Learning and Monte-Carlo Tree Search

  • Huajian Xin
  • Z. Z. Ren
  • Junxiao Song
  • Zhihong Shao
  • Wanjia Zhao
  • Haocheng Wang
  • Bo Liu
  • Liyue Zhang

Lean is an advanced proof assistant designed to facilitate formal theorem proving by providing a variety of interactive feedback. In this paper, we explore methodologies to leverage proof assistant feedback to augment the capabilities of large language models in constructing formal proofs. First, we deploy online reinforcement learning using Lean verification outcomes as the reward signal to improve the proof completion policy. This straightforward approach shows great promise in enhancing the model's alignment with the formal verification system. In addition, we propose RMaxTS, a variant of Monte-Carlo tree search that employs an intrinsic-reward-driven exploration strategy to generate diverse proof paths. The tree structure is organized to represent the transitions of intermediate tactic states, extracted from the compilation messages given by Lean's tactic mode. The intrinsic reward is constructed to incentivize the discovery of novel tactic states, which helps to to mitigate the sparse-reward problem inherent in proof search. These techniques lead to a more efficient planning scheme for formal proof generation, achieving new state-of-the-art results on both miniF2F and ProofNet benchmarks.

Details

IJCAI Conference 2025 Conference Paper

Denoise-then-Retrieve: Text-Conditioned Video Denoising for Video Moment Retrieval

  • Weijia Liu
  • Jiuxin Cao
  • Bo Miao
  • Zhiheng Fu
  • Xuelin Zhu
  • Jiawei Ge
  • Bo Liu
  • Mehwish Nasim

Current text-driven Video Moment Retrieval (VMR) methods encode all video clips, including irrelevant ones, disrupting multimodal alignment and hindering optimization. To this end, we propose a denoise-then-retrieve paradigm that explicitly filters text-irrelevant clips from videos and then retrieves the target moment using purified multimodal representations. Following this paradigm, we introduce the Denoise-then-Retrieve Network (DRNet), comprising Text-Conditioned Denoising (TCD) and Text-Reconstruction Feedback (TRF) modules. TCD integrates cross-attention and structured state space blocks to dynamically identify noisy clips and produce a noise mask to purify multimodal video representations. TRF further distills a single query embedding from purified video representations and aligns it with the text embedding, serving as auxiliary supervision for denoising during training. Finally, we perform conditional retrieval using text embeddings on purified video representations for accurate VMR. Experiments on Charades-STA and QVHighlights demonstrate that our approach surpasses state-of-the-art methods on all metrics. Furthermore, our denoise-then-retrieve paradigm is adaptable and can be seamlessly integrated into advanced VMR models to boost performance.

PDF Details DOI

AAAI Conference 2025 Conference Paper

Differentiable Information Enhanced Model-Based Reinforcement Learning

  • Xiaoyuan Zhang
  • Xinyan Cai
  • Bo Liu
  • Weidong Huang
  • Song-Chun Zhu
  • Siyuan Qi
  • Yaodong Yang

Differentiable environments have heralded new possibilities for learning control policies by offering rich differentiable information that facilitates gradient-based methods. In comparison to prevailing model-free reinforcement learning approaches, model-based reinforcement learning (MBRL) methods exhibit the potential to effectively harness the power of differentiable information for recovering the underlying physical dynamics. However, this presents two primary challenges: effectively utilizing differentiable information to 1) construct models with more accurate dynamic prediction and 2) enhance the stability of policy training. In this paper, we propose a Differentiable Information Enhanced MBRL method, MB-MIX, to address both challenges. Firstly, we adopt a Sobolev model training approach that penalizes incorrect model gradient outputs, enhancing prediction accuracy and yielding more precise models that faithfully capture system dynamics. Secondly, we introduce mixing lengths of truncated learning windows to reduce the variance in policy gradient estimation, resulting in improved stability during policy learning. To validate the effectiveness of our approach in differentiable environments, we provide theoretical analysis and empirical results. Notably, our approach outperforms previous model-based and model-free methods, in multiple challenging tasks involving controllable rigid robots such as humanoid robots' motion control and deformable object manipulation.

PDF Details DOI

AAAI Conference 2025 Conference Paper

External Reliable Information-enhanced Multimodal Contrastive Learning for Fake News Detection

  • Biwei Cao
  • Qihang Wu
  • Jiuxin Cao
  • Bo Liu
  • Jie Gui

With the rapid development of the Internet, the information dissemination paradigm has changed and the efficiency has been improved greatly. While this also brings the quick spread of fake news and leads to negative impacts on cyberspace. Currently, the information presentation formats have evolved gradually, with the news formats shifting from texts to multimodal contents. As a result, detecting multimodal fake news has become one of the research hotspots. However, multimodal fake news detection research field still faces two main challenges: the inability to fully and effectively utilize multimodal information for detection, and the low credibility or static nature of the introduced external information, which limits dynamic updates. To bridge the gaps, we propose ERIC-FND, an external reliable information-enhanced multimodal contrastive learning framework for fake news detection. ERIC-FND strengthens the representation of news contents by entity-enriched external information enhancement method. It also enriches the multimodal news information via multimodal semantic interaction method where the multimodal constrative learning is employed to make different modality representations learn from each other. Moreover, an adaptive fusion method is taken to integrate the news representations from different dimensions for the eventual classification. Experiments are done on two commonly used datasets in different languages, X (Twitter) and Weibo. Experiment results demonstrate that our proposed model ERIC-FND outperforms existing state-of-the-art fake news detection methods under the same settings.

PDF Details DOI

AAAI Conference 2025 Conference Paper

FEAST-Mamba: FEAture and SpaTial Aware Mamba Network with Bidirectional Orthogonal Fusion for Cross-Modal Point Cloud Segmentation

  • Chade Li
  • Pengju Zhang
  • Bo Liu
  • Hao Wei
  • Yihong Wu

Point cloud segmentation has a wide range of applications in autonomous driving, augmented reality and virtual reality. Multi-modal fusion strategies have received increasing attention in point cloud segmentation recently. Despite the success, existing methods usually generate unnecessary information loss or redundancy. In this paper, we propose FEAST-Mamba, a novel FEAture and SpaTial aware Mamba network to tackle multi-modal point cloud segmentation. To exploit the complementarity between different modals, we propose a bidirectional orthogonal attention module, where features are first bidirectionally interacted with each other through cross-modal attention, and then orthogonal fusion is used to reduce feature redundancy. Furthermore, a reordering strategy is proposed for the Mamba architecture that takes into account both spatial and semantic information during cross-modal feature ordering. Experiments on indoor datasets, S3DIS and ScanNet, and outdoor datasets, nuScenes and SemanticKITTI, show that the proposed method achieves state-of-the-art performances.

PDF Details DOI

JBHI Journal 2025 Journal Article

Image Intrinsic-Based Unsupervised Monocular Depth Estimation in Endoscopy

  • Bojian Li
  • Bo Liu
  • Miao Zhu
  • Xiaoyan Luo
  • Fugen Zhou

Unsupervised monocular depth estimation plays a vital role for endoscopy-based minimally invasive surgery (MIS). However, it remains challenging due to the distinctive imaging characteristics of endoscopy which disrupt the assumption of photometric consistency, a foundation relied upon by conventional methods. Distinct from recent approaches taking image pre-processing strategy, this paper introduces a pioneering solution through intrinsic image decomposition (IID) theory. Specifically, we propose a novel end-to-end intrinsic-based unsupervised monocular depth learning framework that is comprised of an image intrinsic decomposition module and a synthesis reconstruction module. This framework seamlessly integrates IID with unsupervised monocular depth estimation, and dedicated losses are meticulously designed to offer robust supervision for network training based on this novel integration. Noteworthy, we rely on the favorable property of the resulting albedo map of IID to circumvent the challenging images characteristics instead of pre-processing the input frames. The proposed method is extensively validated on SCARED and Hamlyn datasets, and better results are obtained than state-of-the-art techniques. Beside, its generalization ability and the effectiveness of the proposed components are also validated. This innovative method has the potential to elevate the quality of 3D reconstruction in monocular endoscopy, thereby enhancing the accuracy and robustness of augmented reality navigation technology in MIS.

Details DOI

NeurIPS Conference 2025 Conference Paper

Physics-informed Neural Operator for Pansharpening

  • Xinyang Liu
  • Junming Hou
  • Chenxu Wu
  • Xiaofeng Cong
  • Zihao Chen
  • Shangqi Deng
  • Junling Li
  • Liang-Jian Deng

Over the past decades, pansharpening has contributed greatly to numerous remote sensing applications, with methods evolving from theoretically grounded models to deep learning approaches and their hybrids. Though promising, existing methods rarely address pansharpening through the lens of underlying physical imaging processes. In this work, we revisit the spectral imaging mechanism and propose a novel physics‐informed neural operator framework for pansharpening, termed PINO, which faithfully models the end‐to‐end electro‐optical sensor process. Specifically, PINO operates as: (1) First, a spatial-spectral encoder pair is introduced to aggregate multi-granularity high-resolution panchromatic (PAN) and low-resolution multispectral (LRMS) features. (2) Subsequently, an iterative neural integral process utilizes these fused spatial-spectral characteristics to learn a continuous radiance field $L_i(x, y, \lambda)$ over spatial coordinates and wavelength, effectively emulating band-wise spectral integration. (3) Finally, the learned radiance field is modulated by the sensor’s spectral responsivity $R_b(\lambda)$ to produce physically consistent spatial–spectral fusion products. This physics-grounded fusion paradigm offers a principled solution for reconstructing high-resolution multispectral and hyperspectral images in accordance with sensor imaging physics, effectively harnessing the unique advantages of spectral data to better uncover real-world characteristics. Experiments on multiple benchmark datasets show that our method surpasses state-of-the-art fusion algorithms, achieving reduced spectral aberrations and finer spatial textures. Furthermore, extension to hyperspectral (HS) data demonstrates its generalizability and universality. The code will be available upon potential acceptance.

PDF Details

AAAI Conference 2025 Conference Paper

SemiDFL: A Semi-Supervised Paradigm for Decentralized Federated Learning

  • Xinyang Liu
  • Pengchao Han
  • Xuan Li
  • Bo Liu

Decentralized federated learning (DFL) realizes cooperative model training among connected clients without relying on a central server, thereby mitigating communication bottlenecks and eliminating the single-point failure issue present in centralized federated learning (CFL). Most existing work on DFL focuses on supervised learning, assuming each client possesses sufficient labeled data for local training. However, in real-world applications, much of the data is unlabeled. We address this by considering a challenging yet practical semi-supervised learning (SSL) scenario in DFL, where clients may have varying data sources: some with few labeled samples, some with purely unlabeled data, and others with both. In this work, we propose SemiDFL, the first semi-supervised DFL method that enhances DFL performance in SSL scenarios by establishing a consensus in both data and model spaces. Specifically, we utilize neighborhood information to improve the quality of pseudo-labeling, which is crucial for effectively leveraging unlabelled data. We then design a consensus-based diffusion model to generate synthesized data, which is used in combination with pseudo-labeled data to create mixed datasets. Additionally, we develop an adaptive aggregation method that leverages the model accuracy of synthesized data to further enhance SemiDFL performance. Through extensive experimentation, we demonstrate the remarkable performance superiority of the proposed DFL-Semi method over existing CFL and DFL schemes in both iid and Non-iid SSL scenarios.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

AdaFlow: Imitation Learning with Variance-Adaptive Flow-Based Policies

  • Xixi Hu
  • Bo Liu
  • Xingchao Liu
  • Qiang Liu

Diffusion-based imitation learning improves Behavioral Cloning (BC) on multi-modal decision-making, but comes at the cost of significantly slower inference due to the recursion in the diffusion process. It urges us to design efficient policy generators while keeping the ability to generate diverse actions. To address this challenge, we propose AdaFlow, an imitation learning framework based on flow-based generative modeling. AdaFlow represents the policy with state-conditioned ordinary differential equations (ODEs), which are known as probability flows. We reveal an intriguing connection between the conditional variance of their training loss and the discretization error of the ODEs. With this insight, we propose a variance-adaptive ODE solver that can adjust its step size in the inference stage, makingAdaFlow an adaptive decision-maker, offering rapid inference without sacrificing diversity. Interestingly, it automatically reduces to a one-step generator when the action distribution is uni-modal. Our comprehensive empirical evaluation shows that AdaFlow achieves high performance with fast inference speed.

PDF Details DOI

IJCAI Conference 2024 Conference Paper

All in One: Multi-task Prompting for Graph Neural Networks (Extended Abstract)

  • Xiangguo Sun
  • Hong Cheng
  • Jia Li
  • Bo Liu
  • Jihong Guan

This paper is an extended abstract of our original work published in KDD23, where we won the best research paper award. The paper introduces a novel approach to bridging the gap between pre-trained graph models and the diverse tasks they’re applied to, inspired by the success of prompt learning in NLP. Recognizing the challenge of aligning pre-trained models with varied graph tasks (node level, edge level, and graph level), which can lead to negative transfer and poor performance, we propose a multi-task prompting method for graphs. This method involves unifying graph and language prompt formats, enabling NLP’s prompting strategies to be adapted for graph tasks. By analyzing the task space of graph applications, we reformulate problems to fit graph-level tasks and apply meta-learning to improve prompt initialization for multiple tasks. Experiments show our method’s effectiveness in enhancing model performance across different graph tasks. Beyond the original work, in this extended abstract, we further discuss the graph prompt from a bigger picture and provide some of the latest work toward this area.

PDF Details DOI

ICLR Conference 2024 Conference Paper

Biased Temporal Convolution Graph Network for Time Series Forecasting with Missing Values

  • Xiaodan Chen
  • Xiucheng Li
  • Bo Liu
  • Zhijun Li 0002

Multivariate time series forecasting plays an important role in various applications ranging from meteorology study, traffic management to economics planning. In the past decades, many efforts have been made toward accurate and reliable forecasting methods development under the assumption of intact input data. However, the time series data from real-world scenarios is often partially observed due to device malfunction or costly data acquisition, which can seriously impede the performance of the existing approaches. A naive employment of imputation methods unavoidably involves error accumulation and leads to suboptimal solutions. Motivated by this, we propose a Biased Temporal Convolution Graph Network that jointly captures the temporal dependencies and spatial structure. In particular, we inject bias into the two carefully developed modules, the Multi-Scale Instance PartialTCN and Biased GCN, to account for missing patterns. The experimental results show that our proposed model is able to achieve up to $9.93$\% improvements over the existing methods on five real-world benchmark datasets. Our code is available at: https://github.com/chenxiaodanhit/BiTGraph.

Details

JBHI Journal 2024 Journal Article

cbPPGGAN: A Generic Enhancement Framework for Unpaired Pulse Waveforms in Camera-Based Photoplethysmography

  • Ze Yang
  • Haofei Wang
  • Bo Liu
  • Feng Lu

Camera-based photoplethysmography (cbP PG) is a non-contact technique that measures cardiac-related blood volume alterations in skin surface vessels through the analysis of facial videos. While traditional approaches can estimate heart rate (HR) under different illuminations, their accuracy can be affected by motion artifacts, leading to poor waveform fidelity and hindering further analysis of heart rate variability (HRV); deep learning-based approaches reconstruct high-quality pulse waveform, yet their performance significantly degrades under illumination variations. In this work, we aim to leverage the strength of these two methods and propose a framework that possesses favorable generalization capabilities while maintaining waveform fidelity. For this purpose, we propose the cbPPGGAN, an enhancement framework for cbPPG that enables the flexible incorporation of both unpaired and paired data sources in the training process. Based on the waveforms extracted by traditional approaches, the cbPPGGAN reconstructs high-quality waveforms that enable accurate HR estimation and HRV analysis. In addition, to address the lack of paired training data problems in real-world applications, we propose a cycle consistency loss that guarantees the time-frequency consistency before/after mapping. The method enhances the waveform quality of traditional POS approaches in different illumination tests (BH-rPPG) and cross-datasets (UBFC-rPPG) with mean absolute error (MAE) values of 1. 34 bpm and 1. 65 bpm, and average beat-to-beat (AVBB) values of 27. 46 ms and 45. 28 ms, respectively. Experimental results demonstrate that the cbPPGGAN enhances cbPPG signal quality and outperforms the state-of-the-art approaches in HR estimation and HRV analysis. The proposed framework opens a new pathway toward accurate HR estimation in an unconstrained environment.

Details DOI

NeurIPS Conference 2024 Conference Paper

Communication Efficient Distributed Training with Distributed Lion

  • Bo Liu
  • Lemeng Wu
  • Lizhang Chen
  • Kaizhao Liang
  • Jiaxu Zhu
  • Chen Liang
  • Raghuraman Krishnamoorthi
  • Qiang Liu

The Lion optimizer has been a promising competitor with the AdamW for training large AI models, with advantages in memory, computation, and sample efficiency. In this paper, we introduce Distributed Lion, an innovative adaptation of Lion for distributed training environments. Leveraging the sign operator in Lion, our Distributed Lion only requires to communicate binary or lower-precision vectorsbetween workers to the center server, significantly reducing the communication cost. Our theoretical analysis confirms Distributed Lion's convergence properties. Empirical results demonstrate its robustness across a range of tasks, worker counts, and batch sizes, on both vision and language problems. Notably, Distributed Lion attains comparable performance to standard Lion or AdamW optimizers applied on aggregated gradients, but with significantly reduced communication bandwidth. This feature is particularly advantageous for training large models. In addition, we also demonstrate that \mavolion{} presents a more favorable performance-bandwidth balance compared to existing efficient distributed methods such as deep gradient compression and ternary gradients.

PDF Details DOI

EAAI Journal 2024 Journal Article

Data extension-based analysis and application selection of process-composition-properties of die casting aluminum alloy

  • Jian Yang
  • Bo Liu
  • Yunbo Zeng
  • Yiben Zhang
  • Haiyou Huang
  • Jichao Hong

This research aims to provide a solution to the scarcity and fragmentation of industrial data on die casting aluminum alloys. Quantifying the coupling between die casting process-composition-properties of aluminum alloys through small datasets, is a critical step in predicting part properties and optimizing process selection. To visualize the connections and discuss the effect of the interaction between different parameters on the property, data is fed into a self-organizing mapping model. Whereafter, an innovative data extension method is proposed to predict both yield and tensile strengths with more than 96% accuracy using a small data set. Moreover, two novel methods of multi-parameter combined range selection, the multi-objective optimization based on the agent model and the superimposition of the contour map, are guided by being informed in which region the mechanical properties fall. Finally, the feasibility of application range selection method is verified experimentally. Mapping based on data-driven process-composition-properties relationships and the free combination of application ranges are reliable theoretical solutions which are valuable for practical applications.

Details DOI

AAMAS Conference 2024 Conference Paper

Dual Role AoI-based Incentive Mechanism for HD map Crowdsourcing

  • Wentao Ye
  • Bo Liu
  • Yuan Luo
  • Jianwei Huang

A high-quality fresh high-definition (HD) map is vital in enhancing transportation efficiency and safety in autonomous driving. Vehiclebased crowdsourcing offers a promising approach for updating HD maps. However, recruiting crowdsourcing vehicles involves making the challenging tradeoff between the HD map freshness and recruitment cost. Existing studies on HD map crowdsourcing often (1) prioritize maximizing spatial coverage, and (2) overlook the dual role of crowdsourcing vehicles in HD maps, as vehicles serve both as contributors and customers of HD maps. This motivates us to propose the Dual-Role Age of Information (AoI) based Incentive Mechanism (DRAIM) to address these issues. DRAIM aims to achieve the company’s tradeoff between freshness and recruitment cost.

PDF

AAAI Conference 2024 Conference Paper

Focus Stacking with High Fidelity and Superior Visual Effects

  • Bo Liu
  • Bin Hu
  • Xiuli Bi
  • Weisheng Li
  • Bin Xiao

Focus stacking is a technique in computational photography, and it synthesizes a single all-in-focus image from different focal plane images. It is difficult for previous works to produce a high-quality all-in-focus image that meets two goals: high-fidelity to its source images and good visual effects without defects or abnormalities. This paper proposes a novel method based on optical imaging process analysis and modeling. Based on a foreground segmentation - diffusion elimination architecture, the foreground segmentation makes most of the areas in full-focus images heritage information from the source images to achieve high fidelity; diffusion elimination models the physical imaging process and is specially used to solve the transition region (TR) problem that is a long-term neglected issue and degrades visual effects of synthesized images. Based on extensive experiments on simulated dataset, existing realistic dataset and our proposed BetaFusion dataset, the results show that our proposed method can generate high-quality all-in-focus images by achieving two goals simultaneously, especially can successfully solve the TR problem and eliminate the visual effect degradation of synthesized images caused by the TR problem.

PDF Details DOI

AAAI Conference 2024 Conference Paper

From Past to Future: Rethinking Eligibility Traces

  • Dhawal Gupta
  • Scott M. Jordan
  • Shreyas Chaudhari
  • Bo Liu
  • Philip S. Thomas
  • Bruno Castro da Silva

In this paper, we introduce a fresh perspective on the challenges of credit assignment and policy evaluation. First, we delve into the nuances of eligibility traces and explore instances where their updates may result in unexpected credit assignment to preceding states. From this investigation emerges the concept of a novel value function, which we refer to as the??????????????????????????. Unlike traditional state value functions, bidirectional value functions account for both future expected returns (rewards anticipated from the current state onward) and past expected returns (cumulative rewards from the episode's start to the present). We derive principled update equations to learn this value function and, through experimentation, demonstrate its efficacy in enhancing the process of policy evaluation. In particular, our results indicate that the proposed learning approach can, in certain challenging contexts, perform policy evaluation more rapidly than TD(λ)–a method that learns forward value functions, v^π,????????. Overall, our findings present a new perspective on eligibility traces and potential advantages associated with the novel value function it inspires, especially for policy evaluation.

PDF Details DOI

JBHI Journal 2024 Journal Article

Fully Automatic Fine-Grained Grading of Lumbar Intervertebral Disc Degeneration Using Regional Feature Recalibration Network

  • Nuo Tong
  • Shuiping Gou
  • Yulin Yang
  • Bo Liu
  • Yufeng Bai
  • Jingzhong Liu
  • Tan Ding

Accurate fine-grained grading of lumbar intervertebral disc (LIVD) degeneration is essential for the diagnosis and treatment design of high-incidence low back pain. However, the grading accuracy is still challenged by lacking the fine-grained degenerative details, which is mainly due to the existing grading methods are easily dominated by the salient nucleus pulposus regions in LIVD, overlooking the inconspicuous degeneration changes of the surrounding structures. In this study, a novel regional feature recalibration network (RFRecNet) is proposed to achieve accurate and reliable LIVD degeneration grading. Detection transformer (DETR) is first utilized to detect all LIVDs and then input to the proposed RFRecNet for the fine-grained grading. To obtain sufficient features from both the salient nucleus pulposus and the surrounding regions, a regional cube-based feature boosting and suppression (RC-FBS) module is designed to adaptively recalibrate the feature extraction and utilization from the various regions in LIVD, and a feature diversification (FD) module is proposed to capture the complementary semantic information from the multi-scale features for the comprehensive fine-grained degeneration grading. Extensive experiments were conducted on a clinically collected dataset, which consists of 500 MR scans with a total of 10225 LIVDs. An average grading accuracy of 90. 5%, specificity of 97. 5%, sensitivity of 90. 8%, and Cohen's kappa correlation coefficient of 0. 876 are obtained, which indicate that the proposed framework is promising to provide doctors with reliable and consistent fine-grained quantitative evaluation results of the LIVD degeneration conditions for the optimal surgical plan design.

Details DOI

AAAI Conference 2024 Conference Paper

Generalizable Fourier Augmentation for Unsupervised Video Object Segmentation

  • Huihui Song
  • Tiankang Su
  • Yuhui Zheng
  • Kaihua Zhang
  • Bo Liu
  • Dong Liu

The performance of existing unsupervised video object segmentation methods typically suffers from severe performance degradation on test videos when tested in out-of-distribution scenarios. The primary reason is that the test data in real- world may not follow the independent and identically distribution (i.i.d.) assumption, leading to domain shift. In this paper, we propose a generalizable fourier augmentation method during training to improve the generalization ability of the model. To achieve this, we perform Fast Fourier Transform (FFT) over the intermediate spatial domain features in each layer to yield corresponding frequency representations, including amplitude components (encoding scene-aware styles such as texture, color, contrast of the scene) and phase components (encoding rich semantics). We produce a variety of style features via Gaussian sampling to augment the training data, thereby improving the generalization capability of the model. To further improve the cross-domain generalization performance of the model, we design a phase feature update strategy via exponential moving average using phase features from past frames in an online update manner, which could help the model to learn cross-domain-invariant features. Extensive experiments show that our proposed method achieves the state-of-the-art performance on popular benchmarks.

PDF Details DOI

JBHI Journal 2024 Journal Article

IoMT-Based Smart Healthcare Detection System Driven by Quantum Blockchain and Quantum Neural Network

  • Zhiguo Qu
  • Wenke Shi
  • Bo Liu
  • Deepak Gupta
  • Prayag Tiwari

Electrocardiogram (ECG) is the main criterion for arrhythmia detection. As a means of identification, ECG leakage seems to be a common occurrence due to the development of the Internet of Medical Things. The advent of the quantum era makes it difficult for classical blockchain technology to provide security for ECG data storage. Therefore, from the perspective of safety and practicality, this article proposes a quantum arrhythmia detection system called QADS, which achieves secure storage and sharing of ECG data based on quantum blockchain technology. Furthermore, a quantum neural network is used in QADS to recognize abnormal ECG data, which contributes to further cardiovascular disease diagnosis. Each quantum block stores the hash of the current and previous block to construct a quantum block network. The new quantum blockchain algorithm introduces a controlled quantum walk hash function and a quantum authentication protocol to guarantee legitimacy and security while creating new blocks. In addition, this article constructs a hybrid quantum convolutional neural network called HQCNN to extract the temporal features of ECG to detect abnormal heartbeats. The simulation experimental results show that HQCNN achieves an average training and testing accuracy of 94. 7% and 93. 6%. And the detection stability is much higher than classical CNN with the same structure. HQCNN also has certain robustness under the perturbation of quantum noise. Besides, this article demonstrates through mathematical analysis that the proposed quantum blockchain algorithm has strong security and can effectively resist various quantum attacks, such as external attacks, Entanglement-Measure attack and Interception-Measurement-Repeat attack.

Details DOI

NeurIPS Conference 2024 Conference Paper

Memory-Efficient LLM Training with Online Subspace Descent

  • Kaizhao Liang
  • Bo Liu
  • Lizhang Chen
  • Qiang Liu

Recently, a wide range of memory-efficient LLM training algorithms have gained substantial popularity. These methods leverage the low-rank structure of gradients to project optimizer states into a subspace using projection matrix found by singular value decomposition (SVD). However, convergence of these algorithms is highly dependent on the update rules of their projection matrix. In this work, we provide the \emph{first} convergence guarantee for arbitrary update rules of projection matrix. This guarantee is generally applicable to optimizers that can be analyzed with Hamiltonian Descent, including most common ones, such as LION, Adam. Inspired by our theoretical understanding, we propose Online Subspace Descent, a new family of subspace descent optimizer without SVD. Instead of updating projection matrix with eigenvectors, Online Subspace Descent updates projection matrix wtih online PCA. Online Subspace Descent is flexible and introduces only minimum overhead to training. We demonstrate that, for the task of pretraining LLaMA models ranging from 60M to 1B parameters on the C4 dataset, Online Subspace Descent achieves lower perplexity than state-of-the-art low-rank training methods across different settings and narrows the gap with full-rank baselines.

PDF Details DOI

AAAI Conference 2024 Conference Paper

MuLTI: Efficient Video-and-Language Understanding with Text-Guided MultiWay-Sampler and Multiple Choice Modeling

  • Jiaqi Xu
  • Bo Liu
  • Yunkuo Chen
  • Mengli Cheng
  • Xing Shi

Video-and-language understanding has a variety of applications in the industry, such as video question answering, text-video retrieval, and multi-label classification. Existing video-and-language understanding methods generally adopt heavy multi-modal encoders and feature fusion modules, which consume high computational costs. Specially, they have difficulty dealing with dense video frames or long text prevalent in industrial applications. This paper proposes MuLTI, a highly accurate and efficient video-and-language understanding model that achieves efficient and effective feature fusion and rapid adaptation to downstream tasks. Specifically, we design a Text-Guided MultiWay-Sampler based on adapt-pooling residual mapping and self-attention modules to sample long sequences and fuse multi-modal features, which reduces the computational costs and addresses performance degradation caused by previous samplers. Therefore, MuLTI can handle longer sequences with limited computational costs. Then, to further enhance the model's performance and fill in the lack of pretraining tasks in the video question answering, we propose a new pretraining task named Multiple Choice Modeling. This task bridges the gap between pretraining and downstream tasks and improves the model's ability to align video and text features. Benefiting from the efficient feature fusion module and the new pretraining task, MuLTI achieves state-of-the-art performance on multiple datasets. Implementation and pretrained models will be released.

PDF Details DOI

AAMAS Conference 2024 Conference Paper

Overview of t-DGR: A Trajectory-Based Deep Generative Replay Method for Continual Learning in Decision Making

  • William Yue
  • Bo Liu
  • Peter Stone

Deep generative replay has emerged as a promising approach for continual learning in decision-making tasks. This approach addresses the problem of catastrophic forgetting by leveraging the generation of trajectories from previously encountered tasks to augment the current dataset. However, existing deep generative replay methods for continual learning rely on autoregressive models, which suffer from compounding errors in the generated trajectories. In this extended abstract, we summarize a simple, scalable, and non-autoregressive method for continual learning in decision-making tasks using a generative model that generates task samples conditioned on the trajectory timestep. We evaluate our method on Continual World benchmarks and find that our approach achieves state-of-the-art performance on the average success rate metric among continual learning methods. Code and a preprint of a complete paper with full details are available at https: //github. com/WilliamYue37/t-DGR.

PDF

EAAI Journal 2024 Journal Article

Progressive structure enhancement graph convolutional network for face clustering

  • Shaoying Li
  • Wei Yao
  • Yuan Gao
  • Yinchi Ma
  • Bo Liu

Face clustering, a technique for automatically annotating large-scale face data, has made significant advancements with the advent of graph convolutional networks (GCNs). Despite their success, GCNs can suffer from decreased performance due to conflicting information passed along noisy edges of a graph. To address this issue, we propose a novel framework named progressive structure enhancement GCN (PSE-GCN), which combines graph structure learning with graph-guided feature aggregation. Our PSE-GCN framework includes a dynamic graph construction (DGC) module that enhances local relationships and suppresses global noise, thereby improving the quality of the graph. By stacking multiple DGCs, PSE-GCN progressively refines the graph quality and yields discriminative features for various clustering tasks. Additionally, we introduce a subgraph-based neighborhood re-ranking (SNR) mechanism that improves graph homogeneity by rearranging the candidate neighbors of each face based on structural similarity at the subgraph level. Our experimental results, conducted on several popular benchmarks, not only demonstrate the effectiveness of PSE-GCN, but also show that it outperforms state-of-the-art methods, e. g. , 93. 50% in pairwise F-score on the MS-Celeb-1M dataset.

Details DOI

AAMAS Conference 2024 Conference Paper

Relaxed Exploration Constrained Reinforcement Learning

  • Shahaf S. Shperberg
  • Bo Liu
  • Peter Stone

This research introduces a novel setting for reinforcement learning with constraints, termed Relaxed Exploration Constrained Reinforcement Learning (RECRL). Similar to standard constrained reinforcement learning (CRL), the objective in RECRL is to discover a policy that maximizes the environmental return while adhering to a predefined set of constraints. However, in some real-world settings, it is possible to train the agent in a setting that does not require strict adherence to the constraints, as long as the agent adheres to them once deployed. To model such settings, we introduce RECRL, which explicitly incorporates an initial training phase where the constraints are relaxed, enabling the agent to explore the environment more freely. Subsequently, during deployment, the agent is obligated to fully satisfy all constraints. To address RECRL problems, we introduce a curriculum-based approach called CLiC, designed to enhance the exploration of existing CRL algorithms during the training phase and facilitate convergence towards a policy that satisfies the full set of constraints by the end of training. Empirical evaluations demonstrate that CLiC yields policies with significantly higher returns during deployment compared to training solely under the strict set of constraints. The code is available at https: //github. com/Shperb/RECRL.

PDF

IJCAI Conference 2024 Conference Paper

When Fairness Meets Privacy: Exploring Privacy Threats in Fair Binary Classifiers via Membership Inference Attacks

  • Huan Tian
  • Guangsheng Zhang
  • Bo Liu
  • Tianqing Zhu
  • Ming Ding
  • Wanlei Zhou

While in-processing fairness approaches show promise in mitigating bias predictions, their potential impact on privacy leakage remains under-explored. We aim to address this gap by assessing the privacy risks of fairness-enhanced binary classifiers with membership inference attacks (MIAs). Surprisingly, our results reveal that these fairness interventions exhibit increased resilience against existing attacks, indicating that enhancing fairness does not necessarily lead to privacy compromises. However, we find current attack methods are ineffective as they typically degrade into simple threshold models with limited attack effectiveness. Following this observation, we discover a novel threat dubbed Fairness Discrepancy Membership Inference Attacks (FD-MIA) that exploits prediction discrepancies between fair and biased models. This attack reveals more potent vulnerabilities and poses significant privacy risks to model privacy. Extensive experiments across multiple datasets, attack methods, and representative fairness approaches confirm our findings and demonstrate the efficacy of the proposed attack method. Our study exposes the overlooked privacy threats in fairness studies, advocating for thorough evaluations of potential security vulnerabilities before model deployments.

PDF Details DOI

JBHI Journal 2023 Journal Article

EEG-Based Parkinson's Disease Recognition via Attention-Based Sparse Graph Convolutional Neural Network

  • Hongli Chang
  • Bo Liu
  • Yuan Zong
  • Cheng Lu
  • Xuenan Wang

Parkinson's disease (PD) is a complicated neurological ailment that affects both the physical and mental wellness of elderly individuals which makes it problematic to diagnose in its initial stages. Electroencephalogram (EEG) promises to be an efficient and cost-effective method for promptly detecting cognitive impairment in PD. Nevertheless, prevailing diagnostic practices utilizing EEG features have failed to examine the functional connectivity among EEG channels and the response of associated brain areas causing an unsatisfactory level of precision. Here, we construct an attention-based sparse graph convolutional neural network (ASGCNN) for diagnosing PD. Our ASGCNN model uses a graph structure to represent channel relationships, the attention mechanism for selecting channels, and the L1 norm to capture channel sparsity. We conduct extensive experiments on the publicly available PD auditory oddball dataset, which consists of 24 PD patients (under ON/OFF drug status) and 24 matched controls, to validate the effectiveness of our method. Our results show that the proposed method provides better results compared to the publicly available baselines. The achieved scores for Recall, Precision, F1-score, Accuracy and Kappa measures are 90. 36%, 88. 43%, 88. 41%, 87. 67%, and 75. 24%, respectively. Our study reveals that the frontal and temporal lobes show significant differences between PD patients and healthy individuals. In addition, EEG features extracted by ASGCNN demonstrate significant asymmetry in the frontal lobe among PD patients. These findings can offer a basis for the establishment of a clinical system for intelligent diagnosis of PD by using auditory cognitive impairment features.

Details DOI

NeurIPS Conference 2023 Conference Paper

FAMO: Fast Adaptive Multitask Optimization

  • Bo Liu
  • Yihao Feng
  • Peter Stone
  • Qiang Liu

One of the grand enduring goals of AI is to create generalist agents that can learn multiple different tasks from diverse data via multitask learning (MTL). However, in practice, applying gradient descent (GD) on the average loss across all tasks may yield poor multitask performance due to severe under-optimization of certain tasks. Previous approaches that manipulate task gradients for a more balanced loss decrease require storing and computing all task gradients ($\mathcal{O}(k)$ space and time where $k$ is the number of tasks), limiting their use in large-scale scenarios. In this work, we introduce Fast Adaptive Multitask Optimization (FAMO), a dynamic weighting method that decreases task losses in a balanced way using $\mathcal{O}(1)$ space and time. We conduct an extensive set of experiments covering multi-task supervised and reinforcement learning problems. Our results indicate that FAMO achieves comparable or superior performance to state-of-the-art gradient manipulation techniques while offering significant improvements in space and computational efficiency. Code is available at \url{https: //github. com/Cranial-XIX/FAMO}.

PDF Details

NeurIPS Conference 2023 Conference Paper

LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning

  • Bo Liu
  • Yifeng Zhu
  • Chongkai Gao
  • Yihao Feng
  • Qiang Liu
  • Yuke Zhu
  • Peter Stone

Lifelong learning offers a promising paradigm of building a generalist agent that learns and adapts over its lifespan. Unlike traditional lifelong learning problems in image and text domains, which primarily involve the transfer of declarative knowledge of entities and concepts, lifelong learning in decision-making (LLDM) also necessitates the transfer of procedural knowledge, such as actions and behaviors. To advance research in LLDM, we introduce LIBERO, a novel benchmark of lifelong learning for robot manipulation. Specifically, LIBERO highlights five key research topics in LLDM: 1) how to efficiently transfer declarative knowledge, procedural knowledge, or the mixture of both; 2) how to design effective policy architectures and 3) effective algorithms for LLDM; 4) the robustness of a lifelong learner with respect to task ordering; and 5) the effect of model pretraining for LLDM. We develop an extendible procedural generation pipeline that can in principle generate infinitely many tasks. For benchmarking purpose, we create four task suites (130 tasks in total) that we use to investigate the above-mentioned research topics. To support sample-efficient learning, we provide high-quality human-teleoperated demonstration data for all tasks. Our extensive experiments present several insightful or even unexpected discoveries: sequential finetuning outperforms existing lifelong learning methods in forward transfer, no single visual encoder architecture excels at all types of knowledge transfer, and naive supervised pretraining can hinder agents' performance in the subsequent LLDM.

PDF Details

AAAI Conference 2023 Conference Paper

Metric Residual Network for Sample Efficient Goal-Conditioned Reinforcement Learning

  • Bo Liu
  • Yihao Feng
  • Qiang Liu
  • Peter Stone

Goal-conditioned reinforcement learning (GCRL) has a wide range of potential real-world applications, including manipulation and navigation problems in robotics. Especially in such robotics tasks, sample efficiency is of the utmost importance for GCRL since, by default, the agent is only rewarded when it reaches its goal. While several methods have been proposed to improve the sample efficiency of GCRL, one relatively under-studied approach is the design of neural architectures to support sample efficiency. In this work, we introduce a novel neural architecture for GCRL that achieves significantly better sample efficiency than the commonly-used monolithic network architecture. The key insight is that the optimal action-value function must satisfy the triangle inequality in a specific sense. Furthermore, we introduce the metric residual network (MRN) that deliberately decomposes the action-value function into the negated summation of a metric plus a residual asymmetric component. MRN provably approximates any optimal action-value function, thus making it a fitting neural architecture for GCRL. We conduct comprehensive experiments across 12 standard benchmark environments in GCRL. The empirical results demonstrate that MRN uniformly outperforms other state-of-the-art GCRL neural architectures in terms of sample efficiency. The code is available at https://github.com/Cranial-XIX/metric-residual-network.

PDF Details DOI

AAMAS Conference 2023 Conference Paper

Relaxed Exploration Constrained Reinforcement Learning

  • Shahaf S. Shperberg
  • Bo Liu
  • Peter Stone

This extended abstract introduces a novel setting of reinforcement learning with constraints, called Relaxed Exploration Constrained Reinforcement Learning (RECRL). As in standard constrained reinforcement learning (CRL), the aim is to find a policy that maximizes environmental return subject to a set of constraints. However, in RECRL there is an initial training phase in which the constraints are relaxed, thus the agent can explore the environment more freely. When training is done, the agent is deployed in the environment and is required to fully satisfy all constraints. As an initial approach to RECRL problems, we introduce a curriculum-based approach, named CLiC, that can be applied to existing CRL algorithms to improve their exploration during the training phase while allowing them to gradually converge to a policy that satisfies the full set of constraints. Empirical evaluation shows that CLiC produces policies with a higher return during deployment than policies learned when training is done using only the strict set of constraints.

PDF

AAAI Conference 2023 Conference Paper

Self-Supervised Image Local Forgery Detection by JPEG Compression Trace

  • Xiuli Bi
  • Wuqing Yan
  • Bo Liu
  • Bin Xiao
  • Weisheng Li
  • Xinbo Gao

For image local forgery detection, the existing methods require a large amount of labeled data for training, and most of them cannot detect multiple types of forgery simultaneously. In this paper, we firstly analyzed the JPEG compression traces which are mainly caused by different JPEG compression chains, and designed a trace extractor to learn such traces. Then, we utilized the trace extractor as the backbone and trained self-supervised to strengthen the discrimination ability of learned traces. With its benefits, regions with different JPEG compression chains can easily be distinguished within a forged image. Furthermore, our method does not rely on a large amount of training data, and even does not require any forged images for training. Experiments show that the proposed method can detect image local forgery on different datasets without re-training, and keep stable performance over various types of image local forgery.

PDF Details DOI

EAAI Journal 2022 Journal Article

A multi-task learning for cavitation detection and cavitation intensity recognition of valve acoustic signals

  • Yu Sha
  • Johannes Faber
  • Shuiping Gou
  • Bo Liu
  • Wei Li
  • Stefan Schramm
  • Horst Stoecker
  • Thomas Steckenreiter

With the rapid development of smart manufacturing, data-driven machinery health management has received a growing attention. As one of the most popular methods in machinery health management, deep learning (DL) has achieved remarkable successes. However, due to the issues of limited samples and poor separability of different cavitation states of acoustic signals, which greatly hinder the eventual performance of DL modes for cavitation intensity recognition and cavitation detection. Also different tasks were performed separately conventionally. In this work, a novel multi-task learning framework for simultaneous cavitation detection and cavitation intensity recognition framework using 1-D double hierarchical residual networks (1-D DHRN) is proposed for analyzing valves acoustic signals. Firstly, a data augmentation method based on sliding window with fast Fourier transform (Swin-FFT) is developed to alleviate the small-sample issue confronted in this study. Secondly, a 1-D double hierarchical residual block (1-D DHRB) is constructed to capture sensitive features from the frequency domain acoustic signals of valve. Then, a new structure of 1-D DHRN is proposed. Finally, the devised 1-D DHRN is evaluated on two datasets of valve acoustic signals without noise ( Dataset 1 and Dataset 2 ) and one dataset of valve acoustic signals with realistic surrounding noise ( Dataset 3 ) provided by SAMSON AG (Frankfurt). Our method has achieved state-of-the-art results. The prediction accuracies of 1-D DHRN for cavitation intensitys recognition are as high as 93. 75%, 94. 31% and 100%, which indicates that 1-D DHRN outperforms other DL models and conventional methods. At the same time, the testing accuracies of 1-D DHRN for cavitation detection are as high as 97. 02%, 97. 64% and 100%. In addition, 1-D DHRN has also been tested for different frequencies of samples and shows excellent results for frequency of samples that mobile phones can accommodate.

Details DOI

NeurIPS Conference 2022 Conference Paper

A Theoretical Understanding of Gradient Bias in Meta-Reinforcement Learning

  • Bo Liu
  • Xidong Feng
  • Jie Ren
  • Luo Mai
  • Rui Zhu
  • Haifeng Zhang
  • Jun Wang
  • Yaodong Yang

Gradient-based Meta-RL (GMRL) refers to methods that maintain two-level optimisation procedures wherein the outer-loop meta-learner guides the inner-loop gradient-based reinforcement learner to achieve fast adaptations. In this paper, we develop a unified framework that describes variations of GMRL algorithms and points out that existing stochastic meta-gradient estimators adopted by GMRL are actually \textbf{biased}. Such meta-gradient bias comes from two sources: 1) the compositional bias incurred by the two-level problem structure, which has an upper bound of $\mathcal{O}\big(K\alpha^{K}\hat{\sigma}_{\text{In}}|\tau|^{-0. 5}\big)$ \emph{w. r. t. } inner-loop update step $K$, learning rate $\alpha$, estimate variance $\hat{\sigma}^{2}_{\text{In}}$ and sample size $|\tau|$, and 2) the multi-step Hessian estimation bias $\hat{\Delta}_{H}$ due to the use of autodiff, which has a polynomial impact $\mathcal{O}\big((K-1)(\hat{\Delta}_{H})^{K-1}\big)$ on the meta-gradient bias. We study tabular MDPs empirically and offer quantitative evidence that testifies our theoretical findings on existing stochastic meta-gradient estimators. Furthermore, we conduct experiments on Iterated Prisoner's Dilemma and Atari games to show how other methods such as off-policy learning and low-bias estimator can help fix the gradient bias for GMRL algorithms in general.

PDF Details

NeurIPS Conference 2022 Conference Paper

BOME! Bilevel Optimization Made Easy: A Simple First-Order Approach

  • Bo Liu
  • Mao Ye
  • Stephen Wright
  • Peter Stone
  • Qiang Liu

Bilevel optimization (BO) is useful for solving a variety of important machine learning problems including but not limited to hyperparameter optimization, meta-learning, continual learning, and reinforcement learning. Conventional BO methods need to differentiate through the low-level optimization process with implicit differentiation, which requires expensive calculations related to the Hessian matrix. There has been a recent quest for first-order methods for BO, but the methods proposed to date tend to be complicated and impractical for large-scale deep learning applications. In this work, we propose a simple first-order BO algorithm that depends only on first-order gradient information, requires no implicit differentiation, and is practical and efficient for large-scale non-convex functions in deep learning. We provide non-asymptotic convergence analysis of the proposed method to stationary points for non-convex objectives and present empirical results that show its superior practical performance.

PDF Details

IJCAI Conference 2022 Conference Paper

Boosting Multi-Label Image Classification with Complementary Parallel Self-Distillation

  • Jiazhi Xu
  • Sheng Huang
  • Fengtao Zhou
  • Luwen Huangfu
  • Daniel Zeng
  • Bo Liu

Multi-Label Image Classification (MLIC) appro-aches usually exploit label correlations to achieve good performance. However, emphasizing correlation like co-occurrence may overlook discriminative features and lead to model overfitting. In this study, we propose a generic framework named Parallel Self-Distillation (PSD) for boosting MLIC models. PSD decomposes the original MLIC task into several simpler MLIC sub-tasks via two elaborated complementary task decomposition strategies named Co-occurrence Graph Partition (CGP) and Dis-occurrence Graph Partition (DGP). Then, the MLIC models of fewer categories are trained with these sub-tasks in parallel for respectively learning the joint patterns and the category-specific patterns of labels. Finally, knowledge distillation is leveraged to learn a compact global ensemble of full categories with these learned patterns for reconciling the label correlation exploitation and model overfitting. Extensive results on MS-COCO and NUS-WIDE datasets demonstrate that our framework can be easily plugged into many MLIC approaches and improve performances of recent state-of-the-art approaches. The source code is released at https: //github. com/Robbie-Xu/CPSD.

PDF Details DOI

NeurIPS Conference 2022 Conference Paper

EnvPool: A Highly Parallel Reinforcement Learning Environment Execution Engine

  • Jiayi Weng
  • Min Lin
  • Shengyi Huang
  • Bo Liu
  • Denys Makoviichuk
  • Viktor Makoviychuk
  • Zichen Liu
  • Yufan Song

There has been significant progress in developing reinforcement learning (RL) training systems. Past works such as IMPALA, Apex, Seed RL, Sample Factory, and others, aim to improve the system's overall throughput. In this paper, we aim to address a common bottleneck in the RL training system, i. e. , parallel environment execution, which is often the slowest part of the whole system but receives little attention. With a curated design for paralleling RL environments, we have improved the RL environment simulation speed across different hardware setups, ranging from a laptop and a modest workstation, to a high-end machine such as NVIDIA DGX-A100. On a high-end machine, EnvPool achieves one million frames per second for the environment execution on Atari environments and three million frames per second on MuJoCo environments. When running EnvPool on a laptop, the speed is 2. 8x that of the Python subprocess. Moreover, great compatibility with existing RL training libraries has been demonstrated in the open-sourced community, including CleanRL, rl_games, DeepMind Acme, etc. Finally, EnvPool allows researchers to iterate their ideas at a much faster pace and has great potential to become the de facto RL environment execution engine. Example runs show that it only takes five minutes to train agents to play Atari Pong and MuJoCo Ant on a laptop. EnvPool is open-sourced at https: //github. com/sail-sg/envpool.

PDF Details

TCS Journal 2022 Journal Article

Some geometrical and topological properties of DNNs' decision boundaries

  • Bo Liu
  • Mengya Shen

Geometry and topology of decision regions are closely related with classification performance and robustness against adversarial attacks. In this paper, we use differential geometry to theoretically explore the geometrical and topological properties of decision regions produced by deep neural networks (DNNs). The goal is to obtain some geometrical and topological properties of decision boundaries for given DNN models, and provide some principled guidance to design and regularization of DNNs. First, we present the curvatures of decision boundaries in terms of network parameters, and give sufficient conditions on network parameters for producing flat or developable decision boundaries. Based on the Gauss-Bonnet-Chern theorem in differential geometry, we then propose a method to compute the Euler characteristics of compact decision boundaries, and verify it with experiments.

Details DOI

NeurIPS Conference 2021 Conference Paper

Conflict-Averse Gradient Descent for Multi-task learning

  • Bo Liu
  • Xingchao Liu
  • Xiaojie Jin
  • Peter Stone
  • Qiang Liu

The goal of multi-task learning is to enable more efficient learning than single task learning by sharing model structures for a diverse set of tasks. A standard multi-task learning objective is to minimize the average loss across all tasks. While straightforward, using this objective often results in much worse final performance for each task than learning them independently. A major challenge in optimizing a multi-task model is the conflicting gradients, where gradients of different task objectives are not well aligned so that following the average gradient direction can be detrimental to specific tasks' performance. Previous work has proposed several heuristics to manipulate the task gradients for mitigating this problem. But most of them lack convergence guarantee and/or could converge to any Pareto-stationary point. In this paper, we introduce Conflict-Averse Gradient descent (CAGrad) which minimizes the average loss function, while leveraging the worst local improvement of individual tasks to regularize the algorithm trajectory. CAGrad balances the objectives automatically and still provably converges to a minimum over the average loss. It includes the regular gradient descent (GD) and the multiple gradient descent algorithm (MGDA) in the multi-objective optimization (MOO) literature as special cases. On a series of challenging multi-task supervised learning and reinforcement learning tasks, CAGrad achieves improved performance over prior state-of-the-art multi-objective gradient manipulation methods.

PDF Details

JBHI Journal 2021 Journal Article

Deep Learning for Hemorrhagic Lesion Detection and Segmentation on Brain CT Images

  • Lu Li
  • Meng Wei
  • Bo Liu
  • Kunakorn Atchaneeyasakul
  • Fugen Zhou
  • Zehao Pan
  • Shimran A. Kumar
  • Jason Y. Zhang

Stroke is an acute cerebral vascular disease that is likely to cause long-term disabilities and death. Immediate emergency care with accurate diagnosis of computed tomographic (CT) images is crucial for dealing with a hemorrhagic stroke. However, due to the high variability of a stroke's location, contrast, and shape, it is challenging and time-consuming even for experienced radiologists to locate them. In this paper, we propose a U-net based deep learning framework to automatically detect and segment hemorrhage strokes in CT brain images. The input of the network is built by concatenating the flipped image with the original CT slice which introduces symmetry constraints of the brain images into the proposed model. This enhances the contrast between hemorrhagic area and normal brain tissue. Various Deep Learning topologies are compared by varying the layers, batch normalization, dilation rates, and pre-train models. This could increase the respective filed and preserves more information on lesion characteristics. Besides, the adversarial training is also adopted in the proposed network to improve the accuracy of the segmentation. The proposed model is trained and evaluated on two different datasets, which achieve the competitive performance with human experts with the highest location accuracy 0. 9859 for detection, 0. 8033 Dice score, and 0. 6919 IoU for segmentation. The results demonstrate the effectiveness, robustness, and advantages of the proposed deep learning model in automatically hemorrhage lesion diagnosis, which make it possible to be a clinical decision support tool in stroke diagnosis.

Details DOI

AAMAS Conference 2021 Conference Paper

Learning Correlated Communication Topology in Multi-Agent Reinforcement learning

  • Yali Du
  • Bo Liu
  • Vincent Moens
  • Ziqi Liu
  • Zhicheng Ren
  • Jun Wang
  • Xu Chen
  • Haifeng Zhang

Communication improves the efficiency and convergence of multiagent learning. Existing study of agent communication has been limited on predefined fixed connections. While an attention mechanism exists and is useful for scheduling the communication between agents, it, however, largely ignores the dynamical nature of communication and thus the correlation between agents’ connections. In this work, we adopt a normalizing flow to encode correlation between agents interactions. The dynamical communication topology is directly learned by maximizing the agent rewards. In our end-to-end formulation, the communication structure is learned by considering it as a hidden dynamical variable. We realize centralized training of critics and graph reasoning policy, and decentralized execution from local observation and message that are received through the learned dynamical communication topology. Experiments on cooperative navigation in the particle world and adaptive traffic control tasks demonstrate the effectiveness of our method.

PDF

AAAI Conference 2021 Conference Paper

LRSC: Learning Representations for Subspace Clustering

  • Changsheng Li
  • Chen Yang
  • Bo Liu
  • Ye Yuan
  • Guoren Wang

Deep learning based subspace clustering methods have attracted increasing attention in recent years, where a basic theme is to non-linearly map data into a latent space, and then uncover subspace structures based upon the data selfexpressiveness property. However, almost all existing deep subspace clustering methods only rely on target domain data, and always resort to shallow neural networks for modeling data, leaving huge room to design more effective representation learning mechanisms tailored for subspace clustering. In this paper, we propose a novel subspace clustering framework through learning precise sample representations. In contrast to previous approaches, the proposed method aims to leverage external data through constructing lots of relevant tasks to guide the training of the encoder, motivated by the idea of meta-learning. Considering limited networks layers of current deep subspace clustering models, we intend to distill knowledge from a deeper network trained on the external data, and transfer it into the shallower model. To reach the above two goals, we propose a new loss function to realize them in a unified framework. Moreover, we propose to construct a new auxiliary task for self-supervised training of the model, such that the representation ability of the model can be further improved. Extensive experiments are performed on four publicly available datasets, and experimental results clearly demonstrate the efficacy of our method, compared to state-of-the-art methods.

PDF Details

NeurIPS Conference 2021 Conference Paper

Machine versus Human Attention in Deep Reinforcement Learning Tasks

  • Suna (Sihang) Guo
  • Ruohan Zhang
  • Bo Liu
  • Yifeng Zhu
  • Dana Ballard
  • Mary Hayhoe
  • Peter Stone

Deep reinforcement learning (RL) algorithms are powerful tools for solving visuomotor decision tasks. However, the trained models are often difficult to interpret, because they are represented as end-to-end deep neural networks. In this paper, we shed light on the inner workings of such trained models by analyzing the pixels that they attend to during task execution, and comparing them with the pixels attended to by humans executing the same tasks. To this end, we investigate the following two questions that, to the best of our knowledge, have not been previously studied. 1) How similar are the visual representations learned by RL agents and humans when performing the same task? and, 2) How do similarities and differences in these learned representations explain RL agents' performance on these tasks? Specifically, we compare the saliency maps of RL agents against visual attention models of human experts when learning to play Atari games. Further, we analyze how hyperparameters of the deep RL algorithm affect the learned representations and saliency maps of the trained agents. The insights provided have the potential to inform novel algorithms for closing the performance gap between human experts and RL agents.

PDF Details

AAAI Conference 2021 Conference Paper

Mean-Variance Policy Iteration for Risk-Averse Reinforcement Learning

  • Shangtong Zhang
  • Bo Liu
  • Shimon Whiteson

We present a mean-variance policy iteration (MVPI) framework for risk-averse control in a discounted infinite horizon MDP optimizing the variance of a per-step reward random variable. MVPI enjoys great flexibility in that any policy evaluation method and risk-neutral control method can be dropped in for risk-averse control off the shelf, in both on- and off-policy settings. This flexibility reduces the gap between risk-neutral control and risk-averse control and is achieved by working on a novel augmented MDP directly. We propose risk-averse TD3 as an example instantiating MVPI, which outperforms vanilla TD3 and many previous riskaverse control methods in challenging Mujoco robot simulation tasks under a risk-aware performance metric. This riskaverse TD3 is the first to introduce deterministic policies and off-policy learning into risk-averse reinforcement learning, both of which are key to the performance boost we show in Mujoco domains.

PDF Details

NeurIPS Conference 2021 Conference Paper

Neural Auto-Curricula in Two-Player Zero-Sum Games

  • Xidong Feng
  • Oliver Slumbers
  • Ziyu Wan
  • Bo Liu
  • Stephen McAleer
  • Ying Wen
  • Jun Wang
  • Yaodong Yang

When solving two-player zero-sum games, multi-agent reinforcement learning (MARL) algorithms often create populations of agents where, at each iteration, a new agent is discovered as the best response to a mixture over the opponent population. Within such a process, the update rules of "who to compete with" (i. e. , the opponent mixture) and "how to beat them" (i. e. , finding best responses) are underpinned by manually developed game theoretical principles such as fictitious play and Double Oracle. In this paper, we introduce a novel framework—Neural Auto-Curricula (NAC)—that leverages meta-gradient descent to automate the discovery of the learning update rule without explicit human design. Specifically, we parameterise the opponent selection module by neural networks and the best-response module by optimisation subroutines, and update their parameters solely via interaction with the game engine, where both players aim to minimise their exploitability. Surprisingly, even without human design, the discovered MARL algorithms achieve competitive or even better performance with the state-of-the-art population-based game solvers (e. g. , PSRO) on Games of Skill, differentiable Lotto, non-transitive Mixture Games, Iterated Matching Pennies, and Kuhn Poker. Additionally, we show that NAC is able to generalise from small games to large games, for example training on Kuhn Poker and outperforming PSRO on Leduc Poker. Our work inspires a promising future direction to discover general MARL algorithms solely from data.

PDF Details

JBHI Journal 2021 Journal Article

scASK: A Novel Ensemble Framework for Classifying Cell Types Based on Single-cell RNA-seq Data

  • Bo Liu
  • Fang-Xiang Wu
  • Xiufen Zou

The Human Cell Atlas (HCA) is a large project that aims to identify all cell types in the human body. The dimension reduction and clustering for identification of cell types from single-cell RNA-sequencing (scRNA-seq) data have become foundational approaches to HCA. The major challenges of current computational analyses are of poor performance on large scale data and sensitive to initial data. We present a new ensemble framework called Adaptive Slice KNNs (scASK) to address the challenges for analyzing scRNA-seq data with high dimensionality. scASK consists of three innovational modules, called DAS (Data Adaptive Slicing), MCS (Meta Classifiers Selecting) and EMS (Ensemble Mode Switching), respectively, which facilitate scASK to approximate a bias-variance tradeoff beyond classification. Thirteen real scRNA-seq datasets are used to evaluate the performance of scASK. Compared with five popular classification algorithms, our experimental results indicate that scASK achieves the best accuracy and robustness among all competing methods. In conclusion, adaptive slicing is an effective structural reduction procedure, and meanwhile scASK provides novel and robust ensemble framework especially for classifying cell types based on scRNA-seq data. scASK is now publically available at https://github.com/liubo2358/scASKcmd.

Details DOI

YNICL Journal 2020 Journal Article

Brain GABA+ changes in primary hypothyroidism patients before and after levothyroxine treatment: A longitudinal magnetic resonance spectroscopy study

  • Bo Liu
  • Zhensong Wang
  • Liangjie Lin
  • Huan Yang
  • Fei Gao
  • Tao Gong
  • Richard A.E. Edden
  • Guangbin Wang

OBJECTIVE: Increasing evidence indicates the involvement of the GABAergic system in the pathophysiology of hypothyroidism. We aimed to investigate longitudinal changes of brain GABA in primary hypothyroidism before and after levothyroxine (L-T4) treatment. MATERIAL AND METHODS: In 18 patients with hypothyroidism, we used the MEGA-PRESS (Mescher-Garwood point-resolved spectroscopy) editing sequence to measure brain GABA levels from medial prefrontal cortex (mPFC) and posterior cingulate cortex (PCC) at baseline and after 6-months of L-T4 treatment. Sex- and age-matched healthy controls (n = 18) were scanned at baseline. Thyroid function and neuropsychological tests were also performed. RESULTS: GABA signals were successfully quantified from all participants with fitting errors lower than 15%. GABA signal was labeled as GABA+ due to contamination from co-edited macromoleculars and homocarnosine. In hypothyroid patients, mean GABA+ was significantly lower in the mPFC region compared with controls (p = 0.031), and the mPFC GABA+ measurements were significantly correlated with depressive symptoms and memory function (r = -0.558, p = 0.016; r = 0.522, p = 0.026, respectively). After adequate L-T4 treatment, the mPFC GABA+ in hypothyroid patients increased to normal level, along with relieved neuropsychological impairments. CONCLUSION: The study suggested the decrease of GABA+ may be an important neurobiological factor in the pathophysiology of hypothyroidism. Treatment of L-T4 may reverse the abnormal GABA+ and hypothyroidism-induced neuropsychiatric impairments, indicating the action mode of L-T4 in adjunctive treatment of affective disorders.

Details DOI

AAAI Conference 2020 Conference Paper

Deep Object Co-Segmentation via Spatial-Semantic Network Modulation

  • Kaihua Zhang
  • Jin Chen
  • Bo Liu
  • Qingshan Liu

Object co-segmentation is to segment the shared objects in multiple relevant images, which has numerous applications in computer vision. This paper presents a spatial and semantic modulated deep network framework for object cosegmentation. A backbone network is adopted to extract multi-resolution image features. With the multi-resolution features of the relevant images as input, we design a spatial modulator to learn a mask for each image. The spatial modulator captures the correlations of image feature descriptors via unsupervised learning. The learned mask can roughly localize the shared foreground object while suppressing the background. For the semantic modulator, we model it as a supervised image classification task. We propose a hierarchical second-order pooling module to transform the image features for classification use. The outputs of the two modulators manipulate the multi-resolution features by a shiftand-scale operation so that the features focus on segmenting co-object regions. The proposed model is trained end-to-end without any intricate post-processing. Extensive experiments on four image co-segmentation benchmark datasets demonstrate the superior accuracy of the proposed method compared to state-of-the-art methods. The codes are available at http: //kaihuazhang. net/.

PDF Details

JMLR Journal 2020 Journal Article

Dual Iterative Hard Thresholding

  • Xiao-Tong Yuan
  • Bo Liu
  • Lezi Wang
  • Qingshan Liu
  • Dimitris N. Metaxas

Iterative Hard Thresholding (IHT) is a popular class of first-order greedy selection methods for loss minimization under cardinality constraint. The existing IHT-style algorithms, however, are proposed for minimizing the primal formulation. It is still an open issue to explore duality theory and algorithms for such a non-convex and NP-hard combinatorial optimization problem. To address this issue, we develop in this article a novel duality theory for $\ell_2$-regularized empirical risk minimization under cardinality constraint, along with an IHT-style algorithm for dual optimization. Our sparse duality theory establishes a set of sufficient and/or necessary conditions under which the original non-convex problem can be equivalently or approximately solved in a concave dual formulation. In view of this theory, we propose the Dual IHT (DIHT) algorithm as a super-gradient ascent method to solve the non-smooth dual problem with provable guarantees on primal-dual gap convergence and sparsity recovery. Numerical results confirm our theoretical predictions and demonstrate the superiority of DIHT to the state-of-the-art primal IHT-style algorithms in model estimation accuracy and computational efficiency. [abs] [ pdf ][ bib ] &copy JMLR 2020. ( edit, beta )

PDF Details

IJCAI Conference 2020 Conference Paper

DualSMC: Tunneling Differentiable Filtering and Planning under Continuous POMDPs

  • Yunbo Wang
  • Bo Liu
  • Jiajun Wu
  • Yuke Zhu
  • Simon S. Du
  • Li Fei-Fei
  • Joshua B. Tenenbaum

A major difficulty of solving continuous POMDPs is to infer the multi-modal distribution of the unobserved true states and to make the planning algorithm dependent on the perceived uncertainty. We cast POMDP filtering and planning problems as two closely related Sequential Monte Carlo (SMC) processes, one over the real states and the other over the future optimal trajectories, and combine the merits of these two parts in a new model named the DualSMC network. In particular, we first introduce an adversarial particle filter that leverages the adversarial relationship between its internal components. Based on the filtering results, we then propose a planning algorithm that extends the previous SMC planning approach [Piche et al. , 2018] to continuous POMDPs with an uncertainty-dependent policy. Crucially, not only can DualSMC handle complex observations such as image input but also it remains highly interpretable. It is shown to be effective in three continuous POMDP domains: the floor positioning domain, the 3D light-dark navigation domain, and a modified Reacher domain.

PDF Details DOI

NeurIPS Conference 2020 Conference Paper

Firefly Neural Architecture Descent: a General Approach for Growing Neural Networks

  • Lemeng Wu
  • Bo Liu
  • Peter Stone
  • Qiang Liu

We propose firefly neural architecture descent, a general framework for progressively and dynamically growing neural networks to jointly optimize the networks' parameters and architectures. Our method works in a steepest descent fashion, which iteratively finds the best network within a functional neighborhood of the original network that includes a diverse set of candidate network structures. By using Taylor approximation, the optimal network structure in the neighborhood can be found with a greedy selection procedure. We show that firefly descent can flexibly grow networks both wider and deeper, and can be applied to learn accurate but resource-efficient neural architectures that avoid catastrophic forgetting in continual learning. Empirically, firefly descent achieves promising results on both neural architecture search and continual learning. In particular, on a challenging continual image classification task, it learns networks that are smaller in size but have higher average accuracy than those learned by the state-of-the-art methods.

PDF Details

IJCAI Conference 2020 Conference Paper

Human Gaze Assisted Artificial Intelligence: A Review

  • Ruohan Zhang
  • Akanksha Saran
  • Bo Liu
  • Yifeng Zhu
  • Sihang Guo
  • Scott Niekum
  • Dana Ballard
  • Mary Hayhoe

Human gaze reveals a wealth of information about internal cognitive state. Thus, gaze-related research has significantly increased in computer vision, natural language processing, decision learning, and robotics in recent years. We provide a high-level overview of the research efforts in these fields, including collecting human gaze data sets, modeling gaze behaviors, and utilizing gaze information in various applications, with the goal of enhancing communication between these research areas. We discuss future challenges and potential applications that work towards a common goal of human-centered artificial intelligence.

PDF Details DOI

AAAI Conference 2020 Conference Paper

Object-Guided Instance Segmentation for Biological Images

  • Jingru Yi
  • Hui Tang
  • Pengxiang Wu
  • Bo Liu
  • Daniel J. Hoeppner
  • Dimitris N. Metaxas
  • Lianyi Han
  • Wei Fan

Instance segmentation of biological images is essential for studying object behaviors and properties. The challenges, such as clustering, occlusion, and adhesion problems of the objects, make instance segmentation a non-trivial task. Current box-free instance segmentation methods typically rely on local pixel-level information. Due to a lack of global object view, these methods are prone to over- or undersegmentation. On the contrary, the box-based instance segmentation methods incorporate object detection into the segmentation, performing better in identifying the individual instances. In this paper, we propose a new box-based instance segmentation method. Mainly, we locate the object bounding boxes from their center points. The object features are subsequently reused in the segmentation branch as a guide to separate the clustered instances within an RoI patch. Along with the instance normalization, the model is able to recover the target object distribution and suppress the distribution of neighboring attached objects. Consequently, the proposed model performs excellently in segmenting the clustered objects while retaining the target object details. The proposed method achieves state-of-the-art performances on three biological datasets: cell nuclei, plant phenotyping dataset, and neural cells.

PDF Details

AAAI Conference 2020 Conference Paper

Robust Conditional GAN from Uncertainty-Aware Pairwise Comparisons

  • Ligong Han
  • Ruijiang Gao
  • Mun Kim
  • Xin Tao
  • Bo Liu
  • Dimitris Metaxas

Conditional generative adversarial networks have shown exceptional generation performance over the past few years. However, they require large numbers of annotations. To address this problem, we propose a novel generative adversarial network utilizing weak supervision in the form of pairwise comparisons (PC-GAN) for image attribute editing. In the light of Bayesian uncertainty estimation and noise-tolerant adversarial training, PC-GAN can estimate attribute rating ef- ficiently and demonstrate robust performance in noise resistance. Through extensive experiments, we show both qualitatively and quantitatively that PC-GAN performs comparably with fully-supervised methods and outperforms unsupervised baselines. Code and Supplementary can be found on the project website∗.

PDF Details

NeurIPS Conference 2020 Conference Paper

Towards Playing Full MOBA Games with Deep Reinforcement Learning

  • Deheng Ye
  • Guibin Chen
  • Wen Zhang
  • Sheng Chen
  • Bo Yuan
  • Bo Liu
  • Jia Chen
  • Zhao Liu

MOBA games, e. g. , Honor of Kings, League of Legends, and Dota 2, pose grand challenges to AI systems such as multi-agent, enormous state-action space, complex action control, etc. Developing AI for playing MOBA games has raised much attention accordingly. However, existing work falls short in handling the raw game complexity caused by the explosion of agent combinations, i. e. , lineups, when expanding the hero pool in case that OpenAI's Dota AI limits the play to a pool of only 17 heroes. As a result, full MOBA games without restrictions are far from being mastered by any existing AI system. In this paper, we propose a MOBA AI learning paradigm that methodologically enables playing full MOBA games with deep reinforcement learning. Specifically, we develop a combination of novel and existing learning techniques, including off-policy adaption, multi-head value estimation, curriculum self-play learning, policy distillation, and Monte-Carlo tree-search, in training and playing a large pool of heroes, meanwhile addressing the scalability issue skillfully. Tested on Honor of Kings, a popular MOBA game, we show how to build superhuman AI agents that can defeat top esports players. The superiority of our AI is demonstrated by the first large-scale performance test of MOBA AI agent in the literature.

PDF Details

AAAI Conference 2020 Conference Paper

Zero-Shot Learning from Adversarial Feature Residual to Compact Visual Feature

  • Bo Liu
  • Qiulei Dong
  • Zhanyi Hu

Recently, many zero-shot learning (ZSL) methods focused on learning discriminative object features in an embedding feature space, however, the distributions of the unseen-class features learned by these methods are prone to be partly overlapped, resulting in inaccurate object recognition. Addressing this problem, we propose a novel adversarial network to synthesize compact semantic visual features for ZSL, consisting of a residual generator, a prototype predictor, and a discriminator. The residual generator is to generate the visual feature residual, which is integrated with a visual prototype predicted via the prototype predictor for synthesizing the visual feature. The discriminator is to distinguish the synthetic visual features from the real ones extracted from an existing categorization CNN. Since the generated residuals are generally numerically much smaller than the distances among all the prototypes, the distributions of the unseen-class features synthesized by the proposed network are less overlapped. In addition, considering that the visual features from categorization CNNs are generally inconsistent with their semantic features, a simple feature selection strategy is introduced for extracting more compact semantic visual features. Extensive experimental results on six benchmark datasets demonstrate that our method could achieve a significantly better performance than existing state-of-the-art methods by ∼1. 2-13. 2% in most cases.

PDF Details

AAAI Conference 2019 Short Paper

Logic-Based Sequential Decision-Making

  • Daoming Lyu
  • Fangkai Yang
  • Bo Liu
  • Daesub Yoon

Deep reinforcement learning (DRL) has gained great success by learning directly from high-dimensional sensory inputs, yet is notorious for the lack of interpretability. Interpretability of the subtasks is critical in hierarchical decision-making as it increases the transparency of black-box-style DRL approach and helps the RL practitioners to understand the high-level behavior of the system better. In this paper, we introduce symbolic planning into DRL and propose a framework of Symbolic Deep Reinforcement Learning (SDRL) that can handle both high-dimensional sensory inputs and symbolic planning. The task-level interpretability is enabled by relating symbolic actions to options. This framework features a planner – controller – meta-controller architecture, which takes charge of subtask scheduling, data-driven subtask learning, and subtask evaluation, respectively. The three components cross-fertilize each other and eventually converge to an optimal symbolic plan along with the learned subtasks, bringing together the advantages of long-term planning capability with symbolic knowledge and end-to-end reinforcement learning directly from a high-dimensional sensory input. Experimental results validate the interpretability of subtasks, along with improved data efficiency compared with state-of-the-art approaches.

PDF Details

AAMAS Conference 2019 Conference Paper

Optimal Control of Complex Systems through Variational Inference with a Discrete Event Decision Process

  • Fan Yang
  • Bo Liu
  • Wen Dong

Complex social systems are composed of interconnected individuals whose interactions result in group behaviors. Optimal control of a real-world complex system has many applications, including road traffic management, epidemic prevention, and information dissemination. However, such real-world complex system control is difficult to achieve because of high-dimensional and non-linear system dynamics, and the exploding state and action spaces for the decision maker. Prior methods can be divided into two categories: simulation-based and analytical approaches. Existing simulation approaches have high-variance in Monte Carlo integration, and the analytical approaches suffer from modeling inaccuracy. We adopted simulation modeling in specifying the complex dynamics of a complex system, and developed analytical solutions for searching optimal strategies in a complex network with high-dimensional state-action space. To capture the complex system dynamics, we formulate the complex social network decision making problem as a discrete event decision process. To address the curse of dimensionality and search in high-dimensional state action spaces in complex systems, we reduce control of a complex system to variational inference and parameter learning, introduce Bethe entropy approximation, and develop an expectation propagation algorithm. Our proposed algorithm leads to higher system expected rewards, faster convergence, and lower variance of value function in a realworld transportation scenario than state-of-the-art analytical and sampling approaches.

PDF

AAAI Conference 2019 Conference Paper

SDRL: Interpretable and Data-Efficient Deep Reinforcement Learning Leveraging Symbolic Planning

  • Daoming Lyu
  • Fangkai Yang
  • Bo Liu
  • Steven Gustafson

Deep reinforcement learning (DRL) has gained great success by learning directly from high-dimensional sensory inputs, yet is notorious for the lack of interpretability. Interpretability of the subtasks is critical in hierarchical decision-making as it increases the transparency of black-box-style DRL approach and helps the RL practitioners to understand the high-level behavior of the system better. In this paper, we introduce symbolic planning into DRL and propose a framework of Symbolic Deep Reinforcement Learning (SDRL) that can handle both high-dimensional sensory inputs and symbolic planning. The task-level interpretability is enabled by relating symbolic actions to options. This framework features a planner – controller – meta-controller architecture, which takes charge of subtask scheduling, data-driven subtask learning, and subtask evaluation, respectively. The three components cross-fertilize each other and eventually converge to an optimal symbolic plan along with the learned subtasks, bringing together the advantages of long-term planning capability with symbolic knowledge and end-to-end reinforcement learning directly from a high-dimensional sensory input. Experimental results validate the interpretability of subtasks, along with improved data efficiency compared with state-of-the-art approaches.

PDF Details

YNICL Journal 2019 Journal Article

Transcutaneous auricular vagus nerve stimulation at 1 Hz modulates locus coeruleus activity and resting state functional connectivity in patients with migraine: An fMRI study

  • Yue Zhang
  • Jiao Liu
  • Hui Li
  • Zhaoxian Yan
  • Xian Liu
  • Jin Cao
  • Joel Park
  • Georgia Wilson

BACKGROUND: Migraine is a common episodic neurological disorder. Literature has shown that transcutaneous auricular vagus nerve stimulation (taVNS) at 1 Hz can significantly relieve migraine symptoms. However, its underlying mechanism remains unclear. This study aims to investigate the neural pathways associated with taVNS treatment of migraine. METHODS: Twenty-nine patients with migraine were recruited from outpatient neurology clinics. Each patient attended two magnetic resonance imaging/functional magnetic resonance imaging (MRI/fMRI) scan sessions separated by one week. Each session included a pre-stimulation resting state fMRI scan, fMRI scans during real or sham 1 Hz taVNS (with block design), and a post-stimulation resting state fMRI scan. RESULTS: Twenty-six patients were included in the final analyses. Real taVNS evoked fMRI signal decreases in brain areas belonging to the default mode network (DMN) and brain stem areas including the locus coeruleus (LC), raphe nuclei, parabrachial nucleus, and solitary nucleus. Sham taVNS evoked fMRI signal decreases in brain areas belonging to the DMN. Compared to sham taVNS, real taVNS produced greater deactivation at the bilateral LC. Resting state functional connectivity (rsFC) analysis showed that after taVNS, LC rsFC with the right temporoparietal junction and left secondary somatosensory cortex (S2) significantly increased compared to sham taVNS. The increased rsFC of the left LC-left S2 was significantly negatively associated with the frequency of migraine attacks during the preceding month. CONCLUSION: Our results suggest that taVNS at 1 Hz can significantly modulate activity/connectivity of brain regions associated with the vagus nerve central pathway and pain modulation system, which may shed light on the neural mechanisms underlying taVNS treatment of migraine.

Details DOI

NeurIPS Conference 2018 Conference Paper

A Block Coordinate Ascent Algorithm for Mean-Variance Optimization

  • Tengyang Xie
  • Bo Liu
  • Yangyang Xu
  • Mohammad Ghavamzadeh
  • Yinlam Chow
  • Daoming Lyu
  • Daesub Yoon

Risk management in dynamic decision problems is a primary concern in many fields, including financial investment, autonomous driving, and healthcare. The mean-variance function is one of the most widely used objective functions in risk management due to its simplicity and interpretability. Existing algorithms for mean-variance optimization are based on multi-time-scale stochastic approximation, whose learning rate schedules are often hard to tune, and have only asymptotic convergence proof. In this paper, we develop a model-free policy search framework for mean-variance optimization with finite-sample error bound analysis (to local optima). Our starting point is a reformulation of the original mean-variance function with its Fenchel dual, from which we propose a stochastic block coordinate ascent policy search algorithm. Both the asymptotic convergence guarantee of the last iteration's solution and the convergence rate of the randomly picked solution are provided, and their applicability is demonstrated on several benchmark domains.

PDF Details

IJCAI Conference 2018 Conference Paper

PEORL: Integrating Symbolic Planning and Hierarchical Reinforcement Learning for Robust Decision-Making

  • Fangkai Yang
  • Daoming Lyu
  • Bo Liu
  • Steven Gustafson

Reinforcement learning and symbolic planning have both been used to build intelligent autonomous agents. Reinforcement learning relies on learning from interactions with real world, which often requires an unfeasibly large amount of experience. Symbolic planning relies on manually crafted symbolic knowledge, which may not be robust to domain uncertainties and changes. In this paper we present a unified framework PEORL that integrates symbolic planning with hierarchical reinforcement learning (HRL) to cope with decision-making in dynamic environment with uncertainties. Symbolic plans are used to guide the agent's task execution and learning, and the learned experience is fed back to symbolic knowledge to improve planning. This method leads to rapid policy search and robust symbolic plans in complex domains. The framework is tested on benchmark domains of HRL.

PDF Details

JAIR Journal 2018 Journal Article

Proximal Gradient Temporal Difference Learning: Stable Reinforcement Learning with Polynomial Sample Complexity

  • Bo Liu
  • Ian Gemp
  • Mohammad Ghavamzadeh
  • Ji Liu
  • Sridhar Mahadevan
  • Marek Petrik

In this paper, we introduce proximal gradient temporal difference learning, which provides a principled way of designing and analyzing true stochastic gradient temporal difference learning algorithms. We show how gradient TD (GTD) reinforcement learning methods can be formally derived, not by starting from their original objective functions, as previously attempted, but rather from a primal-dual saddle-point objective function. We also conduct a saddle-point error analysis to obtain finite-sample bounds on their performance. Previous analyses of this class of algorithms use stochastic approximation techniques to prove asymptotic convergence, and do not provide any finite-sample analysis. We also propose an accelerated algorithm, called GTD2-MP, that uses proximal "mirror maps" to yield an improved convergence rate. The results of our theoretical analysis imply that the GTD family of algorithms are comparable and may indeed be preferred over existing least squares TD methods for off-policy learning, due to their linear complexity. We provide experimental results showing the improved performance of our accelerated gradient TD methods.

PDF Details DOI

AAAI Conference 2018 Conference Paper

Transferable Contextual Bandit for Cross-Domain Recommendation

  • Bo Liu
  • Ying Wei
  • Yu Zhang
  • Zhixian Yan
  • Qiang Yang

Traditional recommendation systems (RecSys) suffer from two problems: the exploitation-exploration dilemma and the cold-start problem. One solution to solving the exploitationexploration dilemma is the contextual bandit policy, which adaptively exploits and explores user interests. As a result, the contextual bandit policy achieves increased rewards in the long run. The contextual bandit policy, however, may cause the system to explore more than needed in the cold-start situations, which can lead to worse short-term rewards. Crossdomain RecSys methods adopt transfer learning to leverage prior knowledge in a source RecSys domain to jump start the cold-start target RecSys. To solve the two problems together, in this paper, we propose the first applicable transferable contextual bandit (TCB) policy for the cross-domain recommendation. TCB not only benefits the exploitation but also accelerates the exploration in the target RecSys. TCB’s exploration, in turn, helps to learn how to transfer between different domains. TCB is a general algorithm for both homogeneous and heterogeneous domains. We perform both theoretical regret analysis and empirical experiments. The empirical results show that TCB outperforms the state-of-the-art algorithms over time.

PDF Details

EAAI Journal 2017 Journal Article

An improved TLBO based memetic algorithm for aerodynamic shape optimization

  • Xinghua Qu
  • Ran Zhang
  • Bo Liu
  • Huifeng Li

Aerodynamic shape optimization (ASO) for aircraft is the focus of concern as well as the subject of substantial research issue in aerospace engineering. This paper proposes a novel TLBO (teaching-learning based optimization based) memetic algorithm (TLBO-MA) for optimizing the aerodynamic shape. In the proposed TLBO-MA, an adaptive teaching factor, conservation of information inspired operator and multi-meme learning are incorporated to enhance the searching behavior of standard TLBO. Simulation based on well-known benchmarks and ASO for HTV-2 prototype demonstrates the efficiency of the proposed TLBO-MA.

Details DOI

IJCAI Conference 2017 Conference Paper

Deep Neural Networks for High Dimension, Low Sample Size Data

  • Bo Liu
  • Ying Wei
  • Yu Zhang
  • Qiang Yang

Deep neural networks (DNN) have achieved breakthroughs in applications with large sample size. However, when facing high dimension, low sample size (HDLSS) data, such as the phenotype prediction problem using genetic data in bioinformatics, DNN suffers from overfitting and high-variance gradients. In this paper, we propose a DNN model tailored for the HDLSS data, named Deep Neural Pursuit (DNP). DNP selects a subset of high dimensional features for the alleviation of overfitting and takes the average over multiple dropouts to calculate gradients with low variance. As the first DNN method applied on the HDLSS data, DNP enjoys the advantages of the high nonlinearity, the robustness to high dimensionality, the capability of learning from a small number of samples, the stability in feature selection, and the end-to-end training. We demonstrate these advantages of DNP via empirical results on both synthetic and real-world biological datasets.

PDF Details

AAAI Conference 2016 Conference Paper

Decentralized Robust Subspace Clustering

  • Bo Liu
  • Xiao-Tong Yuan
  • Yang Yu
  • Qingshan Liu
  • Dimitris Metaxas

We consider the problem of subspace clustering using the SSC (Sparse Subspace Clustering) approach, which has several desirable theoretical properties and has been shown to be effective in various computer vision applications. We develop a large scale distributed framework for the computation of SSC via an alternating direction method of multiplier (ADMM) algorithm. The proposed framework solves SSC in column blocks and only involves parallel multivariate Lasso regression subproblems and sample-wise operations. This appealing property allows us to allocate multiple cores/machines for the processing of individual column blocks. We evaluate our algorithm on a shared-memory architecture. Experimental results on real-world datasets confirm that the proposed block-wise ADMM framework is substantially more efficient than its matrix counterpart used by SSC, without sacrificing accuracy. Moreover, our approach is directly applicable to decentralized neighborhood selection for Gaussian graphical models structure estimation.

PDF Details

IJCAI Conference 2016 Conference Paper

Proximal Gradient Temporal Difference Learning Algorithms

  • Bo Liu
  • Ji Liu
  • Mohammad Ghavamzadeh
  • Sridhar Mahadevan
  • Marek Petrik

In this paper, we describe proximal gradient temporal difference learning, which provides a principled way for designing and analyzing true stochastic gradient temporal difference learning algorithms. We show how gradient TD (GTD) reinforcement learning methods can be formally derived, not with respect to their original objective functions as previously attempted, but rather with respect to primal-dual saddle-point objective functions. We also conduct a saddle-point error analysis to obtain finite-sample bounds on their performance. Previous analyses of this class of algorithms use stochastic approximation techniques to prove asymptotic convergence, and no finite-sample analysis had been attempted. An accelerated algorithm is also proposed, namely GTD2-MP, which use proximal mirror maps to yield acceleration. The results of our theoretical analysis imply that the GTD family of algorithms are comparable and may indeed be preferred over existing least squares TD methods for off-policy learning, due to their linear complexity. We provide experimental results showing the improved performance of our accelerated gradient TD methods.

PDF Details

YNICL Journal 2016 Journal Article

Repeated acupuncture treatments modulate amygdala resting state functional connectivity of depressive patients

  • Xiaoyun Wang
  • Zengjian Wang
  • Jian Liu
  • Jun Chen
  • Xian Liu
  • Guangning Nie
  • Joon-Seok Byun
  • Yilin Liang

As a widely-applied alternative therapy, acupuncture is gaining popularity in Western society. One challenge that remains, however, is incorporating it into mainstream medicine. One solution is to combine acupuncture with other conventional, mainstream treatments. In this study, we investigated the combination effect of acupuncture and the antidepressant fluoxetine, as well as its underlying mechanism using resting state functional connectivity (rsFC) in patients with major depressive disorders. Forty-six female depressed patients were randomized into a verum acupuncture plus fluoxetine or a sham acupuncture plus fluoxetine group for eight weeks. Resting-state fMRI data was collected before the first and last treatments. Results showed that compared with those in the sham acupuncture treatment, verum acupuncture treatment patients showed 1) greater clinical improvement as indicated by Montgomery-Åsberg Depression Rating Scale (MADRS) and Self-Rating Depression Scale (SDS) scores; 2) increased rsFC between the left amygdala and subgenual anterior cingulate cortex (sgACC)/preguenual anterior cingulate cortex (pgACC); 3) increased rsFC between the right amygdala and left parahippocampus (Para)/putamen (Pu). The strength of the amygdala-sgACC/pgACC rsFC was positively associated with corresponding clinical improvement (as indicated by a negative correlation with MADRS and SDS scores). Our findings demonstrate the additive effect of acupuncture to antidepressant treatment and suggest that this effect may be achieved through the limbic system, especially the amygdala and the ACC.

Details DOI

AAAI Conference 2016 Conference Paper

Uncorrelated Group LASSO

  • Deguang Kong
  • Ji Liu
  • Bo Liu
  • Xuan Bao

2, 1-norm is an effective regularization to enforce a simple group sparsity for feature learning. To capture some subtle structures among feature groups, we propose a new regularization called exclusive group 2, 1-norm. It enforces the sparsity at the intra-group level by using 2, 1-norm, while encourages the selected features to distribute in different groups by using 2 norm at the inter-group level. The proposed exclusive group 2, 1-norm is capable of eliminating the feature correlations in the context of feature selection, if highly correlated features are collected in the same groups. To solve the generic exclusive group 2, 1-norm regularized problems, we propose an efficient iterative re-weighting algorithm and provide a rigorous convergence analysis. Experiment results on real world datasets demonstrate the effectiveness of the proposed new regularization and algorithm.

PDF Details

YNIMG Journal 2015 Journal Article

Decreased auditory GABA+ concentrations in presbycusis demonstrated by edited magnetic resonance spectroscopy

  • Fei Gao
  • Guangbin Wang
  • Wen Ma
  • Fuxin Ren
  • Muwei Li
  • Yuling Dong
  • Cheng Liu
  • Bo Liu

Gamma-aminobutyric acid (GABA) is the main inhibitory neurotransmitter in the central auditory system. Altered GABAergic neurotransmission has been found in both the inferior colliculus and the auditory cortex in animal models of presbycusis. Edited magnetic resonance spectroscopy (MRS), using the MEGA-PRESS sequence, is the most widely used technique for detecting GABA in the human brain. However, to date there has been a paucity of studies exploring changes to the GABA concentrations in the auditory region of patients with presbycusis. In this study, sixteen patients with presbycusis (5 males/11 females, mean age 63. 1±2. 6years) and twenty healthy controls (6 males/14 females, mean age 62. 5±2. 3years) underwent audiological and MRS examinations. Pure tone audiometry from 0. 125 to 8kHz and tympanometry were used to assess the hearing abilities of all subjects. The pure tone average (PTA; the average of hearing thresholds at 0. 5, 1, 2 and 4kHz) was calculated. The MEGA-PRESS sequence was used to measure GABA+ concentrations in 4×3×3cm3 volumes centered on the left and right Heschl's gyri. GABA+ concentrations were significantly lower in the presbycusis group compared to the control group (left auditory regions: p =0. 002, right auditory regions: p =0. 008). Significant negative correlations were observed between PTA and GABA+ concentrations in the presbycusis group (r =−0. 57, p =0. 02), while a similar trend was found in the control group (r =−0. 40, p =0. 08). These results are consistent with a hypothesis of dysfunctional GABAergic neurotransmission in the central auditory system in presbycusis and suggest a potential treatment target for presbycusis.

Details DOI

IJCAI Conference 2015 Conference Paper

Image Feature Learning for Cold Start Problem in Display Advertising

  • Kaixiang Mo
  • Bo Liu
  • Lei Xiao
  • Yong Li
  • Jie Jiang

In online display advertising, state-of-the-art Click Through Rate(CTR) prediction algorithms rely heavily on historical information, and they work poorly on growing number of new ads without any historical information. This is known as the the cold start problem. For image ads, current stateof-the-art systems use handcrafted image features such as multimedia features and SIFT features to capture the attractiveness of ads. However, these handcrafted features are task dependent, inflexible and heuristic. In order to tackle the cold start problem in image display ads, we propose a new feature learning architecture to learn the most discriminative image features directly from raw pixels and user feedback in the target task. The proposed method is flexible and does not depend on human heuristic. Extensive experiments on a real world dataset with 47 billion records show that our feature learning method outperforms existing handcrafted features significantly, and it can extract discriminative and meaningful features.

PDF Details

JBHI Journal 2014 Journal Article

Intelligent Closed-Loop Insulin Delivery Systems for ICU Patients

  • Youqing Wang
  • Hongzhi Xie
  • Xu Jiang
  • Bo Liu

Good glycemic control through insulin administration among intensive care unit (ICU) patients can reduce mortality significantly; however, it remains a big challenge because of scarcity of individualized models for ICU patients. To deal with this challenge, a new combination of particle swarm optimization (PSO) and model predictive control (MPC) has been proposed to identify the model online as well as to optimally design the input, i. e. , the insulin delivery rate automatically. According to the population distribution, ten typical linear dynamic models were selected such that any patient's model could be approximated by a linear combination of these ten typical models. PSO was used to update the weight coefficients while MPC was used to design the insulin delivery rate based on the combination model identified by using PSO. The proposed strategy was compared with the Yale protocol on 30 virtual subjects. According to the control-variability grid analysis, the percentage values in A + B zone were, respectively, 100% under the proposed strategy and while 51% under the Yale protocol, which demonstrates the superior performance of the proposed strategy. As a good candidate for the full closed-loop insulin delivery method, this new combination can control the glucose level by bringing it to a safe range promptly thereby reducing the risk of death.

Details DOI

YNIMG Journal 2012 Journal Article

A computational neurodegenerative disease progression score: Method and results with the Alzheimer's disease neuroimaging initiative cohort

  • Bruno M. Jedynak
  • Andrew Lang
  • Bo Liu
  • Elyse Katz
  • Yanwei Zhang
  • Bradley T. Wyman
  • David Raunig
  • C. Pierre Jedynak

While neurodegenerative diseases are characterized by steady degeneration over relatively long timelines, it is widely believed that the early stages are the most promising for therapeutic intervention, before irreversible neuronal loss occurs. Developing a therapeutic response requires a precise measure of disease progression. However, since the early stages are for the most part asymptomatic, obtaining accurate measures of disease progression is difficult. Longitudinal databases of hundreds of subjects observed during several years with tens of validated biomarkers are becoming available, allowing the use of computational methods. We propose a widely applicable statistical methodology for creating a disease progression score (DPS), using multiple biomarkers, for subjects with a neurodegenerative disease. The proposed methodology was evaluated for Alzheimer's disease (AD) using the publicly available AD Neuroimaging Initiative (ADNI) database, yielding an Alzheimer's DPS or ADPS score for each subject and each time-point in the database. In addition, a common description of biomarker changes was produced allowing for an ordering of the biomarkers. The Rey Auditory Verbal Learning Test delayed recall was found to be the earliest biomarker to become abnormal. The group of biomarkers comprising the volume of the hippocampus and the protein concentration amyloid beta and Tau were next in the timeline, and these were followed by three cognitive biomarkers. The proposed methodology thus has potential to stage individuals according to their state of disease progression relative to a population and to deduce common behaviors of biomarkers in the disease itself.

Details DOI

NeurIPS Conference 2012 Conference Paper

Regularized Off-Policy TD-Learning

  • Bo Liu
  • Sridhar Mahadevan
  • Ji Liu

We present a novel $l_1$ regularized off-policy convergent TD-learning method (termed RO-TD), which is able to learn sparse representations of value functions with low computational complexity. The algorithmic framework underlying RO-TD integrates two key ideas: off-policy convergent gradient TD methods, such as TDC, and a convex-concave saddle-point formulation of non-smooth convex optimization, which enables first-order solvers and feature selection using online convex regularization. A detailed theoretical and experimental analysis of RO-TD is presented. A variety of experiments are presented to illustrate the off-policy convergence, sparse feature selection capability and low computational cost of the RO-TD algorithm.

PDF Details

IJCAI Conference 2011 Conference Paper

Similarity-Based Approach for Positive and Unlabeled Learning

  • Yanshan Xiao
  • Bo Liu
  • Jie Yin
  • Longbing Cao
  • Chengqi Zhang
  • Zhifeng Hao

Positive and unlabelled learning (PU learning) has been investigated to deal with the situation where only the positive examples and the unlabelled examples are available. Most of the previous works focus on identifying some negative examples from the unlabelled data, so that the supervised learning methods can be applied to build a classifier. However, for the remaining unlabelled data, which can not be explicitly identified as positive or negative (we call them ambiguous examples), they either exclude them from the training phase or simply enforce them to either class. Consequently, their performance may be constrained. This paper proposes a novel approach, called similarity-based PU learning (SPUL) method, by associating the ambiguous examples with two similarity weights, which indicate the similarity of an ambiguous example towards the positive class and the negative class, respectively. The local similarity-based and global similarity-based mechanisms are proposed to generate the similarity weights. The ambiguous examples and their similarity-weights are thereafter incorporated into an SVM-based learning phase to build a more accurate classifier. Extensive experiments on real-world datasets have shown that SPUL outperforms state-of-the-art PU learning methods.

PDF Details DOI

TIST Journal 2010 Journal Article

Accessible image search for colorblindness

  • Meng Wang
  • Bo Liu
  • Xian-Sheng Hua

This article introduces an intelligent system that accommodates colorblind users in image search. Color plays an important role in the human perception and recognition of images. However, there are about 8% of men and 0.8% of women suffering from colorblindness. We show that the existing image search techniques cannot provide satisfactory results for these users since many images will not be well perceived by them due to the loss of color information. To deal with this difficulty, we introduce a system named Accessible Image Search (AIS) to accommodate these users. Different from the general image search scheme that aims at returning more relevant results, AIS further takes into account the colorblind accessibilities of the returned results, that is, the image qualities in the eyes of colorblind users. The system contains three components: accessibility assessment, accessibility improvement, and color indication. The accessibility assessment component measures the accessibility scores of images, and consequently different reranking methods can be performed to prioritize images with high accessibilities. In the accessibility improvement component, we propose an efficient recoloring algorithm to modify the colors of the images such that they can be better perceived by colorblind users. Color indication aims to indicate the name of the interesting color in an image. We evaluate the introduced system with more than 60 queries and 20 anonymous colorblind users, and the empirical results demonstrate its effectiveness and usefulness.

Details DOI

NeurIPS Conference 2010 Conference Paper

Basis Construction from Power Series Expansions of Value Functions

  • Sridhar Mahadevan
  • Bo Liu

This paper explores links between basis construction methods in Markov decision processes and power series expansions of value functions. This perspective provides a useful framework to analyze properties of existing bases, as well as provides insight into constructing more effective bases. Krylov and Bellman error bases are based on the Neumann series expansion. These bases incur very large initial Bellman errors, and can converge rather slowly as the discount factor approaches unity. The Laurent series expansion, which relates discounted and average-reward formulations, provides both an explanation for this slow convergence as well as suggests a way to construct more efficient basis representations. The first two terms in the Laurent series represent the scaled average-reward and the average-adjusted sum of rewards, and subsequent terms expand the discounted value function using powers of a generalized inverse called the Drazin (or group inverse) of a singular matrix derived from the transition matrix. Experiments show that Drazin bases converge considerably more quickly than several other bases, particularly for large values of the discount factor. An incremental variant of Drazin bases called Bellman average-reward bases (BARBs) is described, which provides some of the same benefits at lower computational cost.

PDF Details
v2026.09.13