Author name cluster

Bo Pang

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

15 papers

2 author rows

IJCAI Conference 2025 Conference Paper

CABIN: Debiasing Vision-Language Models Using Backdoor Adjustments

Bo Pang
Tingrui Qiao
Caroline Walker
Chris Cunningham
Yun Sing Koh

Vision-language models (VLMs) have demonstrated strong zero-shot inference capabilities but may exhibit stereotypical biases toward certain demographic groups. Consequently, downstream tasks leveraging these models may yield unbalanced performance across different target social groups, potentially reinforcing harmful stereotypes. Mitigating such biases is critical for ensuring fairness in practical applications. Existing debiasing approaches typically rely on curated face-centric datasets for fine-tuning or retraining, risking overfitting and limiting generalisability. To address this issue, we propose a novel framework, CABIN (Causal Adjustment Based INtervention). It leverages a causal framework to identify sensitive attributes in images as confounding factors. Employing a learned mapper, which is trained on general large-scale image-text pairs rather than face-centric datasets, CABIN may use text to adjust sensitive attributes in the image embedding, ensuring independence between these sensitive attributes and image embeddings. This independence enables a backdoor adjustment for unbiased inference without the drawbacks of additional fine-tuning or retraining on narrowly tailored datasets. Through comprehensive experiments and analyses, we demonstrate that CABIN effectively mitigates biases and improves fairness metrics while preserving the zero-shot strengths of VLMs. The code is available at: https: //github. com/ipangbo/causal-debias

PDF Details DOI

EAAI Journal 2025 Journal Article

Factors analysis of commercial bank digital transformation with random neural network algorithm

Yixi Ding
Ju Wei
Bo Pang
Ling Tong

With the deepening of global economic and technological integration, digital transformation has emerged as a key factor for enhancing operational efficiency, improving customer experience, and reshaping the strategic and organizational frameworks of commercial banks. Existing literature often treats digital transformation as a derivative process influenced by external technological trends. This paper argues that digital transformation is an original and strategic operation that requires user-centric design and adaptive technological integration. By leveraging an independently developed digital platform, and employing artificial intelligence algorithms and optimization models, this study constructs a comprehensive model to analyze the key factors influencing a bank’s digital transformation from the perspective of considering user-platform interaction. Experimental results show that, in terms of interpretability, the proposed model demonstrates a 9%/7% improvement in R 2 and adjusted R 2, respectively. In terms of prediction, the proposed model shows a clear advantage, providing more accurate and stable results. The findings from this study offer a scientifically grounded reference for commercial banks, enabling them to better understand and address the challenges and opportunities associated with digital transformation, ultimately leading to more efficient and sustainable development.

Details DOI

IROS Conference 2025 Conference Paper

PB-MOT: Pose-aware Association Boosted Online 3D Multi-Object Tracking

Bo Pang
Yang Xu
Jiming Chen
Liang Li

Robotic and autonomous driving platforms necessitate efficient 3D Multi-Object Tracking (MOT) that harmonizes geometric precision, motion robustness, and computational efficiency. Traditional 3D MOT approaches face critical challenges: geometric similarity metrics (e. g. , IoU-based) degrade at long ranges with high computational costs, while distance-based methods fail to capture object orientation and shape; the effects of occlusion and the intricate relative ego-object motion degrade tracking performance in dynamic scenes. To this end, we propose PB-MOT, an online framework integrating two key innovations: ego-motion-compensated state estimation that decouples dynamic interactions; and a rotated ellipse association algorithm unifying pose and shape-aware matching with adaptive distance constraints. Evaluations on the KITTI benchmark show that our PB-MOT achieves state-of-the-art performance with a HOTA score of 81. 94%, while running at an impressive 2, 402. 76 FPS on CPU. This enables real-time, high-fidelity perception and tracking for resource-constrained robotic systems.

Details

IJCAI Conference 2025 Conference Paper

Temporal Consistency Constrained Transferable Adversarial Attacks with Background Mixup for Action Recognition

Ping Li
Jianan Ni
Bo Pang

Action recognition models using deep learning are vulnerable to adversarial examples, which are transferable across other models trained on the same data modality. Existing transferable attack methods face two major challenges: 1) they heavily rely on the assumption that the decision boundaries of the surrogate (a. k. a. , source) model and the target model are similar, which limits the adversarial transferability; and 2) their decision boundary difference makes the attack direction uncertain, which may result in the gradient oscillation, weakening the adversarial attack. This motivates us to propose a Background Mixup-induced Temporal Consistency (BMTC) attack method for action recognition. From the input transformation perspective, we design a model-agnostic background adversarial mixup module to reduce the surrogate-target model dependency. In particular, we randomly sample one video from each category and make its background frame, while selecting the background frame with the top attack ability for mixup with the clean frame by reinforcement learning. Moreover, to ensure an explicit attack direction, we leverage the background category as guidance for updating the gradient of adversarial example, and design a temporal gradient consistency loss, which strengthens the stability of the attack direction on subsequent frames. Empirical studies on two video datasets, i. e. , UCF101 and Kinetics-400, and one image dataset, i. e. , ImageNet, demonstrate that our method significantly boosts the transferability of adversarial examples across several action/image recognition models.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

Latent Plan Transformer for Trajectory Abstraction: Planning as Latent Space Inference

Deqian Kong
Dehong Xu
Minglu Zhao
Bo Pang
Jianwen Xie
Andrew Lizarraga
Yuhao Huang
Sirui Xie

In tasks aiming for long-term returns, planning becomes essential. We study generative modeling for planning with datasets repurposed from offline reinforcement learning. Specifically, we identify temporal consistency in the absence of step-wise rewards as one key technical challenge. We introduce the Latent Plan Transformer (LPT), a novel model that leverages a latent variable to connect a Transformer- based trajectory generator and the final return. LPT can be learned with maximum likelihood estimation on trajectory-return pairs. In learning, posterior sampling of the latent variable naturally integrates sub-trajectories to form a consistent abstrac- tion despite the finite context. At test time, the latent variable is inferred from an expected return before policy execution, realizing the idea of planning as inference. Our experiments demonstrate that LPT can discover improved decisions from sub- optimal trajectories, achieving competitive performance across several benchmarks, including Gym-Mujoco, Franka Kitchen, Maze2D, and Connect Four. It exhibits capabilities in nuanced credit assignments, trajectory stitching, and adaptation to environmental contingencies. These results validate that latent variable inference can be a strong alternative to step-wise reward prompting.

PDF Details DOI

TMLR Journal 2024 Journal Article

RLHF Workflow: From Reward Modeling to Online RLHF

Hanze Dong
Wei Xiong
Bo Pang
Haoxiang Wang
Han Zhao
Yingbo Zhou
Nan Jiang
Doyen Sahoo

We present the workflow of Online Iterative Reinforcement Learning from Human Feedback (RLHF) in this technical report, which is widely reported to outperform its offline counterpart by a large margin in the recent large language model (LLM) literature. However, existing open-source RLHF projects are still largely confined to the offline learning setting. In this technical report, we aim to fill in this gap and provide a detailed recipe that is easy to reproduce for online iterative RLHF. In particular, since online human feedback is usually infeasible for open-source communities with limited resources, we start by constructing preference models using a diverse set of open-source datasets and use the constructed proxy preference model to approximate human feedback. Then, we discuss the theoretical insights and algorithmic principles behind online iterative RLHF, followed by a detailed practical implementation. Our trained LLM achieves impressive performance on LLM chatbot benchmarks, including AlpacaEval-2, Arena-Hard, and MT-Bench, as well as other academic benchmarks such as HumanEval and TruthfulQA. We have shown that supervised fine-tuning (SFT) and iterative RLHF can obtain state-of-the-art performance with fully open-source datasets. Further, we have made our models, curated datasets, and comprehensive step-by-step code guidebooks publicly available.

PDF Details

NeurIPS Conference 2022 Conference Paper

Learning Probabilistic Models from Generator Latent Spaces with Hat EBM

Mitch Hill
Erik Nijkamp
Jonathan Mitchell
Bo Pang
Song-Chun Zhu

This work proposes a method for using any generator network as the foundation of an Energy-Based Model (EBM). Our formulation posits that observed images are the sum of unobserved latent variables passed through the generator network and a residual random variable that spans the gap between the generator output and the image manifold. One can then define an EBM that includes the generator as part of its forward pass, which we call the Hat EBM. The model can be trained without inferring the latent variables of the observed data or calculating the generator Jacobian determinant. This enables explicit probabilistic modeling of the output distribution of any type of generator network. Experiments show strong performance of the proposed method on (1) unconditional ImageNet synthesis at 128$\times$128 resolution, (2) refining the output of existing generators, and (3) learning EBMs that incorporate non-probabilistic generators. Code and pretrained models to reproduce our results are available at https: //github. com/point0bar1/hat-ebm.

PDF Details

AAAI Conference 2022 Conference Paper

Unsupervised Representation for Semantic Segmentation by Implicit Cycle-Attention Contrastive Learning

Bo Pang
Yizhuo Li
Yifan Zhang
Gao Peng
Jiajun Tang
Kaiwen Zha
Jiefeng Li
Cewu Lu

We study the unsupervised representation learning for the semantic segmentation task. Different from previous works that aim at providing unsupervised pre-trained backbones for segmentation models which need further supervised fine-tune, here, we focus on providing representation that is only trained by unsupervised methods. This means models need to directly generate pixel-level, linearly separable semantic results. We first explore and present two factors that have significant effects on segmentation under the contrastive learning framework: 1) the difficulty and diversity of the positive contrastive pairs, 2) the balance of global and local features. With the intention of optimizing these factors, we propose the cycle-attention contrastive learning (CACL). CACL makes use of semantic continuity of video frames, adopting unsupervised cycle-consistent attention mechanism to implicitly conduct contrastive learning with difficult, global-local-balanced positive pixel pairs. Compared with baseline model MoCo-v2 and other unsupervised methods, CACL demonstrates consistently superior performance on PASCAL VOC (+4. 5 mIoU) and Cityscapes (+4. 5 mIoU) datasets.

PDF Details

AAAI Conference 2021 Conference Paper

Robust Reinforcement Learning: A Case Study in Linear Quadratic Regulation

Bo Pang
Zhong-Ping Jiang

This paper studies the robustness of reinforcement learning algorithms to errors in the learning process. Specifically, we revisit the benchmark problem of discrete-time linear quadratic regulation (LQR) and study the long-standing open question: Under what conditions is the policy iteration method robustly stable from a dynamical systems perspective? Using advanced stability results in control theory, it is shown that policy iteration for LQR is inherently robust to small errors in the learning process and enjoys small-disturbance input-to-state stability: whenever the error in each iteration is bounded and small, the solutions of the policy iteration algorithm are also bounded, and, moreover, enter and stay in a small neighbourhood of the optimal LQR solution. As an application, a novel off-policy optimistic least-squares policy iteration for the LQR problem is proposed, when the system dynamics are subjected to additive stochastic disturbances. The proposed new results in robust reinforcement learning are validated by a numerical example.

PDF Details

AAAI Conference 2021 Conference Paper

TDAF: Top-Down Attention Framework for Vision Tasks

Bo Pang
Yizhuo Li
Jiefeng Li
Muchen Li
Hanwen Cao
Cewu Lu

Human attention mechanisms often work in a top-down manner, yet it is not well explored in vision research. Here, we propose the Top-Down Attention Framework (TDAF) to capture top-down attentions, which can be easily adopted in most existing models. The designed Recursive Dual-Directional Nested Structure in it forms two sets of orthogonal paths, recursive and structural ones, where bottom-up spatial features and top-down attention features are extracted respectively. Such spatial and attention features are nested deeply, therefore, the proposed framework works in a mixed top-down and bottom-up manner. Empirical evidence shows that our TDAF can capture effective stratified attention information and boost performance. ResNet with TDAF achieves 2. 0% improvements on ImageNet. For object detection, the performance is improved by 2. 7% AP over FCOS. For pose estimation, TDAF improves the baseline by 1. 6%. And for action recognition, the 3D-ResNet adopting TDAF achieves improvements of 1. 7% accuracy.

PDF Details

AAAI Conference 2020 Conference Paper

Further Understanding Videos through Adverbs: A New Video Task

Bo Pang
Kaiwen Zha
Yifan Zhang
Cewu Lu

Video understanding is a research hotspot of computer vision and signiﬁcant progress has been made on video action recognition recently. However, the semantics information contained in actions is not rich enough to build powerful video understanding models. This paper ﬁrst introduces a new video semantics: the Behavior Adverb (BA), which is a more expressive and difﬁcult one covering subtle and inherent characteristics of human action behavior. To exhaustively decode this semantics, we construct the Videos with Action and Adverb Dataset (VAAD), which is a large-scale dataset with a semantically complete set of BAs. The dataset will be released to the public with this paper. We benchmark several representative video understanding methods (originally for action recognition) on BA and action recognition. The results show that BA recognition task is more challenging than conventional action recognition. Accordingly, we propose the BA Understanding Network (BAUN) to solve this problem and the experiments reveal that our BAUN is more suitable for BA recognition (11% better than I3D). Furthermore, we ﬁnd these two semantics (action and BA) can propel each other forward to better performance: promoting action recognition results by 3. 4% averagely on three standard action recognition datasets (UCF-101, HMDB-51, Kinetics).

PDF Details

NeurIPS Conference 2020 Conference Paper

Learning Latent Space Energy-Based Prior Model

Bo Pang
Tian Han
Erik Nijkamp
Song-Chun Zhu
Ying Nian Wu

We propose an energy-based model (EBM) in the latent space of a generator model, so that the EBM serves as a prior model that stands on the top-down network of the generator model. Both the latent space EBM and the top-down network can be learned jointly by maximum likelihood, which involves short-run MCMC sampling from both the prior and posterior distributions of the latent vector. Due to the low dimensionality of the latent space and the expressiveness of the top-down network, a simple EBM in latent space can capture regularities in the data effectively, and MCMC sampling in latent space is efficient and mixes well. We show that the learned model exhibits strong performances in terms of image and text generation and anomaly detection. The one-page code can be found in supplementary materials.

PDF Details

EAAI Journal 2018 Journal Article

ThermalNet: A deep reinforcement learning-based combustion optimization system for coal-fired boiler

Yin Cheng
Yuexin Huang
Bo Pang
Weidong Zhang

This paper presents a combustion optimization system for coal-fired boilers that includes a trade-off between emissions control and boiler efficiency. Designing an optimizer for this nonlinear, multiple-input multiple-output problem is challenging. This paper describes the development of an integrated combustion optimization system called ThermalNet, which is based on a deep Q-network (DQN) and a long short-term memory (LSTM) module. ThermalNet is a highly automated system consisting of an LSTM–ConvNet predictor and a DQN optimizer. The LSTM–ConvNet extracts the features of boiler behavior from the distributed control system (DCS) operational data of a supercritical thermal plant. The DQN reinforcement learning optimizer contributes to the online development of policies based on static and dynamic states. ThermalNet establishes a sequence of control actions that both reduce emissions and simultaneously enhance fuel utilization. The internal structure of the DQN optimizer demonstrates a greater representation capacity than does the shallow multilayer optimizer. The presented experiments indicate the effectiveness of the proposed optimization system.

Details DOI

SoCS Conference 2012 Conference Paper

Multimapping Abstractions and Hierarchical Heuristic Search

Bo Pang
Robert C. Holte

In this paper we introduce a broadly applicable method, called multimapping abstraction, that allows multiple heuristic values for a state to be extracted from one abstract state space. The key idea is to define an abstraction to be a multimapping, i. e. , a function that maps a state in the original state space to a set of states in the abstract space. We performed a large-scale experiment on several benchmark state spaces to compare the memory requirements and runtime of Hierarchical IDA* (HIDA*) using multimapping domain abstractions to HIDA* with individual domain abstractions and to HIDA* with multiple, independent domain abstractions. Our results show that multimapping domain abstractions are superior to both alternatives in terms of both memory usage and runtime.

Details

SoCS Conference 2011 Conference Paper

State-Set Search

Bo Pang
Robert C. Holte

State-set search is state space search when the states being manipulated by the search algorithm are sets of states from some underlying state space. State-set search arises commonly in planning and abstraction systems, but this paper provides the first formal, general analysis of state-set search. We show that the state-set distance computed by planning systems is different than that computed by abstraction systems and introduce a distance in between the two, dww, the maximum admissible distance. We introduce the concept of a multi-abstraction, which maps a state to more than one abstract state in the same abstract space, describe the first implementation of a multi-abstraction system that computes dww, and give initial experimental evidence that it can be superior to domain abstraction.

Details