Author name cluster

Bo Wan

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

5 papers

2 author rows

JBHI Journal 2026 Journal Article

DiffSpkSync: A Muscle Synergy-Guided Spiking Diffusion Model for EMG Signal Generation to Improve Gesture Recognition Performance

Kejia Su
Bo Wan
Jiayang Huang
Zhi-Qiang Zhang
Junhao Zhang
Pengfei Yang
Quan Wang

High-density surface electromyography (HD-sEMG) based hand gesture recognition (HGR) has shown great promise for intuitive human-machine interaction. However, the performance of HGR model is often hindered by a scarcity of available training data, especially in the fields of gesture recognition, rehabilitation, and medicine. To address these issues, we propose DiffSpkSync, a novel generative framework that integrates (1) muscle synergy-guided diffusion modeling for physiologically plausible signal reconstruction, (2) spiking neuron-based sparsification to reduce energy cost, and (3) a time-series mixup strategy to preserve local dynamics during augmentation. Experiments on a public Hyser dataset and a self-collected XDHDEMG dataset demonstrate that training gesture classifiers with data augmented by DiffSpkSync consistently improves classification accuracy in both intrasession and intersession scenarios. Comparative results further demonstrate superior performance over representative generative baselines, including VAE, DCGAN, DANN-CRC, and PatchEMG. Furthermore, real-time validation demonstrates that the proposed method achieves an average of 130. 22 ms end-to-end latency and an average of 95. 87% accuracy predictions, supporting their applicability in real-world applications.

Details DOI

NeurIPS Conference 2024 Conference Paper

LocCa: Visual Pretraining with Location-aware Captioners

Bo Wan
Michael Tschannen
Yongqin Xian
Filip Pavetic
Ibrahim Alabdulmohsin
Xiao Wang
André S. Pinto
Andreas Steiner

Image captioning was recently found to be an effective pretraining method similar to contrastive pretraining. This opens up the largely-unexplored potential of using natural language as a flexible and powerful interface for handling diverse pretraining tasks. In this paper, we demonstrate this with a novel visual pretraining paradigm, LocCa, that incorporates location-aware tasks into captioners to teach models to extract rich information from images. Specifically, LocCa employs two tasks, bounding box prediction and location-dependent captioning, conditioned on the image pixel input. Thanks to the multitask capabilities of an encoder-decoder architecture, we show that an image captioner can effortlessly handle multiple tasks during pretraining. LocCa significantly outperforms standard captioners on downstream localization tasks, achieving state-of-the-art results on RefCOCO/+/g, while maintaining comparable performance on holistic tasks. Our work paves the way for further exploration of natural language interfaces in visual pretraining.

PDF Details DOI

ICLR Conference 2023 Conference Paper

Weakly-supervised HOI Detection via Prior-guided Bi-level Representation Learning

Bo Wan
Yongfei Liu
Desen Zhou
Tinne Tuytelaars
Xuming He 0001

Human object interaction (HOI) detection plays a crucial role in human-centric scene understanding and serves as a fundamental building block for many vision tasks. One generalizable and scalable strategy for HOI detection is to use weak supervision, learning from image-level annotations only. This is inherently challenging due to ambiguous human-object associations, large search space of detecting HOIs and highly noisy training signal. A promising strategy to address those challenges is to exploit knowledge from large-scale pretrained models (e.g., CLIP), but a direct knowledge distillation strategy does not perform well on the weakly-supervised setting. In contrast, we develop a CLIP-guided HOI representation capable of incorporating the prior knowledge at both image level and HOI instance level, and adopt a self-taught mechanism to prune incorrect human-object associations. Experimental results on HICO-DET and V-COCO show that our method outperforms the previous works by a sizable margin, showing the efficacy of our HOI representation.

Details

ICLR Conference 2022 Conference Paper

Unsupervised Vision-Language Grammar Induction with Shared Structure Modeling

Bo Wan
Wenjuan Han
Zilong Zheng
Tinne Tuytelaars

We introduce a new task, unsupervised vision-language (VL) grammar induction. Given an image-caption pair, the goal is to extract a shared hierarchical structure for both image and language simultaneously. We argue that such structured output, grounded in both modalities, is a clear step towards the high-level understanding of multimodal information. Besides challenges existing in conventional visually grounded grammar induction tasks, VL grammar induction requires a model to capture contextual semantics and perform a fine-grained alignment. To address these challenges, we propose a novel method, CLIORA, which constructs a shared vision-language constituency tree structure with context-dependent semantics for all possible phrases in different levels of the tree. It computes a matching score between each constituent and image region, trained via contrastive learning. It integrates two levels of fusion, namely at feature-level and at score-level, so as to allow fine-grained alignment. We introduce a new evaluation metric for VL grammar induction, CCRA, and show a 3.3% improvement over a strong baseline on Flickr30k Entities. We also evaluate our model via two derived tasks, i.e., language grammar induction and phrase grounding, and improve over the state-of-the-art for both.

Details

AAAI Conference 2020 Conference Paper

Learning Cross-Modal Context Graph for Visual Grounding

Yongfei Liu
Bo Wan
Xiaodan Zhu
Xuming He

Visual grounding is a ubiquitous building block in many vision-language tasks and yet remains challenging due to large variations in visual and linguistic features of grounding entities, strong context effect and the resulting semantic ambiguities. Prior works typically focus on learning representations of individual phrases with limited context information. To address their limitations, this paper proposes a languageguided graph representation to capture the global context of grounding entities and their relations, and develop a crossmodal graph matching strategy for the multiple-phrase visual grounding task. In particular, we introduce a modular graph neural network to compute context-aware representations of phrases and object proposals respectively via message propagation, followed by a graph-based matching module to generate globally consistent localization of grounding phrases. We train the entire graph neural network jointly in a two-stage strategy and evaluate it on the Flickr30K Entities benchmark. Extensive experiments show that our method outperforms the prior state of the arts by a sizable margin, evidencing the efﬁcacy of our grounding framework. Code is available at https: //github. com/youngﬂy11/LCMCG-PyTorch.

PDF Details