Author name cluster

Dapeng Chen

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

11 papers

2 author rows

IJCAI Conference 2025 Conference Paper

Few-Shot Incremental Multi-modal Learning via Touch Guidance and Imaginary Vision Synthesis

Lina Wei
Yuhang Ma
Zhongsheng Lin
Fangfang Wang
Canghong Jin
Hanbin Zhao
Dapeng Chen

Multimodal perception, which integrates vision and touch, is increasingly demonstrating its significance in domains such as embodied intelligence and human-computer interaction. However, in open-world scenarios, multimodal data streams face significant challenges, including catastrophic forgetting and overfitting, during few-shot class incremental learning (FSCIL), leading to a severe degradation in model performance. In this work, we propose a novel approach named Few-Shot Incremental Multi-modal Learning via Touch Guidance and Imaginary Vision Synthesis (TIFS). Our method leverages vision imagination synthesis to enhance the semantic understanding and integrates touch and vision fusion to improve the problem of modal imbalance. Specifically, we introduce a framework that employs touch-guided vision information for cross-modal contrastive learning to address the challenges of few-shot learning. Additionally, we incorporate multiple learning mechanisms, including regularization, memory mechanisms, and attention mechanisms, to mitigate catastrophic forgetting during multi-incremental step learning. Experimental results on the Touch and Go and VisGel datasets demonstrate that the TIFS framework exhibits robust continuous learning capabilities and strong generalization performance in touch-vision few-shot incremental learning tasks. Our code is available at https: //github. com/Vision-Multimodal-Lab-HZCU/TIFS.

PDF Details DOI

EAAI Journal 2025 Journal Article

High-resolution multi-view stereo with multi-scale feature fusion

Dapeng Chen
Qi Jia
Hao Wu
Da Yu
Nanxuan Huang
Jia Liu

To enhance the handling of three-dimensional reconstruction for large-scale scenes and high-resolution images, we introduce a novel multi-view high-resolution three-dimensional reconstruction approach. Our proposed method integrates a Feature Pyramid Network with the Swin Transformer for improved performance. We integrate the Swin Transformer into the feature pyramid. This integration aims to establish long-range feature dependencies, facilitate information exchange between different input positions, and enhance the global consistency of feature representation. This improves the efficiency of the feature extraction stage. Following this, we apply cost volume regularization to mitigate noise and compute depth maps. A Depth Optimization Module is employed to refine the predicted depth maps, thereby enhancing their precision. Experimental results demonstrate the efficacy of our method in generating more accurate depth information, particularly in predicting high-resolution depth maps. Our approach utilizes these depth predictions to generate point clouds, enabling precise matching and reconstruction of multi-view images. Experiments conducted on public datasets validate the effectiveness and superiority of our proposed method.

Details DOI

NeurIPS Conference 2022 Conference Paper

FNeVR: Neural Volume Rendering for Face Animation

Bohan Zeng
Boyu Liu
Hong Li
Xuhui Liu
Jianzhuang Liu
Dapeng Chen
Wei Peng
Baochang Zhang

Face animation, one of the hottest topics in computer vision, has achieved a promising performance with the help of generative models. However, it remains a critical challenge to generate identity preserving and photo-realistic images due to the sophisticated motion deformation and complex facial detail modeling. To address these problems, we propose a Face Neural Volume Rendering (FNeVR) network to fully explore the potential of 2D motion warping and 3D volume rendering in a unified framework. In FNeVR, we design a 3D Face Volume Rendering (FVR) module to enhance the facial details for image rendering. Specifically, we first extract 3D information with a well designed architecture, and then introduce an orthogonal adaptive ray-sampling module for efficient rendering. We also design a lightweight pose editor, enabling FNeVR to edit the facial pose in a simple yet effective way. Extensive experiments show that our FNeVR obtains the best overall quality and performance on widely used talking-head benchmarks.

PDF Details

AAAI Conference 2022 Conference Paper

I Can Find You! Boundary-Guided Separated Attention Network for Camouflaged Object Detection

Hongwei Zhu
Peng Li
Haoran Xie
Xuefeng Yan
Dong Liang
Dapeng Chen
Mingqiang Wei
Jing Qin

Can you find me? By simulating how humans to discover the so-called ‘perfectly’-camouflaged object, we present a novel boundary-guided separated attention network (call BSA-Net). Beyond the existing camouflaged object detection (COD) wisdom, BSA-Net utilizes two-stream separated attention modules to highlight the separator (or say the camouflaged object’s boundary) between an image’s background and foreground: the reverse attention stream helps erase the camouflaged object’s interior to focus on the background, while the normal attention stream recovers the interior and thus pay more attention to the foreground; and both streams are followed by a boundary guider module and combined to strengthen the understanding of the boundary. The core design of such separated attention is motivated by the COD procedure of humans: find the subtle difference between the foreground and background to delineate the boundary of a camouflaged object, then the boundary can help further enhance the COD accuracy. We validate on three benchmark datasets that our BSA-Net is very beneficial to detect camouflaged objects with the blurred boundaries and similar colors/patterns with their backgrounds. Extensive results exhibit very clear COD improvements on our BSA-Net over sixteen SOTAs.

PDF Details

AAAI Conference 2022 Conference Paper

Learning to Predict 3D Lane Shape and Camera Pose from a Single Image via Geometry Constraints

Ruijin Liu
Dapeng Chen
Tie Liu
Zhiliang Xiong
Zejian Yuan

Detecting 3D lanes from the camera is a rising problem for autonomous vehicles. In this task, the correct camera pose is the key to generating accurate lanes, which can transform an image from perspective-view to the top-view. With this transformation, we can get rid of the perspective effects so that 3D lanes would look similar and can accurately be fitted by low-order polynomials. However, mainstream 3D lane detectors rely on perfect camera poses provided by other sensors, which is expensive and encounters multi-sensor calibration issues. To overcome this problem, we propose to predict 3D lanes by estimating camera pose from a single image with a two-stage framework. The first stage aims at the camera pose task from perspective-view images. To improve pose estimation, we introduce an auxiliary 3D lane task and geometry constraints to benefit from multi-task learning, which enhances consistencies between 3D and 2D, as well as compatibility in the above two tasks. The second stage targets the 3D lane task. It uses previously estimated pose to generate topview images containing distance-invariant lane appearances for predicting accurate 3D lanes. Experiments demonstrate that, without ground truth camera pose, our method outperforms the state-of-the-art perfect-camera-pose-based methods and has the fewest parameters and computations. Codes are available at https: //github. com/liuruijin17/CLGo.

PDF Details

AAAI Conference 2021 Conference Paper

Gradient Regularized Contrastive Learning for Continual Domain Adaptation

Shixiang Tang
Peng Su
Dapeng Chen
Wanli Ouyang

Human beings can quickly adapt to environmental changes by leveraging learning experience. However, adapting deep neural networks to dynamic environments by machine learning algorithms remains a challenge. To better understand this issue, we study the problem of continual domain adaptation, where the model is presented with a labelled source domain and a sequence of unlabelled target domains. The obstacles in this problem are both domain shift and catastrophic forgetting. We propose Gradient Regularized Contrastive Learning (GRCL) to solve the obstacles. At the core of our method, gradient regularization plays two key roles: (1) enforcing the gradient not to harm the discriminative ability of source features which can, in turn, benefit the adaptation ability of the model to target domains; (2) constraining the gradient not to increase the classification loss on old target domains, which enables the model to preserve the performance on old target domains when adapting to an in-coming target domain. Experiments on Digits, DomainNet and Office-Caltech benchmarks demonstrate the strong performance of our approach when compared to the state-of-the-art.

PDF Details

AAAI Conference 2021 Conference Paper

SSN3D: Self-Separated Network to Align Parts for 3D Convolution in Video Person Re-Identification

Xiaoke Jiang
Yu Qiao
Junjie Yan
Qichen Li
Wanrong Zheng
Dapeng Chen

Temporal appearance misalignment is a crucial problem in video person re-identification. The same part of person (e. g. head or hand) appearing on different locations in video sequence weakens its discriminative ability, especially when we apply standard temporal aggregation such as 3D convolution or LSTM. To address this issue, we propose Self-Separated network (SSN) to seek out the same parts in different images. As the name implies, SSN, if trained in an unsupervised strategy, guarantees the selected parts distinct. With a few samples of labeled parts to guide SSN training, this semi-supervised trained SSN seeks out the parts that are human-understandable within a frame and stable across a video snippet. Given the distinct and stable person parts, rather than performing aggregation on features, we then apply 3D convolution across different frames for person re-identification. This SSN + 3D pipeline, dubbed SSN3D, is proved to be efficient through extensive experiments on both synthetic and real data.

PDF Details

ICLR Conference 2020 Conference Paper

Mutual Mean-Teaching: Pseudo Label Refinery for Unsupervised Domain Adaptation on Person Re-identification

Yixiao Ge
Dapeng Chen
Hongsheng Li 0001

Person re-identification (re-ID) aims at identifying the same persons' images across different cameras. However, domain diversities between different datasets pose an evident challenge for adapting the re-ID model trained on one dataset to another one. State-of-the-art unsupervised domain adaptation methods for person re-ID transferred the learned knowledge from the source domain by optimizing with pseudo labels created by clustering algorithms on the target domain. Although they achieved state-of-the-art performances, the inevitable label noise caused by the clustering procedure was ignored. Such noisy pseudo labels substantially hinders the model's capability on further improving feature representations on the target domain. In order to mitigate the effects of noisy pseudo labels, we propose to softly refine the pseudo labels in the target domain by proposing an unsupervised framework, Mutual Mean-Teaching (MMT), to learn better features from the target domain via off-line refined hard pseudo labels and on-line refined soft pseudo labels in an alternative training manner. In addition, the common practice is to adopt both the classification loss and the triplet loss jointly for achieving optimal performances in person re-ID models. However, conventional triplet loss cannot work with softly refined labels. To solve this problem, a novel soft softmax-triplet loss is proposed to support learning with soft pseudo triplet labels for achieving the optimal domain adaptation performance. The proposed MMT framework achieves considerable improvements of 14.4%, 18.2%, 13.1% and 16.4% mAP on Market-to-Duke, Duke-to-Market, Market-to-MSMT and Duke-to-MSMT unsupervised domain adaptation tasks.

Details

ICRA Conference 2020 Conference Paper

Real-time Continuous Hand Motion Myoelectric Decoding by Automated Data Labeling *

Xuhui Hu
Hong Zeng 0001
Dapeng Chen
Jiahang Zhu
Aiguo Song

In this paper an automated data labeling (ADL) neural network is proposed to streamline dataset collecting for real-time predicting the continuous motion of hand and wrist, these gestures are only decoded from a surface electromyography (sEMG) array of eight channels. Unlike collecting both the bio-signals and hand motion signals as samples and labels in supervised learning, this algorithm only collects unlabeled sEMG into an unsupervised neural network, in which the hand motion labels are auto-generated. The coefficient of determination (R 2 ) for three DOFs, i. e. wrist flex/extension, wrist pro/supination, hand open/close, was 0. 86, 0. 89 and 0. 87 respectively. The comparison between real motion labels and auto-generated labels shows that the latter has earlier response than former. The results of Fitts’ law test indicate that ADL has capability of controlling multi-DOFs simultaneously even though the training set only contains sEMG data from single DOF gesture. Moreover, no more hand motion measurement needed which greatly helps upper limb amputee imagine the gesture of residual limb to control a dexterous prosthesis.

Details

NeurIPS Conference 2020 Conference Paper

Self-paced Contrastive Learning with Hybrid Memory for Domain Adaptive Object Re-ID

Yixiao Ge
Feng Zhu
Dapeng Chen
Rui Zhao
Hongsheng Li

Domain adaptive object re-ID aims to transfer the learned knowledge from the labeled source domain to the unlabeled target domain to tackle the open-class re-identification problems. Although state-of-the-art pseudo-label-based methods have achieved great success, they did not make full use of all valuable information because of the domain gap and unsatisfying clustering performance. To solve these problems, we propose a novel self-paced contrastive learning framework with hybrid memory. The hybrid memory dynamically generates source-domain class-level, target-domain cluster-level and un-clustered instance-level supervisory signals for learning feature representations. Different from the conventional contrastive learning strategy, the proposed framework jointly distinguishes source-domain classes, and target-domain clusters and un-clustered instances. Most importantly, the proposed self-paced method gradually creates more reliable clusters to refine the hybrid memory and learning targets, and is shown to be the key to our outstanding performance. Our method outperforms state-of-the-arts on multiple domain adaptation tasks of object re-ID and even boosts the performance on the source domain without any extra annotations. Our generalized version on unsupervised object re-ID surpasses state-of-the-art algorithms by considerable 16. 7% and 7. 9% on Market-1501 and MSMT17 benchmarks.

PDF Details

IROS Conference 2019 Conference Paper

Cable-Driven 4-DOF Upper Limb Rehabilitation Robot

Ke Shi 0006
Aiguo Song
Ye Li 0030
Dapeng Chen
Huijun Li

This paper developed a 4-degree-of-freedom cable-driven upper limb rehabilitation robot and proposed a control algorithm of the passive training for this robot. Comparing with the conventional cable-driven rehabilitation robot, the workspace of this robot is increased by optimizing the distribution of the cable attachment points and by improving the mechanical design. The rotation structure of the upper arm module can change the distribution of the attachment points as needed, by which the cable tension planner can be satisfied in almost all cases. At the meantime, the internal/external rotation of shoulder joint can be achieved without the change of the cables configuration, which is also important for increasing the workspace and comfortability of utilization. The activities of daily living (ADLs) training can be achieved well without any manual adjustment. The related controller for passive training is designed, which includes a higher controller for trajectory tracking and a lower controller for keeping cable tension as the output of the tension planner in real-time. The passive training experiments are conducted on five healthy subjects of different body size. The results demonstrated that the passive training can be achieved well on different subjects and the cable tension controller is also working effectively.

Details