Author name cluster

Xingang Wang

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

8 papers

2 author rows

AAAI Conference 2025 Conference Paper

DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generation

Guosheng Zhao
Xiaofeng Wang
Zheng Zhu
Xinze Chen
Guan Huang
Xiaoyi Bao
Xingang Wang

World models have demonstrated superiority in autonomous driving, particularly in the generation of multi-view driving videos. However, significant challenges still exist in generating customized driving videos. In this paper, we propose DriveDreamer-2, which incorporates a Large Language Model (LLM) to facilitate the creation of user-defined driving videos. Specifically, a trajectory generation function library is developed to produce trajectories that conform to user descriptions. Subsequently, an HDMap generator is designed to learn the mapping from trajectories to road structures. Ultimately, we propose the Unified Multi-View Model (UniMVM) to enhance temporal and spatial coherence in the generated multi-view driving videos. To the best of our knowledge, DriveDreamer-2 is the first world model to generate customized driving videos, and it can generate uncommon driving videos (e.g., vehicles abruptly cut in) in a user-friendly manner. Besides, experimental results demonstrate that the generated videos enhance the training of driving perception methods (e.g., 3D detection and tracking). Furthermore, video generation quality of DriveDreamer-2 surpasses other state-of-the-art methods, showcasing FID and FVD scores of 11.2 and 55.7, representing relative improvements of ~30% and ~50%.

PDF Details DOI

EAAI Journal 2024 Journal Article

A dual-channel multimodal sentiment analysis framework based on three-way decision

Xingang Wang
Mengyi Wang
Hai Cui
Yijia Zhang

In the exploration of human–computer interaction, accurately identifying and understanding the user’s sentimental state helps enhance the user experience. Multimodal sentiment analysis aims to identify sentiments through various signals. However, inconsistencies in sentiments between text and visual–audio signals pose a challenge, as models often struggle to capture these differences adequately. To alleviate this issue, we propose a dual-channel sentiment analysis framework based on three-way decision. This framework includes a multimodal model, a visual–audio model, and a three-way decision module. Specifically, the multimodal model and visual–audio model generate predictions, and the three-way decision module selects an appropriate prediction. We improve the binary cross-entropy loss function for the visual–audio model to integrate the three-way decision module better and improve sentiment prediction accuracy. Additionally, we introduce an adversarial training task to ensure differences in predictions between the two models, thereby enhancing the influence of the visual–audio model. Experimental results on four public datasets demonstrate that our framework significantly improves the accuracy of sentiment recognition and reduces the average decision time.

Details DOI

AAAI Conference 2024 Conference Paper

DiffBEV: Conditional Diffusion Model for Bird’s Eye View Perception

Jiayu Zou
Kun Tian
Zheng Zhu
Yun Ye
Xingang Wang

BEV perception is of great importance in the field of autonomous driving, serving as the cornerstone of planning, controlling, and motion prediction. The quality of the BEV feature highly affects the performance of BEV perception. However, taking the noises in camera parameters and LiDAR scans into consideration, we usually obtain BEV representation with harmful noises. Diffusion models naturally have the ability to denoise noisy samples to the ideal data, which motivates us to utilize the diffusion model to get a better BEV representation. In this work, we propose an end-to-end framework, named DiffBEV, to exploit the potential of diffusion model to generate a more comprehensive BEV representation. To the best of our knowledge, we are the first to apply diffusion model to BEV perception. In practice, we design three types of conditions to guide the training of the diffusion model which denoises the coarse samples and refines the semantic feature in a progressive way. What's more, a cross-attention module is leveraged to fuse the context of BEV feature and the semantic content of conditional diffusion model. DiffBEV achieves a 25.9% mIoU on the nuScenes dataset, which is 6.2% higher than the best-performing existing approach. Quantitative and qualitative results on multiple benchmarks demonstrate the effectiveness of DiffBEV in BEV semantic segmentation and 3D object detection tasks.

PDF Details DOI

TAAS Journal 2024 Journal Article

IBAQ: Frequency-Domain Backdoor Attack Threatening Autonomous Driving via Quadratic Phase

Jinghan Qiu
Honglong Chen
Junjian Li
Yudong Gao
Junwei Li
Xingang Wang

The rapid evolution of backdoor attacks has emerged as a significant threat to the security of autonomous driving models. An attacker injects a backdoor into the model by adding triggers to the samples, which can be activated to manipulate the model’s inference. Backdoor attacks can lead to severe consequences, such as misidentifying traffic signs during autonomous driving, posing a risk of causing traffic accidents. Recently, there has been a gradual evolution of frequency-domain backdoor attacks. However, since the change of both amplitude and its corresponding phase will significantly affect image appearance, most of the existing frequency-domain backdoor attacks change only the amplitude, which results in a suboptimal efficacy of the attack. In this work, we propose an attack called IBAQ, to solve this problem by blurring semantic information of the trigger image through the quadratic phase. Initially, we convert the trigger and benign sample to YCrCb space. Then, we perform the fast Fourier transform on the Y channel, blending the trigger image’s amplitude and quadratic phase linearly with the benign sample’s amplitude and phase. IBAQ achieves covert injection of trigger information within amplitude and phase, enhancing the attack effect. We validate the effectiveness and stealthiness of IBAQ through comprehensive experiments.

Details DOI

AAAI Conference 2024 Conference Paper

Relevant Intrinsic Feature Enhancement Network for Few-Shot Semantic Segmentation

Xiaoyi Bao
Jie Qin
Siyang Sun
Xingang Wang
Yun Zheng

For few-shot semantic segmentation, the primary task is to extract class-specific intrinsic information from limited labeled data. However, the semantic ambiguity and inter-class similarity of previous methods limit the accuracy of pixel-level foreground-background classification. To alleviate these issues, we propose the Relevant Intrinsic Feature Enhancement Network (RiFeNet). To improve the semantic consistency of foreground instances, we propose an unlabeled branch as an efficient data utilization method, which teaches the model how to extract intrinsic features robust to intra-class differences. Notably, during testing, the proposed unlabeled branch is excluded without extra unlabeled data and computation. Furthermore, we extend the inter-class variability between foreground and background by proposing a novel multi-level prototype generation and interaction module. The different-grained complementarity between global and local prototypes allows for better distinction between similar categories. The qualitative and quantitative performance of RiFeNet surpasses the state-of-the-art methods on PASCAL-5i and COCO benchmarks.

PDF Details DOI

AAAI Conference 2023 Conference Paper

Crafting Monocular Cues and Velocity Guidance for Self-Supervised Multi-Frame Depth Learning

Xiaofeng Wang
Zheng Zhu
Guan Huang
Xu Chi
Yun Ye
Ziwei Chen
Xingang Wang

Self-supervised monocular methods can efficiently learn depth information of weakly textured surfaces or reflective objects. However, the depth accuracy is limited due to the inherent ambiguity in monocular geometric modeling. In contrast, multi-frame depth estimation methods improve depth accuracy thanks to the success of Multi-View Stereo (MVS), which directly makes use of geometric constraints. Unfortunately, MVS often suffers from texture-less regions, non-Lambertian surfaces, and moving objects, especially in real-world video sequences without known camera motion and depth supervision. Therefore, we propose MOVEDepth, which exploits the MOnocular cues and VElocity guidance to improve multi-frame Depth learning. Unlike existing methods that enforce consistency between MVS depth and monocular depth, MOVEDepth boosts multi-frame depth learning by directly addressing the inherent problems of MVS. The key of our approach is to utilize monocular depth as a geometric priority to construct MVS cost volume, and adjust depth candidates of cost volume under the guidance of predicted camera velocity. We further fuse monocular depth and MVS depth by learning uncertainty in the cost volume, which results in a robust depth estimation against ambiguity in multi-view geometry. Extensive experiments show MOVEDepth achieves state-of-the-art performance: Compared with Monodepth2 and PackNet, our method relatively improves the depth accuracy by 20% and 19.8% on the KITTI benchmark. MOVEDepth also generalizes to the more challenging DDAD benchmark, relatively outperforming ManyDepth by 7.2%. The code is available at https://github.com/JeffWang987/MOVEDepth.

PDF Details DOI

ECAI Conference 2023 Conference Paper

Exploring Information Bottleneck for Weakly Supervised Semantic Segmentation

Jie Qin
Yueming Lyu
Xingang Wang

Image-level weakly supervised semantic segmentation (WSSS) has attracted much attention due to the easily acquired class labels. Most existing methods resort to utilizing Class Activation Maps (CAMs) obtained from the classification network to play as the initial pseudo labels. However, the classifiers only focus on the most discriminative regions of the target objects, which is referred to as the information bottleneck from the perspective of the information theory. To alleviate this information bottleneck limitation, we propose an Information Perturbation Module (IPM) to explicitly obtain the information difference maps, which provide the accurate direction and magnitude of the information compression in the classification network. After that, an information bottleneck breakthrough mechanism with three branches is proposed to overcome the information bottleneck in the classification network for segmentation. Additionally, a diversity regularization on the generated two information difference maps is proposed to improve the diversity of the output CAMs. Extensive experiments on PASCAL VOC2012 val and test sets demonstrate that the proposed method can effectively improve the weakly supervised semantic segmentation performance of the advanced approaches.

Details

AAAI Conference 2022 Conference Paper

Activation Modulation and Recalibration Scheme for Weakly Supervised Semantic Segmentation

Jie Qin
Jie Wu
Xuefeng Xiao
Lujun Li
Xingang Wang

Image-level weakly supervised semantic segmentation (WSSS) is a fundamental yet challenging computer vision task facilitating scene understanding and automatic driving. Most existing methods resort to classification-based Class Activation Maps (CAMs) to play as the initial pseudo labels, which tend to focus on the discriminative image regions and lack customized characteristics for the segmentation task. To alleviate this issue, we propose a novel activation modulation and recalibration (AMR) scheme, which leverages a spotlight branch and a compensation branch to obtain weighted CAMs that can provide recalibration supervision and task-specific concepts. Specifically, an attention modulation module (AMM) is employed to rearrange the distribution of feature importance from the channel-spatial sequential perspective, which helps to explicitly model channelwise interdependencies and spatial encodings to adaptively modulate segmentation-oriented activation responses. Furthermore, we introduce a cross pseudo supervision for dual branches, which can be regarded as a semantic similar regularization to mutually refine two branches. Extensive experiments show that AMR establishes a new state-of-the-art performance on the PASCAL VOC 2012 dataset, surpassing not only current methods trained with the image-level of supervision but also some methods relying on stronger supervision, such as saliency label. Experiments also reveal that our scheme is plug-and-play and can be incorporated with other approaches to boost their performance. Our code is available at: https: //github. com/jieqin-ai/AMR.

PDF Details