Author name cluster

Chengkai Tang

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

3 papers

1 author row

EAAI Journal 2024 Journal Article

Multi-scale spatial pyramid attention mechanism for image recognition: An effective approach

Yang Yu
Yi Zhang
Zeyu Cheng
Zhe Song
Chengkai Tang

Attention mechanisms have gradually become necessary to enhance the representational power of convolutional neural networks (CNNs). Despite recent progress in attention mechanism research, some open problems still exist. Most existing methods ignore modeling multi-scale feature representations, structural information, and long-range channel dependencies, which are essential for delivering more discriminative attention maps. This study proposes a novel, low-overhead, high-performance attention mechanism with strong generalization ability for various networks and datasets. This mechanism is called Multi-Scale Spatial Pyramid Attention (MSPA) and can be used to solve the limitations of other attention methods. For the critical components of MSPA, we not only develop the Hierarchical-Phantom Convolution (HPC) module, which can extract multi-scale spatial information at a more granular level utilizing hierarchical residual-like connections, but also design the Spatial Pyramid Recalibration (SPR) module, which can integrate structural regularization and structural information in an adaptive combination mechanism, while employing the Softmax operation to build long-range channel dependencies. The proposed MSPA is a powerful tool that can be conveniently embedded into various CNNs as a plug-and-play component. Correspondingly, using MSPA to replace the 3 × 3 convolution in the bottleneck residual blocks of ResNets, we created a series of simple and efficient backbones named MSPANet, which naturally inherit the advantages of MSPA. Without bells and whistles, our method substantially outperforms other state-of-the-art counterparts in all evaluation metrics based on extensive experimental results from CIFAR-100 and ImageNet-1K image recognition. When applying MSPA to ResNet-50, our model achieves top-1 classification accuracy of 81. 74% and 78. 40% on the CIFAR-100 and ImageNet-1K benchmarks, exceeding the corresponding baselines by 3. 95% and 2. 27%, respectively. We also obtained promising performance improvements of 1. 15% and 0. 91% compared to the competitive EPSANet-50. In addition, empirical research results in autonomous driving engineering applications also demonstrate that our method can significantly improve the accuracy and real-time performance of image recognition with cheaper overhead. Our code is publicly available at https: //github. com/ndsclark/MSPANet.

Details DOI

EAAI Journal 2024 Journal Article

TinyDepth: Lightweight self-supervised monocular depth estimation based on transformer

Zeyu Cheng
Yi Zhang
Yang Yu
Zhe Song
Chengkai Tang

Monocular depth estimation plays an important role in autonomous driving, virtual reality, augmented reality, and other fields. Self-supervised monocular depth estimation has received much attention because it does not require hard-to-obtain depth labels during training. The previously used convolutional neural network (CNN) has shown limitations in modeling large-scale spatial dependencies. A new idea for monocular depth estimation is replacing the CNN architecture or merging it with a Vision Transformer (ViT) architecture that can model large-scale spatial dependencies in images. However, there are still problems with too many parameters and calculations, making deployment difficult on mobile platforms. In response to these problems, we propose TinyDepth, a lightweight self-supervised monocular depth estimation method based on Transformer that employs hierarchical representation learning suitable for dense prediction, uses mobile convolution to reduce parameters and computational overhead. and includes a novel decoder based on multi-scale fusion attention that improves the local and global inference capability of the network through scale-wise attention processing and layer-wise fusion sampling for more accurate depth prediction. In experiments, TinyDepth achieved state-of-the-art results with few parameters on the Karlsruhe Institute of Technology and Toyota Technological Institute at Chicago (KITTI) dataset, and exhibited good generalization ability on the challenging indoor New York University (NYU) dataset. Source code is available at https: //github. com/ZYCheng777/TinyDepth.

Details DOI

EAAI Journal 2023 Journal Article

MCA: Multidimensional collaborative attention in deep convolutional neural networks for image recognition

Yang Yu
Yi Zhang
Zeyu Cheng
Zhe Song
Chengkai Tang

A broad range of prior research has demonstrated that attention mechanisms offer great potential in advancing the performance of deep convolutional neural networks (CNNs). However, most existing approaches either ignore modeling attention in both channel and spatial dimensions or introduce higher model complexity and heavier computational burden. To alleviate this dilemma, in this paper, we propose a lightweight and efficient multidimensional collaborative attention, MCA, a novel method for simultaneously inferring attention in channel, height, and width dimensions with almost free additional overhead by using a three-branch architecture. For the essential components of MCA, we not only develop an adaptive combination mechanism for merging dual cross-dimension feature responses in squeeze transformation, enhancing the informativeness and discriminability of feature descriptors but also design a gating mechanism in excitation transformation that adaptively determines the coverage of interaction to capture local feature interactions, overcoming the paradox of performance and computational overhead trade-off. Our MCA is simple yet general and can be easily plugged into various classic CNNs as a plug-and-play module and trained along with the vanilla networks in an end-to-end manner. Extensive experimental results for image recognition on CIFAR and ImageNet-1K datasets demonstrate the superiority of our method over other state-of-the-art (SOTA) counterparts. In addition, we also provide insight into the practical benefits of MCA by visually inspecting the GradCAM++ visualization results. The code is available at https: //github. com/ndsclark/MCANet.

Details DOI