Author name cluster

Michael S. Lew

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

4 papers

1 author row

EAAI Journal 2024 Journal Article

Core-attributes enhanced generative adversarial networks for robust image enhancement

Shan Liu
Guoqiang Xiao
Michael S. Lew
Xinbo Gao
Song Wu

Automated image enhancement algorithms have a profound impact on human life today. To solve the problems of luminance, lack of detail information, and overall color tone bias of images taken by mobile devices, a novel framework of core-attributes enhanced generative adversarial network (CAE-GAN) is designed to improve these core attributes of enhanced images. The generator in CAE-GAN mainly consists of a luminance correction encoder (LCE) and a high-frequency supplementary decoder (HFSD). To target the adaptive luminance improvement for each location, the encoder based on LCE is designed by combining the extracted prior knowledge of luminance. Meanwhile, a decoder based on HFSD is proposed to fill in missing edge details during the image reconstruction process. In addition, a multi-scale statistical characteristics distinction branch (MSCDB) is proposed to correct the overall tone. Moreover, an upgrade adversarial loss function is designed to focus on the discrimination of both multi-scale and multi-perspective. The generator and discriminator are iteratively trained under the constraints of the total loss function, resulting in the generator that automatically improves the visualization of the images. Extensive experiments have shown that our CAE-GAN is capable of achieving excellent results in several evaluation metrics and subjective results. The source code of the proposed CAE-GAN is available at https: //github. com/SWU-CS-MediaLab/CAE-GAN.

Details DOI

EAAI Journal 2024 Journal Article

Deep cross-modal hashing with multi-task latent space learning

Song Wu
Xiang Yuan
Guoqiang Xiao
Michael S. Lew
Xinbo Gao

Cross-modal Hashing (CMH) retrieval aims to mutually search data from heterogeneous modalities by projecting original modality data into a common hamming space, with the significant advantages of low storage and computing costs. However, CMH remains challenging for multi-label cross-modal datasets. Firstly, preserving content similarity would inevitably be deficient under the representation of short-length binary codes. Secondly, different semantics are treated independently, whereas their co-occurrences are neglected, reducing retrieval quality. Thirdly, the commonly used metric learning objective is ineffective in capturing similarity information at a fine-grained level, leading to the imprecise preservation of such information. Therefore, we propose a Deep Cross-Modal Hashing with Multi-Task Latent Space Learning (DMLSH) framework to tackle these bottlenecks. For a more thorough excavation of distinctive features with diverse characteristics underneath heterogeneous data, our DMLSH is designed to preserve three different types of knowledge. The first is the semantic relevance and co-occurrence with the integration of the attention module and the Long Short-Term Memory (LSTM) layer; The second is the highly precise pairwise correlation considering the quantification of semantic similarity with self-paced optimization; The last is the pairwise similarity information discovered by a self-supervised semantic network from a perspective of probabilistic knowledge transfer. Abundant knowledge from the latent spaces is seamlessly refined and fused into a common Hamming space by a hashing attention mechanism, facilitating the discrimination of hash codes and the elimination of modalities’ heterogeneity. Exhaustive experiments demonstrate the state-of-the-art performance of our proposed DMLSH on four mainstream cross-modal retrieval benchmarks.

Details DOI

EAAI Journal 2024 Journal Article

Modality Blur and Batch Alignment Learning for Twin Noisy Labels-based Visible–infrared Person Re-identification

Song Wu
Shihao Shan
Guoqiang Xiao
Michael S. Lew
Xinbo Gao

The issue of Twin Noisy Labels, known as Noisy Annotations and Noisy Correspondences, increases the challenge in the engineering application of Visible–Infrared Person Re-identification (VI-ReID). This paper proposes an novel Modality Blur and Batch Alignment (MBBA) framework to address this issue in the practical application of VI-ReID. The MBBA consists of the Label Confidence Learning (LCL) module, the Modality Blur Learning (MBL) module, and the Batch Alignment Learning (BAL) module. The LCL utilizes the memorization effect of deep neural networks to estimate the confidence level of identity labels and rectify the noisy annotations and the noisy correspondences. The MBL uses the noiseless modality labels and the center loss to blur the boundary among modality data to make the discrimination of the learned latent feature space robust. Based on the designed alignment loss, the BAL employs the self-attention mechanism to align the significant prediction distributions among the cross-modal sample pairs from a batch-size perspective. The mean Average Precision (mAP) is improved by 4. 06% and 5. 34% compared to state-of-the-art methods on the 20% noisy RegDB dataset. And our MBBA exhibits contemporary state-of-the-art performance on Twin Noisy Labels based VI-ReID. Our MBBA is available at https: //github. com/SWU-CS-MediaLab/MBBA.

Details DOI

AAAI Conference 2023 Conference Paper

COCA: COllaborative CAusal Regularization for Audio-Visual Question Answering

Mingrui Lao
Nan Pu
Yu Liu
Kai He
Erwin M. Bakker
Michael S. Lew

Audio-Visual Question Answering (AVQA) is a sophisticated QA task, which aims at answering textual questions over given video-audio pairs with comprehensive multimodal reasoning. Through detailed causal-graph analyses and careful inspections of their learning processes, we reveal that AVQA models are not only prone to over-exploit prevalent language bias, but also suffer from additional joint-modal biases caused by the shortcut relations between textual-auditory/visual co-occurrences and dominated answers. In this paper, we propose a COllabrative CAusal (COCA) Regularization to remedy this more challenging issue of data biases. Specifically, a novel Bias-centered Causal Regularization (BCR) is proposed to alleviate specific shortcut biases by intervening bias-irrelevant causal effects, and further introspect the predictions of AVQA models in counterfactual and factual scenarios. Based on the fact that the dominated bias impairing model robustness for different samples tends to be different, we introduce a Multi-shortcut Collaborative Debiasing (MCD) to measure how each sample suffers from different biases, and dynamically adjust their debiasing concentration to different shortcut correlations. Extensive experiments demonstrate the effectiveness as well as backbone-agnostic ability of our COCA strategy, and it achieves state-of-the-art performance on the large-scale MUSIC-AVQA dataset.

PDF Details DOI