Just Noticeable Visual Redundancy Forecasting: A Deep Multimodal-Driven Approach

Just noticeable difference (JND) refers to the maximum visual change that human eyes cannot perceive, and it has a wide range of applications in multimedia systems. However, most existing JND approaches only focus on a single modality, and rarely consider the complementary effects of multimodal information. In this article, we investigate the JND modeling from an end-to-end homologous multimodal perspective, namely hmJND-Net. Specifically, we explore three important visually sensitive modalities, including saliency, depth, and segmentation. To better utilize homologous multimodal information, we establish an effective fusion method via summation enhancement and subtractive offset, and align homologous multimodal features based on a self-attention driven encoder-decoder paradigm. Extensive experimental results on eight different benchmark datasets validate the superiority of our hmJND-Net over eight representative methods.

Authors

• Wuyuan Xie College of Computer Science and Software Engineering, Guangdong Key Laboratory of Intelligent Information Processing, Shenzhen University
• Shukang Wang Shenzhen University
• Sukun Tian Peking University
• Lirong Huang Guangdong Key Laboratory of Intelligent Information Processing, Shenzhen University, China
• Ye Liu
• Miaohui Wang College of Computer Science and Software Engineering, Guangdong Key Laboratory of Intelligent Information Processing, Shenzhen University

Keywords

• CMS: Brain Modeling
• CV: Applications
• CV: Low Level & Physics-based Vision
• CV: Multi-modal Vision