Intentional Evolutionary Learning for Untrimmed Videos with Long Tail Distribution

Human intention understanding in untrimmed videos aims to watch a natural video and predict what the person’s intention is. Currently, exploration of predicting human intentions in untrimmed videos is far from enough. On the one hand, untrimmed videos with mixed actions and backgrounds have a significant long-tail distribution with concept drift characteristics. On the other hand, most methods can only perceive instantaneous intentions, but cannot determine the evolution of intentions. To solve the above challenges, we propose a loss based on Instance Confidence and Class Accuracy (ICCA), which aims to alleviate the prediction bias caused by the long-tail distribution with concept drift characteristics in video streams. In addition, we propose an intention-oriented evolutionary learning method to determine the intention evolution pattern (from what action to what action) and the time of evolution (when the action evolves). We conducted extensive experiments on two untrimmed video datasets (THUMOS14 and ActivityNET v1.3), and our method has achieved excellent results compared to SOTA methods. The code and supplementary materials are available at https://github.com/Jennifer123www/UntrimmedVideo.

Authors

• Yuxi Zhou Tianjin University of Technology Tsinghua University
• Xiujie Wang Tianjin University of Technology
• Jianhua Zhang School of Computer Science and Engineering, Tianjin University of Technology, Tianjin, 300384, China
• Jiajia Wang Tianjin University of Technology
• Jie Yu National University of Defense Technology
• Hao Zhou Naval University of Engineering
• Yi Gao
• Shengyong Chen School of Computer Science and Engineering, Tianjin University of Technology, Tianjin, 300384, China

Keywords

• APP: Other Applications
• CV: 3D Computer Vision
• CV: Applications
• CV: Video Understanding & Activity Analysis
• DMKM: Mining of Visual, Multimedia & Multimodal Data
• HAI: Applications
• HAI: Human-Aware Planning and Behavior Prediction
• HAI: Human-Computer Interaction
• ML: Applications
• ML: Deep Neural Architectures and Foundation Models