Explainable and Efficient Sequential Correlation Network for 3D Single Person Concurrent Activity Detection

We present the sequential correlation network (SCN) to improve concurrent activity detection. SCN combines a recurrent neural network and a correlation model hierarchically to model the complex correlations and temporal dynamics of concurrent activities. SCN has several advantages that enable effective learning even from a small dataset for real-world deployment. Unlike the majority of approaches assuming that each subject performs one activity at a time, SCN is end-to- end trainable, i. e. , it can automatically learn the inclusive or exclusive relations of concurrent activities. SCN is lightweight in design using only a small set of learnable parameters to model the spatio-temporal correlations of activities. This also enhances the explainability of the learned parameters. Furthermore, the learning of SCN can benefit from the initialization using semantically meaningful priors. We evaluate the proposed method against the state-of-the-art method on two benchmark datasets with human skeletal data, SCN achieves comparable performance to the SOTA but with much faster inference speed and less memory usage.

Authors

• Yi Wei 0006
• Wenbo Li 0001
• Ming-Ching Chang
• Hongxia Jin
• Siwei Lyu

Keywords

• Correlation
• Three-dimensional displays
• Recurrent neural networks
• Memory management
• Streaming media
• Reliability
• Intelligent robots
• Single Person
• Concurrent Activation
• Concurrent Detection
• Neural Network
• Recurrent Network
• Learnable Parameters
• Correlation Of Activity
• Spatiotemporal Correlation
• Body Parts
• Greater Than Or Equal
• Marginal Likelihood
• Graphical Model
• Black Arrows
• Temporal Correlation
• Action Recognition
• Activity Classification
• Pose Estimation
• Time Axis
• Graph Convolutional Network
• Random Initialization
• Graph Neural Networks
• Amount Of Parameters
• Wear Shoes
• Recurrent Layers
• Softplus
• Spatiotemporal Model