Decoupled Graph Spatial-Temporal Transformer Networks for traffic flow forecasting

Recent research has extensively applied the Transformer architecture to traffic flow prediction, proposing various solutions to effectively capture the nonlinear spatial-temporal correlations of traffic data. To better model traffic flow data, some works have highlighted the need to simultaneously consider the spatial-temporal heterogeneity during the learning of spatial-temporal correlations of traffic flow data. However, none of the existing methods have successfully achieved the goal of effectively capturing spatial-temporal heterogeneity while learning spatial-temporal correlations. To address these challenges, we propose a model for traffic flow prediction, named Decoupled Graph Spatial-Temporal Transformer Networks (DSTTN). The core of this model is a Spatial-Temporal Decoupled Representation Learning module designed to decouple spatial-temporal embedding and apply it to Transformer Networks, integrating with the Multi-Head Attention Mechanism. Through the Spatial-Temporal Decoupled Representation Learning Attention Mechanism, the model effectively achieves spatial-temporal decoupled learning of input feature embedding after spatial-temporal embedding structure embedding, thereby enhancing the ability of the model to learn spatial-temporal features for representation learning. This enables the model to extract spatial-temporal correlations and model spatial-temporal heterogeneity based on the Graph Transformer model. Experiments conducted on six real-world traffic datasets demonstrate that our model outperforms twelve comparative models, validating the ability of the model to perform more accurate prediction tasks. Additionally, ablation experiments confirm the effectiveness of the proposed spatial-temporal decoupling structure based on Transformer.

Authors

• Wei Sun IBM Research
• Rongzhang Cheng
• Yingqi Jiao
• Junbo Gao

Keywords

• Transformer
• Traffic flow prediction
• Spatial-temporal decoupled representation learning
• Spatial-temporal heterogeneity