DGETP: Dynamic Graph Attention Network for Embodied Task Planning

With the development of embodied intelligence, many studies have made progress by incorporating scene graphs and GNN into task planning. However, most methods still face challenges in fully capturing the sequential relationships between agent actions and the environment, making it difficult to handle dynamic changes and complexity inherent in embodied tasks. This paper proposes a Dynamic Graph Attention Network for Embodied Task Planning (DGETP) to process scene graph sequences and robot graphs for dynamic environment perception. In DGETP, we design a Hierarchical Dynamic Graph Attention network (H-DGAT) by employing both structural and temporal attention mechanisms to model the dynamic evolution feature of the scene. A Dual-branch Action-object Predictor (DAP) is proposed in DGETP through introducing sequences of previous actions and objects to efficiently aggregate historical information. DAP captures temporal dependencies between past and future actions through explicit sequence modeling, and reduces prediction complexity via a dual-branch architecture that separates action and object prediction while preserving their correlations through targeted feature fusion. Experiments show that DGETP improves task accuracy by over 30% in seen scenes and over 15% in unseen scenes compared to other baselines. In complex scenes, DGETP demonstrates strong generalization ability. Finally, the simulation environment indicates that DGETP achieves more goals than most of the advanced task planning method.

Authors

• Pengfei Sun
• Guiling Wang
• Xinli Zhang
• Jian Yu

Keywords

• Accuracy
• Correlation
• Attention mechanisms
• Aggregates
• Predictive models
• Planning
• Complexity theory
• Intelligent robots
• Faces
• Dynamic Network
• Task Planning
• Graph Attention Network
• Dynamic Changes
• Sequence Of Actions
• Attention Mechanism
• Simulation Environment
• Historical Information
• Explicit Model
• Activity Prediction
• Task Accuracy
• Temporal Dependencies
• Sequence Graph
• Hierarchical Graph
• Scene Graph
• Hierarchical Attention
• Semantic
• Time Step
• Output Layer
• Complex Environment
• Nodes In The Graph
• Number Of Objects
• Task Instructions
• Node Embeddings
• Baseline Methods
• Current Time Step
• Self-attention Layer
• Sequential Steps
• Temporal Layer
• Longer Sequences