AAMAS Conference 2026 Conference Paper
Towards Failure-Resilient Lifelong Learning Agents through Scene Graph-Guided Proactive Replanning
- Che Rin Yu
- Daewon Chae
- Dabin Seo
- Sangwon Lee
- Hyeongwoo IM
- Jinkyu Kim
When humans perform everyday tasks, we naturally adjust our actions based on the current state of the environment. For example, if weplantoheatabowlofsoupinthemicrowaveandseethatthere’s already a plate inside, we first remove it before proceeding. However, many autonomous robots lack this adaptive awareness. They often follow pre-planned actions that may overlook subtle yet critical changes in the scene, which can result in actions being executed under outdated assumptions and eventual failure. While replanning is critical for robust autonomy, most existing methods respond only after failures occur, when recovery may be inefficient or infeasible. In this work, we present a proactive replanning framework that anticipates and prevents failures before action execution. The key idea is to learn visual preconditions from successful demonstrations. Before each subtask, the system builds a scene graph from RGB-D observations and compares it with an expanding buffer of reference graphs from successful trials. When the similarity falls below a threshold, a lightweight reasoning module diagnoses the discrepancy and generates a corrective sub-plan. As the experience buffer grows, the system scales and generalizes more efficiently, highlighting a progressive path toward more autonomous and effective replanning. Experiments in both the AI2-THOR simulator and real-world platforms show that our approach improves task success and execution performance compared to baselines.