CAPE: Corrective Actions from Precondition Errors using Large Language Models

Extracting knowledge and reasoning from large language models (LLMs) offers a path to designing intelligent robots. Common approaches that leverage LLMs for planning are unable to recover when actions fail and resort to retrying failed actions without resolving the underlying cause. We propose a novel approach (CAPE) that generates corrective actions to resolve precondition errors during planning. CAPE improves the quality of generated plans through few-shot reasoning on action preconditions. Our approach enables embodied agents to execute more tasks than baseline methods while maintaining semantic correctness and minimizing re-prompting. In VirtualHome, CAPE improves a human-annotated plan correctness metric from 28. 89% to 49. 63% over SayCan, whilst achieving competitive executability. Our improvements transfer to a Boston Dynamics Spot robot initialized with a set of skills (specified in language) and associated preconditions, where CAPE improves correctness by 76. 49% with higher executability compared to SayCan. Our approach enables embodied agents to follow natural language commands and robustly recover from failures.

Authors

• Shreyas Sundara Raman
• Vanya Cohen
• Ifrah Idrees
• Eric Rosen
• Raymond Mooney
• Stefanie Tellex
• David Paulius

Keywords

• Measurement
• Large language models
• Semantics
• Natural languages
• Cognition
• Planning
• Task analysis
• Preconditioning
• Corrective Actions
• Natural Language
• Baseline Methods
• Ablation
• Sequence Of Actions
• Cause Of Failure
• Semantic Similarity
• Path Planning
• Single Arm
• Log Probability
• Executive Skills
• Task Planning
• Few-shot Learning
• Error Feedback
• Degree Of Detail
• Environmental Feedback
• Plan Generation
• Scene Graph
• Planning Domain