ToolACE-R: Model-aware Iterative Training and Adaptive Refinement for Tool learning

Tool learning, which allows Large Language Models (LLMs) to leverage external tools for solving complex user tasks, has emerged as a promising avenue for extending model capabilities. However, existing approaches primarily focus on data synthesis for fine-tuning LLMs to invoke tools effectively, largely ignoring how to fully stimulate the potential of the model. In this paper, we propose ToolACE-R, a novel framework that includes both model-aware iterative training and adaptive refinement for tool learning. ToolACE-R features a model-aware iterative training procedure that progressively adjust training samples based on the model’s evolving capabilities to maximize its potential. Additionally, it incorporates self-refinement training corpus which emphasizes LLM's ability to iteratively refine their tool calls, optimizing performance without requiring external feedback. Furthermore, we introduce adaptive self-refinement for efficient test-time scaling, where the trained model can autonomously determine when to stop the process based on iterative self-refinement. We conduct extensive experiments across several benchmark datasets, showing that ToolACE-R achieves competitive performance compared to advanced LLMs. The performance can be further improved efficiently through adaptive self-refinement. These results highlight the effectiveness and generalizability of ToolACE-R, offering a promising direction for more efficient and scalable tool learning.

Authors

• Xingshan Zeng Huawei Noah's Ark Lab, Hong Kong, China
• Weiwen Liu Shanghai Jiao Tong University
• Xu Huang Harbin Institute of Technology, Shenzhen
• Zezhong Wang Huawei Technologies Ltd.
• Lingzhi Wang Harbin Institute of Technology, Shenzhen, China
• Liangyou Li
• Yasheng Wang Huawei Noah’s Ark Lab
• Lifeng Shang Huawei Noah’s Ark Lab, Shenzhen, Guangdong, China
• Xin Jiang School of Electronic Engineering, Xidian University, Xi’an, China
• Ruiming Tang Huawei Noah's Ark Lab
• Qun Liu Huawei Noah's Ark Lab

Keywords

No keywords are indexed for this paper.