Towards Verifiable Text Generation with Generative Agent

Text generation with citations makes it easy to verify the factuality of Large Language Models’ (LLMs) generations. Existing one-step generation studies expose distinct shortages in answer refinement and in-context demonstration matching. In light of these challenges, we propose R2-MGA, a Retrieval and Reflection Memory-augmented Generative Agent. Specifically, it first retrieves the memory bank to obtain the best-matched memory snippet, then reflects the retrieved snippet as a reasoning rationale, next combines the snippet and the rationale as the best-matched in-context demonstration. Additionally, it is capable of in-depth answer refinement with two specifically designed modules. We evaluate R2-MGA across five LLMs on the ALCE benchmark. The results reveal R2-MGA’ exceptional capabilities in text generation with citations. In particular, compared to the selected baselines, it delivers up to +58.8% and +154.7% relative performance gains on answer correctness and citation quality, respectively. Extensive analyses strongly support the motivations of R2-MGA.

Authors

• Bin Ji National University of Defense Technology
• Huijun Liu National University of Defense Technology
• Mingzhe Du Nanyang Technological University
• Shasha Li
• Xiaodong Liu National University of Defense Technology
• Jun Ma University of Electronic Science and Technology of China
• Jie Yu National University of Defense Technology
• See-Kiong Ng National University of Singapore

Keywords

No keywords are indexed for this paper.