Self-verified user simulator via code-based interpretation in task-oriented dialogues

User simulators are essential for training and evaluating task-oriented dialogue systems (TODs). Recently, large language models (LLMs) have been increasingly adopted to construct user simulators by prompting them to generate natural language utterances and dialogue actions. However, due to the difficulty of controlling structured outputs through natural language prompts alone, these LLM-based simulators often produce incomplete, inconsistent, or invalid dialogue actions, limiting their effectiveness. To tackle this, this paper proposes a self-verified code-based user simulator that guides LLMs to generate intermediate Python code for structured dialogue actions. These code snippets are executed and validated by an external interpreter, and the verified outputs are used to refine the simulator’s behavior. Experiments on the Multi-Domain Wizard-of-Oz (MultiWOZ) dataset demonstrate that our method improves dialogue action accuracy by 4. 0%, and significantly enhances utterance diversity, achieving 12. 1% more trigrams, an increase of 0. 93 in entropy, and a 13. 2% gain in measure of textual lexical diversity (MTLD) over 100 dialogue turns. These results highlight the effectiveness of code-level verification in improving the controllability, correctness, and expressiveness of LLM-based user simulators.

Authors

• Xiang Luo
• Jin Wang
• Liang-Chih Yu Yuan Ze University
• Xuejie Zhang Yunnan University

Keywords

• Task-oriented dialogue
• Large language model
• User simulator