Dynamic-Static Synergistic Selection Method for Candidate Code Solutions with Generated Test Cases

Large language models (LLMs) show significant improvement in code generation. A common practice is sampling multiple candidate codes to increase the likelihood of producing an accurate solution. However, effectively identifying the best candidate from the pool is a significant challenge. Although existing code consensus methods attempt to solve this issue, they suffer from a critical problem: relying on test cases generated by LLMs, which can be flawed or provide incomplete coverage. This problem can result in erroneous validations, causing correct code to fail flawed tests and preventing the detection of functional differences in candidate code solutions. To address these issues, we present the Dynamic-Static Synergistic Selection Method, a novel framework that combines two complementary analytical approaches. First, it uses the abstract syntax tree (AST) to detect and filter candidate solutions and test cases. Second, the method statically analyzes the quality of the solutions and then dynamically validates functional consistency based on the execution results of the extracted inputs, thereby neutralizing the impact of faulty tests. Extensive experiments demonstrate that this synergistic approach significantly outperforms existing methods, substantially enhancing the correctness of the selected code.

Authors

• Ren-Biao Liu Nanjing University
• Jiang-Tian Xue Nanjing University
• Chao-Zeng Ma Nanjing University
• Hui Sun Nankai-Baidu Joint Laboratory (NBJL), College of Computer Science, Nankai University (NKU), Tianjin 300350, China.
• Xin-Ye Li Nanjing University
• Ming Li

Keywords

No keywords are indexed for this paper.