Author name cluster

Weiwen Liu

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

12 papers

2 author rows

AAMAS Conference 2026 Conference Paper

Automatically Benchmarking LLM Code Agents through Agent-driven Annotation and Evaluation

Lingyue Fu
Bolun Zhang
Hao Guan
Yaoming Zhu
Lin Qiu
Weiwen Liu
Xuezhi Cao
Xunliang Cai

Recent advances in code agents have enabled automated software development at the project level, supported by large language models(LLMs)andwidelyadoptedtools. However, existingbenchmarks for code agent evaluation face two major limitations: high annotation cost and expertise requirements, and rigid evaluation metrics thatrelyprimarilyonunittests. Toaddressthesechallenges, wepropose an agent-driven benchmark construction pipeline that leverages human supervision to efficiently generate diverse and challenging project-level tasks. Based on this approach, we introduce PRDBench, a novel benchmark comprising 50 real-world Python projects across 20 domains, each with structured Product Requirement Document (PRD) requirements, comprehensive evaluation criteria, and reference implementations. PRDBench features rich data sources, high task complexity, and flexible metrics. We further employ an Agent-as-a-Judge paradigm to score agent outputs, enabling the evaluation of various test types beyond unit tests. Extensive experiments on PRDBench demonstrate its effectiveness in assessing the capabilities of both code agents and evaluation agents, providing a scalable and robust framework for annotation and evaluation.

PDF

TCS Journal 2026 Journal Article

Contextual combinatorial conservative bandits

Xiaojin Zhang
Weiwen Liu
Shuai Li
Shengyu Zhang

The problem of multi-armed bandits (MAB) asks to make sequential decisions while balancing between exploitation and exploration, and have been successfully applied to a wide range of practical scenarios. Various algorithms have been designed to achieve a high reward in a long term. However, its short-term performance might be rather low, which is injurious in risk sensitive applications. Building on previous work of conservative bandits, we bring up a framework of contextual combinatorial conservative bandits. An algorithm is presented and a regret bound of $\tilde O(d^2+d\sqrt{T})$ is proven, where $d$ is the dimension of the feature vectors, and $T$ is the total number of time steps. We further provide an algorithm as well as regret analysis for the case when the conservative reward is unknown. Experiments are conducted, and the results validate the effectiveness of our algorithm.

Details DOI

AAMAS Conference 2026 Conference Paper

DebugTA: An LLM-Based Agent for Simplifying Debugging and Teaching in Programming Education

Lingyue Fu
Datong Chen
Haowei Yuan
Xinyi Dai
Qingyao Li
Weinan Zhang
Weiwen Liu
Yong Yu

In programming education, Debugging and Teaching (DT) task is a common scenario which requires generating modification suggestions from erroneous code, error messages, reference solutions, and problem descriptions. Existing approaches struggle with complex multi-source reasoning and underutilize available reference code, limiting the effectiveness of large language models (LLMs) in DT tasks. To address these challenges, we propose DebugTA, a novel LLM-based debugging and teaching agent with specialized tools for standard code retrieval, variable substitution to align reference code, and an external compiler for real-time code analysis. Guided by pedagogical and debugging principles, DebugTA decomposes complexDTtasksintostructuredLLM–toolinteractionsthatreduce reasoning complexity. By aligning reference code with erroneous code, DebugTA enables the LLM to focus on logical errors and improves suggestion accuracy. To rigorously assess the quality of modification suggestions, we introduce a student simulator-teacher interactionparadigm. Experimentalresultsonthreereal-worldcode datasets demonstrate that DebugTA consistently improves teaching effectiveness while significantly reducing computational costs.

PDF

AAAI Conference 2026 Conference Paper

RecCocktail: A Generalizable and Efficient Framework for LLM-Based Recommendation

Min Hou
Chenxi Bai
Le Wu
Hao Liu
Kai Zhang
Weiwen Liu
Richang Hong
Ruiming Tang

Large Language Models (LLMs) have achieved remarkable success in recent years, owing to their impressive generalization capabilities and rich world knowledge. To capitalize on the potential of using LLMs as recommender systems, mainstream approaches typically focus on two paradigms. The first paradigm designs multi-domain or multi-task instruction data for generalizable recommendation, so as to align LLMs with general recommendation areas and deal with cold-start recommendation. The second paradigm focuses on enhancing domain-specific recommendation tasks, improving performance in warm recommendation scenarios. While most previous works treat these two paradigms separately, we argue that they have complementary advantages, and combining them can yield better results. In this paper, we propose a generalizable and efficient LLM-based recommendation framework RecCocktail. Our approach begins with fine-tuning a "base spirit" LoRA module using domain-general recommendation instruction data to align LLM with recommendation knowledge. Next, given users' behavior of a specific domain, we construct a domain-specific "ingredient" LoRA module. We then provide an entropy-guided adaptive merging method to mix the "base spirit" and the "ingredient" in the weight space. Please note that, RecCocktail combines the advantages of the existing two paradigms without introducing additional time or space overhead during the inference phase. Moreover, RecCocktail is efficient with plug and play, as the "base spirit" LoRA is trained only once, and any domain-specific "ingredient" can be efficiently mixed with only domain-specific fine-tuning. Extensive experiments on multiple datasets under both warm and cold-start recommendation scenarios validate the effectiveness and generality of the proposed RecCocktail.

PDF Details DOI

AAAI Conference 2026 Conference Paper

ToolACE-R: Model-aware Iterative Training and Adaptive Refinement for Tool learning

Xingshan Zeng
Weiwen Liu
Xu Huang
Zezhong Wang
Lingzhi Wang
Liangyou Li
Yasheng Wang
Lifeng Shang

Tool learning, which allows Large Language Models (LLMs) to leverage external tools for solving complex user tasks, has emerged as a promising avenue for extending model capabilities. However, existing approaches primarily focus on data synthesis for fine-tuning LLMs to invoke tools effectively, largely ignoring how to fully stimulate the potential of the model. In this paper, we propose ToolACE-R, a novel framework that includes both model-aware iterative training and adaptive refinement for tool learning. ToolACE-R features a model-aware iterative training procedure that progressively adjust training samples based on the model’s evolving capabilities to maximize its potential. Additionally, it incorporates self-refinement training corpus which emphasizes LLM's ability to iteratively refine their tool calls, optimizing performance without requiring external feedback. Furthermore, we introduce adaptive self-refinement for efficient test-time scaling, where the trained model can autonomously determine when to stop the process based on iterative self-refinement. We conduct extensive experiments across several benchmark datasets, showing that ToolACE-R achieves competitive performance compared to advanced LLMs. The performance can be further improved efficiently through adaptive self-refinement. These results highlight the effectiveness and generalizability of ToolACE-R, offering a promising direction for more efficient and scalable tool learning.

PDF Details DOI

ICLR Conference 2025 Conference Paper

Proactive Agent: Shifting LLM Agents from Reactive Responses to Active Assistance

Yaxi Lu
Shenzhi Yang
Cheng Qian 0008
Guirong Chen
Qinyu Luo
Yesai Wu
Huadong Wang
Xin Cong

Agents powered by large language models have shown remarkable abilities in solving complex tasks. However, most agent systems remain reactive, limiting their effectiveness in scenarios requiring foresight and autonomous decision-making. In this paper, we tackle the challenge of developing proactive agents capable of anticipating and initiating tasks without explicit human instructions. We propose a novel data-driven approach for this problem. Firstly, we collect real-world human activities to generate proactive task predictions. These predictions are then labeled by human annotators as either accepted or rejected. The labeled data is used to train a reward model that simulates human judgment and serves as an automatic evaluator of the proactiveness of LLM agents. Building on this, we develop a comprehensive data generation pipeline to create a diverse dataset, ProactiveBench, containing 6,790 events. Finally, we demonstrate that fine-tuning models with the proposed ProactiveBench can significantly elicit the proactiveness of LLM agents. Experimental results show that our fine-tuned model achieves an F1-Score of 66.47% in proactively offering assistance, outperforming all open-source and close-source models. These results highlight the potential of our method in creating more proactive and effective agent systems, paving the way for future advancements in human-agent collaboration.

Details

ICLR Conference 2025 Conference Paper

Spa-Bench: a comprehensive Benchmark for Smartphone Agent Evaluation

Jingxuan Chen
Derek Yuen
Bin Xie
Yuhao Yang 0008
Gongwei Chen
Zhihao Wu
Li Yixing
Xurui Zhou

Smartphone agents are increasingly important for helping users control devices efficiently, with (Multimodal) Large Language Model (MLLM)-based approaches emerging as key contenders. Fairly comparing these agents is essential but challenging, requiring a varied task scope, the integration of agents with different implementations, and a generalisable evaluation pipeline to assess their strengths and weaknesses. In this paper, we present SPA-Bench, a comprehensive SmartPhone Agent Benchmark designed to evaluate (M)LLM-based agents in an interactive environment that simulates real-world conditions. SPA-Bench offers three key contributions: (1) A diverse set of tasks covering system and third-party apps in both English and Chinese, focusing on features commonly used in daily routines; (2) A plug-and-play framework enabling real-time agent interaction with Android devices, integrating over ten agents with the flexibility to add more; (3) A novel evaluation pipeline that automatically assesses agent performance across multiple dimensions, encompassing seven metrics related to task completion and resource consumption. Our extensive experiments across tasks and agents reveal challenges like interpreting mobile user interfaces, action grounding, memory retention, and execution costs. We propose future research directions to ease these difficulties, moving closer to real-world smartphone agent applications.

Details

ICLR Conference 2025 Conference Paper

ToolACE: Winning the Points of LLM Function Calling

Weiwen Liu
Xu Huang 0008
Xingshan Zeng
Xinlong Hao
Shuai Yu
Dexun Li
Shuai Wang 0020
Weinan Gan

Function calling significantly extends the application boundary of large language models (LLMs), where high-quality and diverse training data is critical for unlocking this capability. However, collecting and annotating real function-calling data is challenging, while synthetic data from existing pipelines often lack coverage and accuracy. In this paper, we present ToolACE, an automatic agentic pipeline designed to generate accurate, complex, and diverse tool-learning data, specifically tailored to the capabilities of LLMs. ToolACE leverages a novel self-evolution synthesis process to curate a comprehensive API pool of 26,507 diverse APIs. Dialogs are further generated through the interplay among multiple agents, under the guidance of a complexity evaluator. To ensure data accuracy, we implement a dual-layer verification system combining rule-based and model-based checks. We demonstrate that models trained on our synthesized data---even with only 8B parameters---achieve state-of-the-art performance, comparable to the latest GPT-4 models. Our model and a subset of the data are publicly available at https://huggingface.co/Team-ACE.

Details

IJCAI Conference 2023 Conference Paper

A Survey on User Behavior Modeling in Recommender Systems

Zhicheng He
Weiwen Liu
Wei Guo
Jiarui Qin
Yingxue Zhang
Yaochen Hu
Ruiming Tang

User Behavior Modeling (UBM) plays a critical role in user interest learning, which has been extensively used in recommender systems. Crucial interactive patterns between users and items have been exploited, which brings compelling improvements in many recommendation tasks. In this paper, we attempt to provide a thorough survey of this research topic. We start by reviewing the research background of UBM. Then, we provide a systematic taxonomy of existing UBM research works, which can be categorized into four different directions including Conventional UBM, Long-Sequence UBM, Multi-Type UBM, and UBM with Side Information. Within each direction, representative models and their strengths and weaknesses are comprehensively discussed. Besides, we elaborate on the industrial practices of UBM methods with the hope of providing insights into the application value of existing UBM solutions. Finally, we summarize the survey and discuss the future prospects of this field.

PDF Details DOI

AAAI Conference 2023 Conference Paper

Set-to-Sequence Ranking-Based Concept-Aware Learning Path Recommendation

Xianyu Chen
Jian Shen
Wei Xia
Jiarui Jin
Yakun Song
Weinan Zhang
Weiwen Liu
Menghui Zhu

With the development of the online education system, personalized education recommendation has played an essential role. In this paper, we focus on developing path recommendation systems that aim to generating and recommending an entire learning path to the given user in each session. Noticing that existing approaches fail to consider the correlations of concepts in the path, we propose a novel framework named Set-to-Sequence Ranking-based Concept-aware Learning Path Recommendation (SRC), which formulates the recommendation task under a set-to-sequence paradigm. Specifically, we first design a concept-aware encoder module which can capture the correlations among the input learning concepts. The outputs are then fed into a decoder module that sequentially generates a path through an attention mechanism that handles correlations between the learning and target concepts. Our recommendation policy is optimized by policy gradient. In addition, we also introduce an auxiliary module based on knowledge tracing to enhance the model’s stability by evaluating students’ learning effects on learning concepts. We conduct extensive experiments on two real-world public datasets and one industrial dataset, and the experimental results demonstrate the superiority and effectiveness of SRC. Code now is available at https://gitee.com/mindspore/models/tree/master/research/recommend/SRC.

PDF Details DOI

IJCAI Conference 2022 Conference Paper

Neural Re-ranking in Multi-stage Recommender Systems: A Review

Weiwen Liu
Yunjia Xi
Jiarui Qin
Fei Sun
Bo Chen
Weinan Zhang
Rui Zhang
Ruiming Tang

As the final stage of the multi-stage recommender system (MRS), re-ranking directly affects users’ experience and satisfaction by rearranging the input ranking lists, and thereby plays a critical role in MRS. With the advances in deep learning, neural re-ranking has become a trending topic and been widely adopted in industrial applications. This review aims at integrating re-ranking algorithms into a broader picture, and paving ways for more comprehensive solutions for future research. For this purpose, we first present a taxonomy of current methods on neural re-ranking. Then we give a description of these methods along with the historic development according to their objectives. The network structure, personalization, and complexity are also discussed and compared. Next, we provide a benchmark for the major neural re-ranking models and quantitatively analyze their re-ranking performance. Finally, the review concludes with a discussion on future prospects of this field. A list of papers discussed in this review, the benchmark datasets, our re-ranking library LibRerank, and detailed parameter settings are publicly available at https: //github. com/LibRerank-Community/LibRerank.

PDF Details DOI

ECAI Conference 2020 Conference Paper

Adversarial Learning for Overlapping Community Detection and Network Embedding

Junyang Chen 0001
Zhiguo Gong
Quanyu Dai
Chunyuan Yuan
Weiwen Liu

Network Embedding (NE) aims at modeling network graph by encoding vertices and edges into a low-dimensional space. These learned vectors which preserve proximities can be used for subsequent applications, such as vertex classification and link prediction. Skip-gram with negative sampling is the most widely used method for existing NE models to approximate their objective functions. However, this method only focuses on learning representation from the local connectivity of vertices (i. e. , neighbors). In real-world scenarios, a vertex may have multifaceted aspects and should belong to overlapping communities. For example, in a social network, a user may subscribe to political, economic and sports channels simultaneously, but the politics share more common attributes with the economy and less with the sports. In this paper, we propose an adversarial learning approach for modeling overlapping communities of vertices. Each community and vertex are mapped into an embedding space, while we also learn the association between each pair of community and vertex. The experimental results show that our proposed model not only can outperform the state-of-the-art (including GANs-based) models on vertex classification tasks but also can achieve superior performances on overlapping community detection.

Details