Zeyu Sun

AAAI Conference 2026 Conference Paper

DSCodeBench: A Realistic Benchmark for Data Science Code Generation

• Shuyin Ouyang
• Dong Huang
• Jingwen Guo
• Zeyu Sun
• Qihao Zhu
• Jie M. Zhang

We introduce DSCodeBench, a new benchmark designed to evaluate large language models (LLMs) on complicated and realistic data science code generation tasks. DSCodeBench consists of 1,000 carefully constructed problems sourced from realistic problems from GitHub across ten widely used Python data science libraries. DSCodeBench offers a more challenging and representative testbed, more complex code solutions, more comprehensive data science libraries, clearer and better structured problem descriptions, and stronger test suites. To construct the DSCodeBench, we develop a robust pipeline that combines task scope selection, code construction, test case generation, and problem description synthesis. The process is paired with rigorous manual editing to ensure alignment and enhance the reliability of the evaluation. Experimental result shows that DSCodeBench exhibits robust scaling behavior, where larger models systematically outperform smaller ones, validating its ability to distinguish model capabilities. The best LLM we test, GPT-4o, has a pass@1 of 0.392, indicating that LLMs still have a large room to improve for realistic data science code generation tasks. We believe DSCodeBench will serve as a rigorous and trustworthy foundation for advancing LLM-based data science programming.

EAAI Journal 2025 Journal Article

Corrigendum to “Neural network energy management strategy for plug-in hybrid electric combine harvesters based on quasi-periodic samples” [Eng. Appl. Artiϧ. Intell. 136 (2024) 109051]

• Shuofeng Weng
• Chaochun Yuan
• Youguo He
• Jie Shen
• Long Chen
• Lizhang Xu
• Zhihao Zhu
• Qiuye Yu

EAAI Journal 2024 Journal Article

Neural network energy management strategy for plug-in hybrid electric combine harvesters based on quasi-periodic samples

• Shuofeng Weng
• Chaochun Yuan
• Youguo He
• Jie Shen
• Long Chen
• Lizhang Xu
• Zhihao Zhu
• Qiuye Yu

Energy management strategies are crucial for Plug-in Hybrid Electric Combine Harvester (PHECH). However, many existing approaches rely on rigid, pre-setting rules that struggle to adjust to the PHECH operational conditions. This paper first introduces a power estimation model tailored to the quasi-periodic process of harvester activity. Then, Dynamic Programming (DP) is applied to derive optimal samples of engine power ratio across various scenarios. Building on the samples, a Neural Network (NN) is developed to enhance the strategy's economic and real-time performance. Simulation tests evaluate the proposed algorithm's efficacy and its energy conservation potential. The findings suggest that, compared to fuel-driven harvesters, the NN strategy achieves similar energy cost savings to the DP approach, exceeding 11%, which is better than the Charge Depleting and Charge Sustaining (CDCS) strategy's 7. 22% and the MPC-ECMS strategy's 7. 87%. Moreover, the NN strategy reduces the time expense to roughly one-fifth of that required by the DP approach.

AAAI Conference 2024 Conference Paper

Performative Federated Learning: A Solution to Model-Dependent and Heterogeneous Distribution Shifts

• Kun Jin
• Tongxin Yin
• Zhongzhu Chen
• Zeyu Sun
• Xueru Zhang
• Yang Liu
• Mingyan Liu

We consider a federated learning (FL) system consisting of multiple clients and a server, where the clients aim to collaboratively learn a common decision model from their distributed data. Unlike the conventional FL framework that assumes the client's data is static, we consider scenarios where the clients' data distributions may be reshaped by the deployed decision model. In this work, we leverage the idea of distribution shift mappings in performative prediction to formalize this model-dependent data distribution shift and propose a performative FL framework. We first introduce necessary and sufficient conditions for the existence of a unique performative stable solution and characterize its distance to the performative optimal solution. Then we propose the performative FedAvg algorithm and show that it converges to the performative stable solution at a rate of O(1/T) under both full and partial participation schemes. In particular, we use novel proof techniques and show how the clients' heterogeneity influences the convergence. Numerical results validate our analysis and provide valuable insights into real-world applications.

NeurIPS Conference 2023 Conference Paper

Minimum-Risk Recalibration of Classifiers

• Zeyu Sun
• Dogyoon Song
• Alfred Hero

Recalibrating probabilistic classifiers is vital for enhancing the reliability and accuracy of predictive models. Despite the development of numerous recalibration algorithms, there is still a lack of a comprehensive theory that integrates calibration and sharpness (which is essential for maintaining predictive power). In this paper, we introduce the concept of minimum-risk recalibration within the framework of mean-squared-error (MSE) decomposition, offering a principled approach for evaluating and recalibrating probabilistic classifiers. Using this framework, we analyze the uniform-mass binning (UMB) recalibration method and establish a finite-sample risk upper bound of order $\tilde{O}(B/n + 1/B^2)$ where $B$ is the number of bins and $n$ is the sample size. By balancing calibration and sharpness, we further determine that the optimal number of bins for UMB scales with $n^{1/3}$, resulting in a risk bound of approximately $O(n^{-2/3})$. Additionally, we tackle the challenge of label shift by proposing a two-stage approach that adjusts the recalibration function using limited labeled data from the target domain. Our results show that transferring a calibrated classifier requires significantly fewer target samples compared to recalibrating from scratch. We validate our theoretical findings through numerical simulations, which confirm the tightness of the proposed bounds, the optimal number of bins, and the effectiveness of label shift adaptation.

AAAI Conference 2022 Conference Paper

Generalized Equivariance and Preferential Labeling for GNN Node Classification

• Zeyu Sun
• Wenjie Zhang
• Lili Mou
• Qihao Zhu
• Yingfei Xiong
• Lu Zhang

Existing graph neural networks (GNNs) largely rely on node embeddings, which represent a node as a vector by its identity, type, or content. However, graphs with unattributed nodes widely exist in real-world applications (e. g. , anonymized social networks). Previous GNNs either assign random labels to nodes (which introduces artefacts to the GNN) or assign one embedding to all nodes (which fails to explicitly distinguish one node from another). Further, when these GNNs are applied to unattributed node classification problems, they have an undesired equivariance property, which are fundamentally unable to address the data with multiple possible outputs. In this paper, we analyze the limitation of existing approaches to node classification problems. Inspired by our analysis, we propose a generalized equivariance property and a Preferential Labeling technique that satisfies the desired property asymptotically. Experimental results show that we achieve high performance in several unattributed node classification tasks.

IJCAI Conference 2022 Conference Paper

Grape: Grammar-Preserving Rule Embedding

• Qihao Zhu
• Zeyu Sun
• Wenjie Zhang
• Yingfei Xiong
• Lu Zhang

Word embedding has been widely used in various areas to boost the performance of the neural models. However, when processing context-free languages, embedding grammar rules with word embedding loses two types of information. One is the structural relationship between the grammar rules, and the other one is the content information of the rule definition. In this paper, we make the first attempt to learn a grammar-preserving rule embedding. We first introduce a novel graph structure to represent the context-free grammar. Then, we apply a Graph Neural Network (GNN) to extract the structural information and use a gating layer to integrate content information. We conducted experiments on six widely-used benchmarks containing four context-free languages. The results show that our approach improves the accuracy of the base model by 0. 8 to 6. 4 percentage points. Furthermore, Grape also achieves 1. 6 F1 score improvement on the method naming task which shows the generality of our approach.

IJCAI Conference 2022 Conference Paper

Lyra: A Benchmark for Turducken-Style Code Generation

• Qingyuan Liang
• Zeyu Sun
• Qihao Zhu
• Wenjie Zhang
• Lian Yu
• Yingfei Xiong
• Lu Zhang

Recently, neural techniques have been used to generate source code automatically. While promising for declarative languages, these approaches achieve much poorer performance on datasets for imperative languages. Since a declarative language is typically embedded in an imperative language (i. e. , the turducken-style programming) in real-world software development, the promising results on declarative languages can hardly lead to significant reduction of manual software development efforts. In this paper, we define a new code generation task: given a natural language comment, this task aims to generate a program in a base imperative language with an embedded declarative language. To our knowledge, this is the first turducken-style code generation task. For this task, we present Lyra: a dataset in Python with embedded SQL. This dataset contains 2, 000 carefully annotated database manipulation programs from real usage projects. Each program is paired with both a Chinese comment and an English comment. In our experiment, we adopted Transformer, BERT-style, and GPT-style models as baselines. In the best setting, GPT-style model can achieve 24% and 25. 5% AST exact matching accuracy using Chinese and English comments, respectively. Therefore, we believe that Lyra provides a new challenge for code generation. Yet, overcoming this challenge may significantly boost the applicability of code generation techniques for real-world software development.

IJCAI Conference 2020 Conference Paper

NLocalSAT: Boosting Local Search with Solution Prediction

• Wenjie Zhang
• Zeyu Sun
• Qihao Zhu
• Ge Li
• Shaowei Cai
• Yingfei Xiong
• Lu Zhang

The Boolean satisfiability problem (SAT) is a famous NP-complete problem in computer science. An effective way for solving a satisfiable SAT problem is the stochastic local search (SLS). However, in this method, the initialization is assigned in a random manner, which impacts the effectiveness of SLS solvers. To address this problem, we propose NLocalSAT. NLocalSAT combines SLS with a solution prediction model, which boosts SLS by changing initialization assignments with a neural network. We evaluated NLocalSAT on five SLS solvers (CCAnr, Sparrow, CPSparrow, YalSAT, and probSAT) with instances in the random track of SAT Competition 2018. The experimental results show that solvers with NLocalSAT achieve 27% ~ 62% improvement over the original SLS solvers.

AAAI Conference 2020 Conference Paper

TreeGen: A Tree-Based Transformer Architecture for Code Generation

• Zeyu Sun
• Qihao Zhu
• Yingfei Xiong
• Yican Sun
• Lili Mou
• Lu Zhang

A code generation system generates programming language code based on an input natural language description. State-ofthe-art approaches rely on neural networks for code generation. However, these code generators suffer from two problems. One is the long dependency problem, where a code element often depends on another far-away code element. A variable reference, for example, depends on its definition, which may appear quite a few lines before. The other problem is structure modeling, as programs contain rich structural information. In this paper, we propose a novel tree-based neural architecture, TreeGen, for code generation. TreeGen uses the attention mechanism of Transformers to alleviate the longdependency problem, and introduces a novel AST reader (encoder) to incorporate grammar rules and AST structures into the network. We evaluated TreeGen on a Python benchmark, HearthStone, and two semantic parsing benchmarks, ATIS and GEO. TreeGen outperformed the previous state-of-theart approach by 4. 5 percentage points on HearthStone, and achieved the best accuracy among neural network-based approaches on ATIS (89. 1%) and GEO (89. 6%). We also conducted an ablation test to better understand each component of our model.

AAAI Conference 2019 Conference Paper

A Grammar-Based Structural CNN Decoder for Code Generation

• Zeyu Sun
• Qihao Zhu
• Lili Mou
• Yingfei Xiong
• Ge Li
• Lu Zhang

Code generation maps a program description to executable source code in a programming language. Existing approaches mainly rely on a recurrent neural network (RNN) as the decoder. However, we find that a program contains significantly more tokens than a natural language sentence, and thus it may be inappropriate for RNN to capture such a long sequence. In this paper, we propose a grammar-based structural convolutional neural network (CNN) for code generation. Our model generates a program by predicting the grammar rules of the programming language; we design several CNN modules, including the tree-based convolution and pre-order convolution, whose information is further aggregated by dedicated attentive pooling layers. Experimental results on the HearthStone benchmark dataset show that our CNN code generator significantly outperforms the previous state-of-the-art method by 5 percentage points; additional experiments on several semantic parsing tasks demonstrate the robustness of our model. We also conduct in-depth ablation test to better understand each component of our model.