CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation

Benchmark datasets have a significant impact on accelerating research in programming language tasks. In this paper, we introduce CodeXGLUE, a benchmark dataset to foster machine learning research for program understanding and generation. CodeXGLUE includes a collection of 10 tasks across 14 datasets and a platform for model evaluation and comparison. CodeXGLUE also features three baseline systems, including the BERT-style, GPT-style, and Encoder-Decoder models, to make it easy for researchers to use the platform. The availability of such data and baselines can help the development and validation of new methods that can be applied to various program understanding and generation problems.

Authors

• Shuai Lu Tsinghua University
• Daya Guo
• Shuo Ren
• Junjie Huang Shanghai Jiao Tong University
• Alexey Svyatkovskiy
• Ambrosio Blanco
• Colin Clement
• Dawn Drain
• Daxin Jiang Microsoft Corporation
• Duyu Tang Huawei Inc.
• Ge Li Guangdong Provincial Key Laboratory of Ultra High Definition Immersive Media Technology, Shenzhen Graduate School, Peking University
• Lidong Zhou
• Linjun Shou Microsoft Search Technology Center Asia (STCA), Beijing, China
• Long Zhou
• Michele Tufano
• Ming Gong Microsoft Search Technology Center Asia (STCA), Beijing, China
• Ming Zhou Langboat Technology, Beijing, China
• Nan Duan Microsoft
• Neel Sundaresan
• Shao Kun Deng
• Shengyu Fu
• Shujie Liu

Keywords

No keywords are indexed for this paper.