CurBench: Curriculum Learning Benchmark

Curriculum learning is a training paradigm where machine learning models are trained in a meaningful order, inspired by the way humans learn curricula. Due to its capability to improve model generalization and convergence, curriculum learning has gained considerable attention and has been widely applied to various research domains. Nevertheless, as new curriculum learning methods continue to emerge, it remains an open issue to benchmark them fairly. Therefore, we develop CurBench, the first benchmark that supports systematic evaluations for curriculum learning. Specifically, it consists of 15 datasets spanning 3 research domains: computer vision, natural language processing, and graph machine learning, along with 3 settings: standard, noise, and imbalance. To facilitate a comprehensive comparison, we establish the evaluation from 2 dimensions: performance and complexity. CurBench also provides a unified toolkit that plugs automatic curricula into general machine learning processes, enabling the implementation of 15 core curriculum learning methods. On the basis of this benchmark, we conduct comparative experiments and make empirical analyses of existing methods. CurBench is open-source and publicly available at https: //github. com/THUMNLab/CurBench.

Authors

• Yuwei Zhou
• Zirui Pan
• Xin Wang 0019
• Hong Chen 0011
• Haoyang Li 0001
• Yanwen Huang
• Zhixiao Xiong
• Fangzhou Xiong
• Peiyang Xu
• Shengnan Liu
• Wenwu Zhu 0001

Keywords

No keywords are indexed for this paper.