Author name cluster

Xiaoyun Mo

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

2 papers

1 author row

ECAI Conference 2025 Conference Paper

Diversity-Aware Self-Paced Data Selection for LLM Fine-Tuning

Yingxuan Yang
Huayi Wang
Muning Wen
Xiaoyun Mo
Qiuying Peng
Jun Wang 0020
Weinan Zhang 0001

Fine-tuning large language models (LLMs) is challenged by the presence of noisy data and the high computational cost when training on large-scale datasets. While data selection has emerged as a promising approach to reduce training cost and improve data quality, existing methods often rely on static heuristics or manual metrics. These approaches struggle to adapt to the model’s evolving capabilities during training, as its understanding of tasks improves. As the model becomes more powerful, its requirements for data that can enhance performance also change, making it crucial to incorporate this dynamic into the data selection process. Moreover, ensuring data diversity throughout different stages of training is essential for preventing redundancy, reducing overfitting. To address these issues, we propose DSP, a Diversity-Aware Self-Paced data selection framework that evolves with the model. DSP progressively selects training samples based on the model’s own outputs and incorporates a diversity-aware mechanism to enhance generalization and mitigate overfitting. Unlike prior static or rule-based strategies, DSP adaptively adjusts to the model’s internal feedback and training stage. Experiments on two public benchmarks demonstrate that DSP consistently outperforms static and heuristic-based baselines across multiple datasets and backbone models. Our findings highlight the critical role of dynamic, diversity-aware data selection in effective LLM fine-tuning.

Details

ICLR Conference 2025 Conference Paper

Robust Function-Calling for On-Device Language Model via Function Masking

Qiqiang Lin
Muning Wen
Qiuying Peng
Guanyu Nie
Junwei Liao
Xiaoyun Mo
Jiamu Zhou
Cheng Cheng

Large language models have demonstrated impressive value in performing as autonomous agents when equipped with external tools and API calls. Nonetheless, effectively harnessing their potential for executing complex tasks crucially relies on enhancements in their function-calling capabilities. This paper identifies a critical gap in existing function-calling models, where performance varies significantly across benchmarks, often due to over-fitting to specific naming conventions. To address such an issue, we introduce Hammer, a novel family of foundation models specifically engineered for on-device function calling. Hammer employs an augmented dataset that enhances models’ sensitivity to irrelevant functions and incorporates function masking techniques to minimize over-fitting. Our empirical evaluations reveal that Hammer not only outperforms larger models but also demonstrates robust generalization across diverse benchmarks, achieving state-of-the-art results. Our open-source contributions include a specialized dataset for irrelevance detection, a tuning framework for enhanced generalization, and the Hammer models, establishing a new standard for function-calling performance.

Details