Arrow Research search

Author name cluster

Fei Peng

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

2 papers
1 author row

Possible papers

2

NeurIPS Conference 2025 Conference Paper

Group-Level Data Selection for Efficient Pretraining

  • Zichun Yu
  • Fei Peng
  • Jie Lei
  • Arnold Overwijk
  • Scott Yih
  • Chenyan Xiong

The efficiency and quality of language model pretraining are largely determined by the way pretraining data are selected. In this paper, we introduce Group-MATES, an efficient group-level data selection approach to optimize the speed-quality frontier of language model pretraining. Specifically, Group-MATES parameterizes costly group-level selection with a relational data influence model. To train this model, we sample training trajectories of the language model and collect oracle data influences alongside. The relational data influence model approximates the oracle data influence by weighting individual influence with relationships among training data. To enable efficient selection with our relational data influence model, we partition the dataset into small clusters using relationship weights and select data within each cluster independently. Experiments on DCLM 400M-4x, 1B-1x, and 3B-1x show that Group-MATES achieves 3. 5\%-9. 4\% relative performance gains over random selection across 22 downstream tasks, nearly doubling the improvements achieved by state-of-the-art individual data selection baselines. Furthermore, Group-MATES reduces the number of tokens required to reach a certain downstream performance by up to 1. 75x, substantially elevating the speed-quality frontier. Further analyses highlight the critical role of relationship weights in the relational data influence model and the effectiveness of our cluster-based inference. Our code is open-sourced at https: //github. com/facebookresearch/Group-MATES.

PDF Details

IJCAI Conference 2016 Conference Paper

Scalable Segment Abstraction Method for Advertising Campaign Admission and Inventory Allocation Optimization

  • Fei Peng
  • Tuomas Sandholm

As publishers gather more information about their users, they can use that information to enable advertisers to create increasingly targeted campaigns. This enables better usage of advertising inventory. However, it also dramatically increases the complexity that the publisher faces when optimizing campaign admission decisions and inventory allocation to campaigns. We develop an optimal anytime algorithm for abstracting fine-grained audience segments into coarser abstract segments that are not too numerous for use in such optimization. Compared to the segment abstraction algorithm by Walsh et al. [2010] for the same problem, it yields two orders of magnitude improvement in run time and significant improvement in abstraction quality. These benefits hold both for guaranteed and non-guaranteed campaigns. The performance stems from three improvements: 1) a quadratic-time (as opposed to doubly exponential or heuristic) algorithm for finding an optimal split of an abstract segment, 2) a better scoring function for evaluating splits, and 3) splitting time lossily like any other targeting attribute (instead of losslessly segmenting time first).

PDF Details