Author name cluster

Yimeng Zeng

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

5 papers

2 author rows

NeurIPS Conference 2025 Conference Paper

Covering Multiple Objectives with a Small Set of Solutions Using Bayesian Optimization

Natalie Maus
Kyurae Kim
Yimeng Zeng
Haydn Jones
Fangping Wan
Marcelo Der Torossian Torres
Cesar de la Fuente-Nunez
Jacob Gardner

In multi-objective black-box optimization, the goal is typically to find solutions that optimize a set of $T$ black-box objective functions, $f_1, \ldots f_T$, simultaneously. Traditional approaches often seek a single Pareto-optimal set that balances trade-offs among all objectives. In contrast, we consider a problem setting that departs from this paradigm: finding a small set of $K < T$ solutions, that collectively "cover" the $T$ objectives. A set of solutions is defined as "covering" if, for each objective $f_1, \ldots f_T$, there is at least one good solution. A motivating example for this problem setting occurs in drug design. For example, we may have $T$ pathogens and aim to identify a set of $K < T$ antibiotics such that at least one antibiotic can be used to treat each pathogen. This problem, known as coverage optimization, has yet to be tackled with the Bayesian optimization (BO) framework. To fill this void, we develop Multi-Objective Coverage Bayesian Optimization (MOCOBO), a BO algorithm for solving coverage optimization. Our approach is based on a new acquisition function reminiscent of expected improvement in the vanilla BO setup. We demonstrate the performance of our method on high-dimensional black-box optimization tasks, including applications in peptide and molecular design. Results show that the coverage of the $K < T$ solutions found by MOCOBO matches or nearly matches the coverage of $T$ solutions obtained by optimizing each objective individually. Furthermore, in *in vitro* experiments, the peptides found by MOCOBO exhibited high potency against drug-resistant pathogens, further demonstrating the potential of MOCOBO for drug discovery. All of our code is publicly available at the following link: https: //github. com/nataliemaus/mocobo.

PDF Details

ICLR Conference 2025 Conference Paper

Zeroth-Order Fine-Tuning of LLMs with Transferable Static Sparsity

Wentao Guo
Jikai Long
Yimeng Zeng
Zirui Liu 0001
Xinyu Yang 0002
Yide Ran
Jacob R. Gardner
Osbert Bastani

Zeroth-order optimization (ZO) is a memory-efficient strategy for fine-tuning Large Language Models using only forward passes. However, applying ZO fine-tuning in memory-constrained settings such as mobile phones and laptops remains challenging since these settings often involve weight quantization, while ZO requires full-precision perturbation and update. In this study, we address this limitation by combining static sparse ZO fine-tuning with quantization. Our approach transfers a small, static subset (0.1%) of "sensitive" parameters from pre-training to downstream tasks, focusing fine-tuning on this sparse set of parameters. The remaining untuned parameters are quantized, reducing memory demands. Our proposed workflow enables efficient ZO fine-tuning of an Llama2-7B model on a GPU device with less than 8GB of memory while outperforming full model ZO fine-tuning performance and in-context learning.

Details

NeurIPS Conference 2024 Conference Paper

Generative Adversarial Model-Based Optimization via Source Critic Regularization

Michael S. Yao
Yimeng Zeng
Hamsa Bastani
Jacob Gardner
James C. Gee
Osbert Bastani

Offline model-based optimization seeks to optimize against a learned surrogate model without querying the true oracle objective function during optimization. Such tasks are commonly encountered in protein design, robotics, and clinical medicine where evaluating the oracle function is prohibitively expensive. However, inaccurate surrogate model predictions are frequently encountered along offline optimization trajectories. To address this limitation, we propose generative adversarial model-based optimization using adaptive source critic regularization (aSCR) —a task- and optimizer- agnostic framework for constraining the optimization trajectory to regions of the design space where the surrogate function is reliable. We propose a computationally tractable algorithm to dynamically adjust the strength of this constraint, and show how leveraging aSCR with standard Bayesian optimization outperforms existing methods on a suite of offline generative design tasks. Our code is available at https: //github. com/michael-s-yao/gabo.

PDF Details DOI

ICLR Conference 2024 Conference Paper

Learning Performance-Improving Code Edits

Alexander Shypula
Aman Madaan
Yimeng Zeng
Uri Alon 0002
Jacob R. Gardner
Yiming Yang 0002
Milad Hashemi
Graham Neubig

With the decline of Moore's law, optimizing program performance has become a major focus of software research. However, high-level optimizations such as API and algorithm changes remain elusive due to the difficulty of understanding the semantics of code. Simultaneously, pretrained large language models (LLMs) have demonstrated strong capabilities at solving a wide range of programming tasks. To that end, we introduce a framework for adapting LLMs to high-level program optimization. First, we curate a dataset of performance-improving edits made by human programmers of over 77,000 competitive C++ programming submission pairs, accompanied by extensive unit tests. A major challenge is the significant variability of measuring performance on commodity hardware, which can lead to spurious "improvements." To isolate and reliably evaluate the impact of program optimizations, we design an environment based on the gem5 full system simulator, the de facto simulator used in academia and industry. Next, we propose a broad range of adaptation strategies for code optimization; for prompting, these include retrieval-based few-shot prompting and chain-of-thought, and for finetuning, these include performance-conditioned generation and synthetic data augmentation based on self-play. A combination of these techniques achieves a mean speedup of 6.86$\times$ with eight generations, higher than average optimizations from individual programmers (3.66$\times$). Using our model's fastest generations, we set a new upper limit on the fastest speedup possible for our dataset at 9.64$\times$ compared to using the fastest human submissions available (9.56$\times$).

Details

YNIMG Journal 2021 Journal Article

Dynamic integration and segregation of amygdala subregional functional circuits linking to physiological arousal

Yimeng Zeng
Fuxiang Tao
Zaixu Cui
Liyun Wu
Jiahua Xu
Wenshan Dong
Chao Liu
Zhi Yang

The dynamical organization of brain networks is essential to support human cognition and emotion for rapid adaption to ever-changing environment. As the core nodes of emotion-related brain circuitry, the basolateral amygdala (BLA) and centromedial amygdala (CMA) as two major amygdalar nuclei, are recognized to play distinct roles in affective functions and internal states, via their unique connections with cortical and subcortical structures in rodents. However, little is known how the dynamical organization of emotion-related brain circuitry reflects internal autonomic responses in humans. Using resting-state functional magnetic resonance imaging (fMRI) with K-means clustering approach in a total of 79 young healthy individuals (cohort 1: 42; cohort 2: 37), we identified two distinct states of BLA- and CMA-based intrinsic connectivity patterns, with one state (integration) showing generally stronger BLA- and CMA-based intrinsic connectivity with multiple brain networks, while the other (segregation) exhibiting weaker yet dissociable connectivity patterns. In an independent cohort 2 of fMRI data with concurrent recording of skin conductance, we replicated two similar dynamic states and further found higher skin conductance level in the integration than segregation state. Moreover, machine learning-based Elastic-net regression analyses revealed that time-varying BLA and CMA intrinsic connectivity with distinct network configurations yield higher predictive values for spontaneous fluctuations of skin conductance level in the integration than segregation state. Our findings highlight dynamic functional organization of emotion-related amygdala nuclei circuits and networks and its links to spontaneous autonomic arousal in humans.

Details DOI