TMLR Journal 2025 Journal Article
Preferential Multi-Objective Bayesian Optimization
- Raul Astudillo
- Kejun Li
- Maegan Tucker
- Chu Xin Cheng
- Aaron Ames
- Yisong Yue
Preferential Bayesian optimization (PBO) is a framework for optimizing a decision-maker’s latent preferences over available design choices. While real-world problems often involve multiple conflicting objectives, existing PBO methods assume that preferences can be encoded by a single objective function. For instance, in the customization of robotic assistive devices, technicians aim to maximize user comfort while minimizing energy consumption to extend battery life. Likewise, in autonomous driving policy design, stakeholders must evaluate safety and performance trade-offs before committing to a policy. To bridge this gap, we introduce the first framework for PBO with multiple objectives. Within this framework, we propose dueling scalarized Thompson sampling (DSTS), a multi-objective generalization of the popular dueling Thompson sampling algorithm, which may also be of independent interest beyond our setting. We evaluate DSTS across four synthetic test functions and two simulated tasks—exoskeleton personalization and driving policy design—demonstrating that it outperforms several benchmarks. Finally, we prove that DSTS is asymptotically consistent. Along the way, we provide, to our knowledge, the first convergence guarantee for dueling Thompson sampling in single-objective PBO.