Geometry-Aware Collaborative Multi-Solutions Optimizer for Model Fine-Tuning with Parameter Efficiency

We propose a framework grounded in gradient flow theory and informed by geometric structure that provides multiple diverse solutions for a given task, ensuring collaborative results that enhance performance and adaptability across different tasks. This framework enables flexibility, allowing for efficient task-specific fine-tuning while preserving the knowledge of the pre-trained foundation models. Extensive experiments across transfer learning, few-shot learning, and domain generalization show that our proposed approach consistently outperforms existing Bayesian methods, delivering strong performance with affordable computational overhead and offering a practical solution by updating only a small subset of parameters.

Authors

• Van-Anh Nguyen
• Trung Le
• Mehrtash Harandi Monash University
• Ehsan Abbasnejad University of Adelaide
• Thanh-Toan Do Monash University, Australia
• Dinh Phung

Keywords

No keywords are indexed for this paper.