Author name cluster

Quyen Tran

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

9 papers

2 author rows

AAAI Conference 2026 Conference Paper

MCW-KD: Multi-Cost Wasserstein Knowledge Distillation for Large Language Models

Hoang Tran Vuong
Tue Le
Quyen Tran
Linh Ngo Van
Trung Le

Knowledge distillation (KD) is widely recognized as an effective approach for compressing large language models (LLMs). However, standard KD methods often falter when confronted with architectural or tokenization heterogeneity between teacher and student models, which creates a mismatch in their representations. While Optimal Transport (OT) provides a promising solution to align these representations, most OT-based methods rely on a single cost function, which isn’t enough to capture the multifaceted discrepancies between models with distinct designs. To address this limitation, we introduce Multi-Cost Wasserstein Knowledge Distillation (MCW-KD), a novel framework that enhances KD by simultaneously optimizing several cost functions within a unified OT formulation. MCW-KD employs specific cost matrices to effectively align both the final hidden states and the output distributions of the models. We also provide a rigorous theoretical foundation for the proposed Multi-Cost Wasserstein Distance, ensuring both mathematical validity and computational ability. Extensive experiments on instruction-following datasets demonstrate that MCW-KD significantly improves student model performance compared to state-of-the-art KD baselines, especially when teacher and student models have different tokenizers.

PDF Details DOI

ICLR Conference 2025 Conference Paper

Boosting Multiple Views for pretrained-based Continual Learning

Quyen Tran
Tung Lam Tran
Khanh Doan
Toan Tran 0003
Dinh Q. Phung
Khoat Than
Trung Le 0001

Recent research has shown that Random Projection (RP) can effectively improve the performance of pre-trained models in Continual learning (CL). The authors hypothesized that using RP to map features onto a higher-dimensional space can make them more linearly separable. In this work, we theoretically analyze the role of RP and present its benefits for improving the model’s generalization ability in each task and facilitating CL overall. Additionally, we take this result to the next level by proposing a Multi-View Random Projection scheme for a stronger ensemble classifier. In particular, we train a set of linear experts, among which diversity is encouraged based on the principle of AdaBoost, which was initially very challenging to apply to CL. Moreover, we employ a task-based adaptive backbone with distinct prompts dedicated to each task for better representation learning. To properly select these task-specific components and mitigate potential feature shifts caused by misprediction, we introduce a simple yet effective technique called the self-improvement process. Experimentally, our method consistently outperforms state-of-the-art baselines across a wide range of datasets.

Details

AAAI Conference 2025 Conference Paper

Few-Shot, No Problem: Descriptive Continual Relation Extraction

Nguyen Xuan Thanh
Anh Duc Le
Quyen Tran
Thanh-Thien Le
Linh Ngo Van
Thien Huu Nguyen

Few-shot Continual Relation Extraction is a crucial challenge for enabling AI systems to identify and adapt to evolving relationships in dynamic real-world domains. Traditional memory-based approaches often overfit to limited samples, failing to reinforce old knowledge, with the scarcity of data in few-shot scenarios further exacerbating these issues by hindering effective data augmentation in the latent space. In this paper, we propose a novel retrieval-based solution, starting with a large language model to generate descriptions for each relation. From these descriptions, we introduce a bi-encoder retrieval training paradigm to enrich both sample and class representation learning. Leveraging these enhanced representations, we design a retrieval-based prediction method where each sample "retrieves" the best fitting relation via a reciprocal rank fusion score that integrates both relation description vectors and class prototypes. Extensive experiments on multiple datasets demonstrate that our method significantly advances the state-of-the-art by maintaining robust performance across sequential tasks, effectively addressing catastrophic forgetting.

PDF Details DOI

ICML Conference 2025 Conference Paper

Improving Generalization with Flat Hilbert Bayesian Inference

Tuan Truong
Quyen Tran
Ngoc-Quan Pham
Nhat Ho
Dinh Q. Phung
Trung Le 0001

We introduce Flat Hilbert Bayesian Inference (FHBI), an algorithm designed to enhance generalization in Bayesian inference. Our approach involves an iterative two-step procedure with an adversarial functional perturbation step and a functional descent step within the reproducing kernel Hilbert spaces. This methodology is supported by a theoretical analysis that extends previous findings on generalization ability from finite-dimensional Euclidean spaces to infinite-dimensional functional spaces. To evaluate the effectiveness of FHBI, we conduct comprehensive comparisons against nine baseline methods on the VTAB-1K benchmark, which encompasses 19 diverse datasets across various domains with diverse semantics. Empirical results demonstrate that FHBI consistently outperforms the baselines by notable margins, highlighting its practical efficacy.

Details

ICML Conference 2025 Conference Paper

Promoting Ensemble Diversity with Interactive Bayesian Distributional Robustness for Fine-tuning Foundation Models

Ngoc-Quan Pham
Tuan Truong
Quyen Tran
Tan Minh Nguyen
Dinh Q. Phung
Trung Le 0001

We introduce Interactive Bayesian Distributional Robustness (IBDR), a novel Bayesian inference framework that allows modeling the interactions between particles, thereby enhancing ensemble quality through increased particle diversity. IBDR is grounded in a generalized theoretical framework that connects the distributional population loss with the approximate posterior, motivating a practical dual optimization procedure that enforces distributional robustness while fostering particle diversity. We evaluate IBDR’s performance against various baseline methods using the VTAB-1K benchmark and the common reasoning language task. The results consistently show that IBDR outperforms these baselines, underscoring its effectiveness in real-world applications.

Details

ICLR Conference 2025 Conference Paper

Revisiting Prefix-tuning: Statistical Benefits of Reparameterization among Prompts

Minh Le
Chau Nguyen
Huy Nguyen
Quyen Tran
Trung Le 0001
Nhat Ho

Prompt-based techniques, such as prompt-tuning and prefix-tuning, have gained prominence for their efficiency in fine-tuning large pre-trained models. Despite their widespread adoption, the theoretical foundations of these methods remain limited. For instance, in prefix-tuning, we observe that a key factor in achieving performance parity with full fine-tuning lies in the reparameterization strategy. However, the theoretical principles underpinning the effectiveness of this approach have yet to be thoroughly examined. Our study demonstrates that reparameterization is not merely an engineering trick but is grounded in deep theoretical foundations. Specifically, we show that the reparameterization strategy implicitly encodes a shared structure between prefix key and value vectors. Building on recent insights into the connection between prefix-tuning and mixture of experts models, we further illustrate that this shared structure significantly improves sample efficiency in parameter estimation compared to non-shared alternatives. The effectiveness of prefix-tuning across diverse tasks is empirically confirmed to be enhanced by the shared structure, through extensive experiments in both visual and language domains. Additionally, we uncover similar structural benefits in prompt-tuning, offering new perspectives on its success. Our findings provide theoretical and empirical contributions, advancing the understanding of prompt-based methods and their underlying mechanisms.

Details

NeurIPS Conference 2025 Conference Paper

Token-Level Self-Play with Importance-Aware Guidance for Large Language Models

Tue Le
Hoang Tran
Quyen Tran
Linh Ngo
Mehrtash Harandi
Trung Le

Leveraging the power of Large Language Models (LLMs) through preference optimization is crucial for aligning model outputs with human values. Direct Preference Optimization (DPO) has recently emerged as a simple yet effective method by directly optimizing on preference data without the need for explicit reward models. However, DPO typically relies on human-labeled preference data, which can limit its scalability. Self-Play Fine-Tuning (SPIN) addresses this by allowing models to generate their own rejected samples, reducing the dependence on human annotations. Nevertheless, SPIN uniformly applies learning signals across all tokens, ignoring the fine-grained quality variations within responses. As the model improves, rejected samples increasingly contain high-quality tokens, making the uniform treatment of tokens suboptimal. In this paper, we propose SWIFT (Self-Play Weighted Fine-Tuning), a fine-grained self-refinement method that assigns token-level importance weights estimated from a stronger teacher model. Beyond alignment, we also demonstrate that SWIFT serves as an effective knowledge distillation strategy by using the teacher not for logits matching, but for reward-guided token weighting. Extensive experiments on diverse benchmarks and settings demonstrate that SWIFT consistently surpasses both existing alignment approaches and conventional knowledge distillation methods.

PDF Details

NeurIPS Conference 2024 Conference Paper

Enhancing Domain Adaptation through Prompt Gradient Alignment

Hoang Phan
Lam Tran
Quyen Tran
Trung Le

Prior Unsupervised Domain Adaptation (UDA) methods often aim to train a domain-invariant feature extractor, which may hinder the model from learning sufficiently discriminative features. To tackle this, a line of works based on prompt learning leverages the power of large-scale pre-trained vision-language models to learn both domain-invariant and specific features through a set of domain-agnostic and domain-specific learnable prompts. Those studies typically enforce invariant constraints on representation, output, or prompt space to learn such prompts. Differently, we cast UDA as a multiple-objective optimization problem in which each objective is represented by a domain loss. Under this new framework, we propose aligning per-objective gradients to foster consensus between them. Additionally, to prevent potential overfitting when fine-tuning this deep learning architecture, we penalize the norm of these gradients. To achieve these goals, we devise a practical gradient update procedure that can work under both single-source and multi-source UDA. Empirically, our method consistently surpasses other vision language model adaptation methods by a large margin on a wide range of benchmarks. The implementation is available at https: //github. com/VietHoang1512/PGA.

PDF Details DOI

YNIMG Journal 2024 Journal Article

Optimization-derived blood input function using a kernel method and its evaluation with total-body PET for brain parametric imaging

Yansong Zhu
Quyen Tran
Yiran Wang
Ramsey D. Badawi
Simon R. Cherry
Jinyi Qi
Shiva Abbaszadeh
Guobao Wang

Dynamic PET allows quantification of physiological parameters through tracer kinetic modeling. For dynamic imaging of brain or head and neck cancer on conventional PET scanners with a short axial field of view, the image-derived input function (ID-IF) from intracranial blood vessels such as the carotid artery (CA) suffers from severe partial volume effects. Alternatively, optimization-derived input function (OD-IF) by the simultaneous estimation (SIME) method does not rely on an ID-IF but derives the input function directly from the data. However, the optimization problem is often highly ill-posed. We proposed a new method that combines the ideas of OD-IF and ID-IF together through a kernel framework. While evaluation of such a method is challenging in human subjects, we used the uEXPLORER total-body PET system that covers major blood pools to provide a reference for validation. METHODS: F-fluorodeoxyglucose studies with both computer simulations and 20 human-subject scans acquired on the uEXPLORER scanner. The effect of the number of ROIs on kernel SIME was also explored. RESULTS: The estimated OD-IF by kernel SIME showed a good match with the reference input function and provided more accurate estimation of kinetic parameters for both simulation and human-subject data. The kernel SIME led to the highest correlation coefficient (R = 0.97) and the lowest mean absolute error (MAE = 10.5 %) compared to using the CA ID-IF (R = 0.86, MAE = 108.2 %) and conventional SIME (R = 0.57, MAE = 78.7 %) in the human-subject evaluation. Adding more ROIs improved the overall performance of the kernel SIME method. CONCLUSION: The proposed kernel SIME method shows promise to provide an accurate estimation of the blood input function and kinetic parameters for brain PET parametric imaging.

Details DOI