Author name cluster

Mengwei Xu

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

5 papers

1 author row

NeurIPS Conference 2025 Conference Paper

LoRASuite: Efficient LoRA Adaptation Across Large Language Model Upgrades

Yanan Li
Fanxu Meng
Muhan Zhang
Shiai Zhu
Shangguang Wang
Mengwei Xu

As Large Language Models (LLMs) are frequently updated, LoRA weights trained on earlier versions quickly become obsolete. The conventional practice of retraining LoRA weights from scratch on the latest model is costly, time-consuming, and environmentally detrimental, particularly as the diversity of LLMs and downstream tasks expands. This motivates a critical question: "How can we efficiently leverage existing LoRA weights to adapt to newer model versions? " To address this, we propose LoRASuite, a modular approach tailored specifically to various types of LLM updates. First, we compute a transfer matrix utilizing known parameters from both old and new LLMs. Next, we allocate corresponding layers and attention heads based on centered kernel alignment and cosine similarity metrics, respectively. A subsequent small-scale, skillful fine-tuning step ensures numerical stability. Experimental evaluations demonstrate that LoRASuite consistently surpasses small-scale vanilla LoRA methods. Notably, on backbone LLMs such as MiniCPM and Qwen, LoRASuite even exceeds the performance of full-scale LoRA retraining, with average improvements of +1. 4 and +6. 6 points on math tasks, respectively. Additionally, LoRASuite significantly reduces memory consumption by 5. 5 GB and computational time by 78. 23%.

PDF Details

AAMAS Conference 2025 Conference Paper

Quantitative Operational Monitoring for BDI Agents

Marie Farrell
Angelo Ferrando
Mengwei Xu

Belief-Desire-Intention (BDI) architecture is a popular framework for designing autonomous systems. As these systems make independent decisions and execute actions independent from humans, ensuring their safety and reliability becomes a major concern. Traditional verification methods often fail to give run-time operational insights into an agent’s behaviours, especially with quantitative assessments under uncertain conditions, such as imprecise actuating. Meanwhile, BDI agents, which rely on context-sensitive subtask expansion, act as they go e. g. selecting plans at run time. To address this, we have developed a monitoring method that combines realtime operational data with probabilistic verification. This approach allows us to quantitatively analyse the decisions of BDI agents as they occur to understand the impact of each decision as it happens.

PDF

AAMAS Conference 2025 Conference Paper

Uncertain Machine Ethics Planning

Simon Kolker
Louise A. Dennis
Ramon Fraga Pereira
Mengwei Xu

Machine Ethics decisions should consider the implications of uncertainty over decisions. Decisions should be made over sequences of actions to reach preferable outcomes long term. The evaluation of outcomes, however, may invoke one or more moral theories, which might have conflicting judgements. Each theory will require differing representations of the ethical situation. For example, Utilitarianism measures numerical values, Deontology analyses duties, and Virtue Ethics emphasises moral character. While balancing potentially conflicting moral considerations, decisions may need to be made, for example, to achieve morally neutral goals with minimal costs. In this paper, we formalise the problem as a Multi-Moral Markov Decision Process and a Multi-Moral Stochastic Shortest Path Problem. We develop a heuristic algorithm based on Multi-Objective AO*, utilising Sven-Ove Hansson’s Hypothetical Retrospection procedure for ethical reasoning under uncertainty. Our approach is validated by a case study from Machine Ethics literature: the problem of whether to steal insulin for someone who needs it.

PDF

AAMAS Conference 2024 Conference Paper

Safeguard Privacy for Minimal Data Collection with Trustworthy Autonomous Agents

Mengwei Xu
Louise A. Dennis
Mustafa A. Mustafa

Ensuring digital privacy necessitates users giving well-considered consent to online service providers for data usage, creating an unsustainable and error-prone decision load. Software privacy agents can help make data consent decisions on behalf of users, but a compromised agent could be more detrimental than the absence of such an agent. In response, we employ trustworthy autonomous agents to safeguard users’ privacy at the stage of data collection. Drawing upon General Data Protection Regulation (GDPR) principles, notably data minimisation, our autonomous agent guarantees that GDPR-reflected privacy requirements are met through strong proof. We provide a computational encoding of a typical data collection scenario—where data are requested and decisions are made about these requests—as a cognitive agent that makes decisions based on how an agent’s beliefs and goals lead to particular choices. Importantly, our approach provides verifiable assurance about decisions made by these cognitive agents through formal verification, supporting both simultaneous (data requested at the same time) and sequential (data requested one after the other) situations. We provide a templated implementation of these privacy agents and a small example of a mobile app serves to illustrate how a privacy agent can be designed in practice. An in-depth evaluation is given to demonstrate its computational practicality in making privacy decisions in real time and its computational complexity in verifying them. This approach represents a promising step towards trustworthy computational stewardship in data management.

PDF

NeurIPS Conference 2024 Conference Paper

SILENCE: Protecting privacy in offloaded speech understanding on resource-constrained devices

Dongqi Cai
Shangguang Wang
Zeling Zhang
Felix Xiaozhu Lin
Mengwei Xu

Speech serves as a ubiquitous input interface for embedded mobile devices. Cloud-based solutions, while offering powerful speech understanding services, raise significant concerns regarding user privacy. To address this, disentanglement-based encoders have been proposed to remove sensitive information from speech signals without compromising the speech understanding functionality. However, these encoders demand high memory usage and computation complexity, making them impractical for resource-constrained wimpy devices. Our solution is based on a key observation that speech understanding hinges on long-term dependency knowledge of the entire utterance, in contrast to privacy-sensitive elements that are short-term dependent. Exploiting this observation, we propose SILENCE, a lightweight system that selectively obscuring short-term details, without damaging the long-term dependent speech understanding performance. The crucial part of SILENCE is a differential mask generator derived from interpretable learning to automatically configure the masking process. We have implemented SILENCE on the STM32H7 microcontroller and evaluate its efficacy under different attacking scenarios. Our results demonstrate that SILENCE offers speech understanding performance and privacy protection capacity comparable to existing encoders, while achieving up to 53. 3$\times$ speedup and 134. 1$\times$ reduction in memory footprint.

PDF Details DOI