Arrow Research search

Author name cluster

Mohammad Rostami

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

22 papers
1 author row

Possible papers

22

JAIR Journal 2025 Journal Article

CluMo: Cluster-based Modality Fusion Prompt for Continual Learning in Visual Question Answering

  • Yuliang Cai
  • Mohammad Rostami

Large vision-language models (VLMs) have shown significant performance boost in various application domains. However, adopting them to deal with several sequentially encountered tasks has been challenging because finetuning a VLM on a task normally leads to reducing its generalization power and the capacity of learning new tasks as well as causing catastrophic forgetting on previously learned tasks. Enabling using VLMs in multimodal continual learning (CL) settings can help to address such scenarios. To improve generalization capacity and prevent catastrophic forgetting, we propose a novel prompt-based CL method for VLMs, namely Cluster-based Modality Fusion Prompt (CluMo). We design a novel Key-Key-Prompt pair, where each prompt is associated with a visual prompt key and a textual prompt key. We adopt a two-stage training strategy. During the first stage, the single-modal keys are trained via K-means clustering algorithm to help select the best semantically matched prompt. During the second stage, the prompt keys are frozen, the selected prompt is attached to the input for training the VLM in the CL scenario. Experiments on two benchmarks demonstrate that our method achieves SOTA performance. The code is publicly available here.

PDF Details DOI

TMLR Journal 2025 Journal Article

Curvature Diversity-Driven Deformation and Domain Alignment for Point Cloud

  • Mengxi Wu
  • Hao Huang
  • Yi Fang
  • Mohammad Rostami

Unsupervised Domain Adaptation is crucial for point cloud learning due to geometric variations across different generation methods and sensors. To tackle this challenge, we propose Curvature Diversity-Driven Nuclear-Norm Wasserstein Domain Alignment (CDND). We first introduce a Curvature Diversity-driven Deformation Reconstruction (CurvRec) task, enabling the model to extract salient features from semantically rich regions of a given point cloud. We then propose a theoretical framework for Deformation-based Nuclear-norm Wasserstein Discrepancy (D-NWD), extending the Nuclear-norm Wasserstein Discrepancy to original and deformed samples. Our theoretical analysis demonstrates that D-NWD is effective for any deformation method. Empirical experiment results show that our CDND achieves state-of-the-art performance by a noticeable margin over existing approaches.

PDF Details

JBHI Journal 2025 Journal Article

Multi-Source Data Integration for Segmentation of Unannotated MRI Images

  • Navapat Nananukul
  • Hamid Soltanian-Zadeh
  • Mohammad Rostami

Automatic semantic segmentation of magnetic resonance imaging (MRI) images using deep neural networks greatly assists in evaluating and planning treatments for various clinical applications. However, training these models is conditioned on the availability of abundant annotated data. Even if we annotate enough data, MRI images display considerable variability due to factors such as differences among patients, MRI scanners, and imaging protocols. This variability necessitates retraining neural networks for each specific application domain, which, in turn requires manual annotation by expert radiologists for all new domains. To relax the need for persistent data annotation, we develop a method for unsupervised federated domain adaptation using multiple annotated source domains. Our approach enables the transfer of knowledge from several annotated source domains for use in an unannotated target domain. Initially, we ensure that the target domain data shares similar representations with each source domain in a latent embedding space by minimizing the pair-wise distances between the distributions for the target and the source domains. We then employ an ensemble approach to leverage the knowledge obtained from all domains to build an integrated outcome. We perform experiments on two datasets to demonstrate our method is effective.

Details DOI

TMLR Journal 2025 Journal Article

Theoretical Insights into Overparameterized Models in Multi-Task and Replay-Based Continual Learning

  • Mohammadamin Banayeeanzade
  • Mahdi Soltanolkotabi
  • Mohammad Rostami

Multi-task learning (MTL) is a machine learning paradigm that aims to improve the generalization performance of a model on multiple related tasks by training it simultaneously on those tasks. Unlike MTL, where the model has instant access to the training data of all tasks, continual learning (CL) involves adapting to new sequentially arriving tasks over time without forgetting the previously acquired knowledge. Despite the wide practical adoption of CL and MTL and extensive literature on both areas, there remains a gap in the theoretical understanding of these methods when used with overparameterized models such as deep neural networks. This paper studies the overparameterized linear models as a proxy for more complex models. We develop theoretical results describing the effect of various system parameters on the model's performance in an MTL setup. Specifically, we study the impact of model size, dataset size, and task similarity on the generalization error and knowledge transfer. Additionally, we present theoretical results to characterize the performance of replay-based CL models. Our results reveal the impact of buffer size and model capacity on the forgetting rate in a CL setup and help shed light on some of the state-of-the-art CL methods. Finally, through extensive empirical evaluations, we demonstrate that our theoretical findings are also applicable to deep neural networks, offering valuable guidance for designing MTL and CL models in practice.

PDF Details

TMLR Journal 2024 Journal Article

Graph Harmony: Denoising and Nuclear-Norm Wasserstein Adaptation for Enhanced Domain Transfer in Graph-Structured Data

  • Mengxi Wu
  • Mohammad Rostami

Graph-structured data is prevalent in numerous fields, but the scarcity of labeled instances often limits the effective application of deep learning techniques. Traditional unsupervised domain adaptation (UDA) strategies for graphs typically rely on adversarial learning and pseudo-labeling. However, these methods often fail to leverage the discriminative features of graphs, resulting in class mismatches and unreliable label quality. To overcome these challenges, we developed the Denoising and Nuclear-Norm Wasserstein Adaptation Network (DNAN). DNAN utilizes the Nuclear-Norm Wasserstein Discrepancy (NWD), which simultaneously achieves domain alignment and class distinction. The NWD is integrated with a denoising mechanism using a variational graph autoencoder, with a theoretical analysis provided for the denoising process. This denoising mechanism aims to address domain shifts in structural patterns between the source and target domains. Our comprehensive experiments demonstrate that DNAN outperforms state-of-the-art methods on standard UDA benchmarks for graph classification, highlighting its effectiveness and robustness.

PDF Details

JAIR Journal 2024 Journal Article

Preserving Fairness in AI under Domain Shift

  • Serban Stan
  • Mohammad Rostami

Existing algorithms for ensuring fairness in AI use a single-shot training strategy, where an AI model is trained on an annotated training dataset with sensitive attributes and then fielded for utilization. This training strategy is effective in problems with stationary distributions, where both the training and testing data are drawn from the same distribution. However, it is vulnerable with respect to distributional shifts in the input space that may occur after the initial training phase. As a result, the time-dependent nature of data can introduce biases and performance degradation into the model predictions, even if the model is initially fair. Model retraining from scratch using a new annotated dataset is a naive solution that is expensive and time-consuming. We develop an algorithm to adapt a fair model to remain fair and generalizable under domain shift using solely new unannotated data points. We recast this learning setting as an unsupervised domain adaptation (UDA) problem. Our algorithm is based on updating the model such that the internal representation of data remains unbiased despite distributional shifts in the input space. We provide empirical validation on three common fairness datasets to show that the challenge exists in practical setting and to demonstrate the effectiveness of our algorithm.

PDF Details DOI

IJCAI Conference 2023 Conference Paper

Cognitively Inspired Learning of Incremental Drifting Concepts

  • Mohammad Rostami
  • Aram Galstyan

Humans continually expand their learned knowledge to new domains and learn new concepts without any interference with past learned experiences. In contrast, machine learning models perform poorly in a continual learning setting, where input data distribution changes over time. Inspired by the nervous system learning mechanisms, we develop a computational model that enables a deep neural network to learn new concepts and expand its learned knowledge to new domains incrementally in a continual learning setting. We rely on the Parallel Distributed Processing theory to encode abstract concepts in an embedding space in terms of a multimodal distribution. This embedding space is modeled by internal data representations in a hidden network layer. We also leverage the Complementary Learning Systems theory to equip the model with a memory mechanism to overcome catastrophic forgetting through implementing pseudo-rehearsal. Our model can generate pseudo-data points for experience replay and accumulate new experiences to past learned experiences without causing cross-task interference.

PDF Details DOI

AAAI Conference 2023 Conference Paper

Overcoming Concept Shift in Domain-Aware Settings through Consolidated Internal Distributions

  • Mohammad Rostami
  • Aram Galstyan

We develop an algorithm to improve the predictive performance of a pre-trained model under \textit{concept shift} without retraining the model from scratch when only unannotated samples of initial concepts are accessible. We model this problem as a domain adaptation problem, where the source domain data is inaccessible during model adaptation. The core idea is based on consolidating the intermediate internal distribution, learned to represent the source domain data, after adapting the model. We provide theoretical analysis and conduct extensive experiments on five benchmark datasets to demonstrate that the proposed method is effective.

PDF Details DOI

NeurIPS Conference 2022 Conference Paper

CLiMB: A Continual Learning Benchmark for Vision-and-Language Tasks

  • Tejas Srinivasan
  • Ting-Yun Chang
  • Leticia Pinto Alva
  • Georgios Chochlakis
  • Mohammad Rostami
  • Jesse Thomason

Current state-of-the-art vision-and-language models are evaluated on tasks either individually or in a multi-task setting, overlooking the challenges of continually learning (CL) tasks as they arrive. Existing CL benchmarks have facilitated research on task adaptation and mitigating "catastrophic forgetting", but are limited to vision-only and language-only tasks. We present CLiMB, a benchmark to study the challenge of learning multimodal tasks in a CL setting, and to systematically evaluate how upstream continual learning can rapidly generalize to new multimodal and unimodal tasks. CLiMB includes implementations of several CL algorithms and a modified Vision-Language Transformer (ViLT) model that can be deployed on both multimodal and unimodal tasks. We find that common CL methods can help mitigate forgetting during multimodal task learning, but do not enable cross-task knowledge transfer. We envision that CLiMB will facilitate research on a new class of CL algorithms for this challenging multimodal setting.

PDF Details

TMLR Journal 2022 Journal Article

Secure Domain Adaptation with Multiple Sources

  • Serban Stan
  • Mohammad Rostami

Multi-source unsupervised domain adaptation (MUDA) is a framework to address the challenge of annotated data scarcity in a target domain via transferring knowledge from multiple annotated source domains. When the source domains are distributed, data privacy and security can become significant concerns and protocols may limit data sharing, yet existing MUDA methods overlook these constraints. We develop an algorithm to address MUDA when source domain data cannot be shared with the target or across the source domains. Our method is based on aligning the distributions of source and target domains indirectly via estimating the source feature embeddings and predicting over a confidence based combination of domain specific model predictions. We provide theoretical analysis to support our approach and conduct empirical experiments to demonstrate that our algorithm is effective.

PDF Details

NeurIPS Conference 2021 Conference Paper

Lifelong Domain Adaptation via Consolidated Internal Distribution

  • Mohammad Rostami

We develop an algorithm to address unsupervised domain adaptation (UDA) in continual learning (CL) settings. The goal is to update a model continually to learn distributional shifts across sequentially arriving tasks with unlabeled data while retaining the knowledge about the past learned tasks. Existing UDA algorithms address the challenge of domain shift, but they require simultaneous access to the datasets of the source and the target domains. On the other hand, existing works on CL can handle tasks with labeled data. Our solution is based on consolidating the learned internal distribution for improved model generalization on new domains and benefitting from experience replay to overcome catastrophic forgetting.

PDF Details

AAAI Conference 2021 Conference Paper

Unsupervised Model Adaptation for Continual Semantic Segmentation

  • Serban Stan
  • Mohammad Rostami

We develop an algorithm for adapting a semantic segmentation model that is trained using a labeled source domain to generalize well in an unlabeled target domain. A similar problem has been studied extensively in the unsupervised domain adaptation (UDA) literature, but existing UDA algorithms require access to both the source domain labeled data and the target domain unlabeled data for training a domain agnostic semantic segmentation model. Relaxing this constraint enables a user to adapt pretrained models to generalize in a target domain, without requiring access to source data. To this end, we learn a prototypical distribution for the source domain in an intermediate embedding space. This distribution encodes the abstract knowledge that is learned from the source domain. We then use this distribution for aligning the target domain distribution with the source domain distribution in the embedding space. We provide theoretical analysis and explain conditions under which our algorithm is effective. Experiments on benchmark adaptation tasks demonstrate our method achieves competitive performance even compared with joint UDA approaches.

PDF Details

AAAI Conference 2020 Conference Paper

Generative Continual Concept Learning

  • Mohammad Rostami
  • Soheil Kolouri
  • Praveen Pilly
  • James McClelland

After learning a concept, humans are also able to continually generalize their learned concepts to new domains by observing only a few labeled instances without any interference with the past learned knowledge. In contrast, learning concepts efficiently in a continual learning setting remains an open challenge for current Artificial Intelligence algorithms as persistent model retraining is necessary. Inspired by the Parallel Distributed Processing learning and the Complementary Learning Systems theories, we develop a computational model that is able to expand its previously learned concepts efficiently to new domains using a few labeled samples. We couple the new form of a concept to its past learned forms in an embedding space for effective continual learning. Doing so, a generative distribution is learned such that it is shared across the tasks in the embedding space and models the abstract concepts. This procedure enables the model to generate pseudo-data points to replay the past experience to tackle catastrophic forgetting.

PDF Details

JAIR Journal 2020 Journal Article

Using Task Descriptions in Lifelong Machine Learning for Improved Performance and Zero-Shot Transfer

  • Mohammad Rostami
  • David Isele
  • Eric Eaton

Knowledge transfer between tasks can improve the performance of learned models, but requires an accurate estimate of inter-task relationships to identify the relevant knowledge to transfer. These inter-task relationships are typically estimated based on training data for each task, which is inefficient in lifelong learning settings where the goal is to learn each consecutive task rapidly from as little data as possible. To reduce this burden, we develop a lifelong learning method based on coupled dictionary learning that utilizes high-level task descriptions to model inter-task relationships. We show that using task descriptors improves the performance of the learned task policies, providing both theoretical justification for the benefit and empirical demonstration of the improvement across a variety of learning problems. Given only the descriptor for a new task, the lifelong learner is also able to accurately predict a model for the new task through zero-shot learning using the coupled dictionary, eliminating the need to gather training data before addressing the task.

PDF Details DOI

IJCAI Conference 2019 Conference Paper

Complementary Learning for Overcoming Catastrophic Forgetting Using Experience Replay

  • Mohammad Rostami
  • Soheil Kolouri
  • Praveen K. Pilly

Despite huge success, deep networks are unable to learn effectively in sequential multitask learning settings as they forget the past learned tasks after learning new tasks. Inspired from complementary learning systems theory, we address this challenge by learning a generative model that couples the current task to the past learned tasks through a discriminative embedding space. We learn an abstract generative distribution in the embedding that allows generation of data points to represent past experience. We sample from this distribution and utilize experience replay to avoid forgetting and simultaneously accumulate new knowledge to the abstract distribution in order to couple the current task with past experience. We demonstrate theoretically and empirically that our framework learns a distribution in the embedding, which is shared across all tasks, and as a result tackles catastrophic forgetting.

PDF Details

AAAI Conference 2018 Conference Paper

Joint Dictionaries for Zero-Shot Learning

  • Soheil Kolouri
  • Mohammad Rostami
  • Yuri Owechko
  • Kyungnam Kim

A classic approach toward zero-shot learning (ZSL) is to map the input domain to a set of semantically meaningful attributes that could be used later on to classify unseen classes of data (e. g. visual data). In this paper, we propose to learn a visual feature dictionary that has semantically meaningful atoms. Such a dictionary is learned via joint dictionary learning for the visual domain and the attribute domain, while enforcing the same sparse coding for both dictionaries. Our novel attribute aware formulation provides an algorithmic solution to the domain shift/hubness problem in ZSL. Upon learning the joint dictionaries, images from unseen classes can be mapped into the attribute space by finding the attribute aware joint sparse representation using solely the visual data. We demonstrate that our approach provides superior or comparable performance to that of the state of the art on benchmark datasets.

PDF Details

AAAI Conference 2018 Short Paper

Lifelong Learning Networks: Beyond Single Agent Lifelong Learning

  • Mohammad Rostami
  • Eric Eaton

Lifelong machine learning (LML) is a paradigm to design adaptive agents that can learn in dynamic environments. Current LML algorithms consider a single agent that has centralized access to all data. However, given privacy and security constraints, data might be distributed among multiple agents that can collaborate and learn from collective experience. Our goal is to extend LML from a single agent to a network of multiple agents that collectively learn a series of tasks.

PDF Details

AAMAS Conference 2018 Conference Paper

Multi-Agent Distributed Lifelong Learning for Collective Knowledge Acquisition

  • Mohammad Rostami
  • Soheil Kolouri
  • Kyungnam Kim
  • Eric Eaton

Lifelong machine learning methods acquire knowledge over a series of consecutive tasks, continually building upon their experience. Current lifelong learning algorithms rely upon a single learning agent that has centralized access to all data. In this paper, we extend the idea of lifelong learning from a single agent to a network of multiple agents that collectively learn a series of tasks. Each agent faces some (potentially unique) set of tasks; the key idea is that knowledge learned from these tasks may benefit other agents trying to learn different (but related) tasks. Our Collective Lifelong Learning Algorithm (CoLLA) provides an efficient way for a network of agents to share their learned knowledge in a distributed and decentralized manner, while eliminating the need to share locally observed data. We provide theoretical guarantees for robust performance of the algorithm and empirically demonstrate that CoLLA outperforms existing approaches for distributed multi-task learning on a variety of datasets.

PDF

EAAI Journal 2017 Journal Article

The two stage assembly flow-shop scheduling problem with batching and delivery

  • Hamed Kazemi
  • Mohammad Mahdavi Mazdeh
  • Mohammad Rostami

This paper investigates the two-stage assembly flow shop scheduling problem with a batched delivery system where there are m independent machines at the first stage doing the components of a job and multiple identical assembly machines at the second stage, each of which can assemble the components and complete the job. The objective is to schedule the jobs, to form them into batches so as to minimize the sum of tardiness plus delivery costs. To the best of our knowledge, the assembly flow shop scheduling problem with this objective function has not been addressed so far. A mathematical model for this problem is presented. However, due to the fact that this model happens to be a mixed integer nonlinear programming model and cannot guarantee to reach the solution at reasonable time we developed the imperialist competitive algorithm (ICA) and a hybrid algorithm (HICA) by incorporating the dominance relations. Computational results show that HICA performs better than ICA with respect to the value of the objective function, However the runtime of the ICA is less than HICA.

Details DOI

AAAI Conference 2017 Short Paper

Transfer of Knowledge through Collective Learning

  • Mohammad Rostami

Learning fast and efficiently using minimal data has been consistently a challenge in machine learning. In my thesis, I explore this problem for knowledge transfer for multiagent multi-task learning in a life-long learning paradigm. My goal is to demonstrate that by sharing knowledge between agents and similar tasks, efficient algorithms can be designed that can increase the speed of learning as well as improve performance. Moreover, this would allow for handling hard tasks through collective learning of multiple agents that share knowledge. As an initial step, I study the problem of incorporating task descriptors into lifelong learning of related tasks to perform zero-shot knowledge transfer. Zero-shot learning is highly desirable because it leads to considerable speedup in handling similar sequential tasks. Then I focus on a multiagent learning setting, where related tasks are learned collectively and/or address privacy concerns.

PDF Details

IJCAI Conference 2016 Conference Paper

Using Task Features for Zero-Shot Knowledge Transfer in Lifelong Learning

  • David Isele
  • Mohammad Rostami
  • Eric Eaton

Knowledge transfer between tasks can improve the performance of learned models, but requires an accurate estimate of the inter-task relationships to identify the relevant knowledge to transfer. These inter-task relationships are typically estimated based on training data for each task, which is inefficient in lifelong learning settings where the goal is to learn each consecutive task rapidly from as little data as possible. To reduce this burden, we develop a lifelong reinforcement learning method based on coupled dictionary learning that incorporates high-level task descriptors to model the inter-task relationships. We show that using task descriptors improves the performance of the learned task policies, providing both theoretical justification for the benefit and empirical demonstration of the improvement across a variety of dynamical control problems. Given only the descriptor for a new task, the lifelong learner is also able to accurately predict the task policy through zero-shot learning using the coupled dictionary, eliminating the need to pause to gather training data before addressing the task.

PDF Details
v2026.09.13