Author name cluster

Mahsa Shoaran

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

5 papers

2 author rows

NeurIPS Conference 2025 Conference Paper

Linear Attention for Efficient Bidirectional Sequence Modeling

Arshia Afzal
Elias Abad Rocamora
Leyla Candogan
Pol Puigdemont
Francesco Tonin
Yongtao Wu
Mahsa Shoaran
Volkan Cevher

Linear Transformers and State Space Models have emerged as efficient alternatives to softmax Transformers for causal sequence modeling, enabling parallel training via matrix multiplication and efficient RNN-style inference. However, despite their success in causal tasks, no unified framework exists for applying Linear Transformers to bidirectional sequence modeling. We introduce LION, the first framework to systematically extend Linear Transformers to the bidirectional setting. LION generalizes three core representations commonly used in the causal case—full Linear Attention, bidirectional RNN, and chunkwise parallel form—to the bidirectional setting. These forms are theoretically equivalent and enable models to exploit the strengths of each during training and inference. We prove that a broad class of Linear Transformers can be extended using LION and validate our framework via three core examples based on the choice of decay type: LION-LIT, the bidirectional extension of [25]; LION-D, based on [44]; and LION-S, a variant using selective decay [34, 13]. Across standard bidirectional tasks, LION enables models to match or exceed the performance of softmax Transformers, while offering significantly faster training and more efficient inference than existing State Space Models.

PDF Details

ICML Conference 2024 Conference Paper

REST: Efficient and Accelerated EEG Seizure Analysis through Residual State Updates

Arshia Afzal
Grigorios Chrysos 0002
Volkan Cevher
Mahsa Shoaran

EEG-based seizure detection models face challenges in terms of inference speed and memory efficiency, limiting their real-time implementation in clinical devices. This paper introduces a novel graph-based residual state update mechanism (REST) for real-time EEG signal analysis in applications such as epileptic seizure detection. By leveraging a combination of graph neural networks and recurrent structures, REST efficiently captures both non-Euclidean geometry and temporal dependencies within EEG data. Our model demonstrates high accuracy in both seizure detection and classification tasks. Notably, REST achieves a remarkable 9-fold acceleration in inference speed compared to state-of-the-art models, while simultaneously demanding substantially less memory than the smallest model employed for this task. These attributes position REST as a promising candidate for real-time implementation in clinical devices, such as Responsive Neurostimulation or seizure alert systems.

Details

ICML Conference 2023 Conference Paper

XTab: Cross-table Pretraining for Tabular Transformers

Bingzhao Zhu
Xingjian Shi
Nick Erickson
Mu Li 0003
George Karypis
Mahsa Shoaran

The success of self-supervised learning in computer vision and natural language processing has motivated pretraining methods on tabular data. However, most existing tabular self-supervised learning models fail to leverage information across multiple data tables and cannot generalize to new tables. In this work, we introduce XTab, a framework for cross-table pretraining of tabular transformers on datasets from various domains. We address the challenge of inconsistent column types and quantities among tables by utilizing independent featurizers and using federated learning to pretrain the shared component. Tested on 84 tabular prediction tasks from the OpenML-AutoML Benchmark (AMLB), we show that (1) XTab consistently boosts the generalizability, learning speed, and performance of multiple tabular transformers, (2) by pretraining FT-Transformer via XTab, we achieve superior performance than other state-of-the-art tabular deep learning models on various tasks such as regression, binary, and multiclass classification.

Details

NeurIPS Conference 2021 Conference Paper

Tree in Tree: from Decision Trees to Decision Graphs

Bingzhao Zhu
Mahsa Shoaran

Decision trees have been widely used as classifiers in many machine learning applications thanks to their lightweight and interpretable decision process. This paper introduces Tree in Tree decision graph (TnT), a framework that extends the conventional decision tree to a more generic and powerful directed acyclic graph. TnT constructs decision graphs by recursively growing decision trees inside the internal or leaf nodes instead of greedy training. The time complexity of TnT is linear to the number of nodes in the graph, therefore it can construct decision graphs on large datasets. Compared to decision trees, we show that TnT achieves better classification performance with reduced model size, both as a stand-alone classifier and as a base-estimator in bagging/AdaBoost ensembles. Our proposed model is a novel, more efficient and accurate alternative to the widely-used decision trees.

PDF Details

YNIMG Journal 2019 Journal Article

Neural variability quenching during decision-making: Neural individuality and its prestimulus complexity

Annemarie Wolff
Lin Yao
Javier Gomez-Pilar
Mahsa Shoaran
Ning Jiang
Georg Northoff

The spontaneous activity of the brain interacts with stimulus-induced activity which is manifested in event-related amplitude and its trial-to-trial variability (TTV). TTV describes the variability in the amplitude of the stimulus-evoked response across trials, and it is generally observed to be reduced, or quenched. While such TTV quenching has been observed on both the cellular and regional levels, its exact behavioral relevance and neuronal basis remains unclear. Applying a novel paradigm for testing neural markers of individuality in internally-guided decision-making, we here investigated whether TTV (i) represents an individually specific response by comparing individualized vs shared stimuli; and (ii) is mediated by the complexity of prestimulus activity as measured by the Lempel-Ziv Complexity index (LZC). We observed that TTV - and other electrophysiological markers such as ERP, ERSP, and ITC – showed first significant differences between individualized and shared stimuli (while controlling for task-related effects) specifically in the alpha and beta frequency bands, and secondly that TTV in the beta band correlated significantly with reaction time and eLORETA activity. Moreover, we demonstrate that the complexity (LZC) of neuronal activity is higher in the prestimulus period while it decreases during the poststimulus period, with the former also correlating specifically with poststimulus individualized TTV in alpha (but not with shared TTV). Together, our results show that the TTV represents a marker of ‘neural individualization’ which, being related to internal processes on both neural and psychological levels, is mediated by the information complexity of prestimulus activity. More generally, our results inform the pre-post-stimulus dynamics of rest-stimulus interaction, which is a basic and ubiquitous neural phenomenon in the brain and highly relevant for mental features including their individuality.

Details DOI