Author name cluster

Sriparna Saha

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

15 papers

1 author row

AAAI Conference 2026 Conference Paper

Crossing Borders: A Multimodal Challenge for Indian Poetry Translation and Image Generation

Sofia Jamil
Kotla Sai Charan
Sriparna Saha
Koustava Goswami
Joseph K J

Indian poetry, known for its linguistic complexity and deep cultural resonance, has a rich and varied heritage spanning thousands of years. However, its layered meanings, cultural allusions, and sophisticated grammatical constructions often pose challenges for comprehension, especially for non-native speakers or readers unfamiliar with its context and language. Despite its cultural significance, existing works on poetry have largely overlooked Indian language poems. In this paper, we propose the Translation and Image Generation (TAI) framework, leveraging Large Language Models (LLMs) and Latent Diffusion Models through appropriate prompt tuning. Our framework supports the United Nations Sustainable Development Goals of Quality Education (SDG 4) and Reduced Inequalities (SDG 10), by enhancing the accessibility of culturally rich Indian-language poetry to a global audience. It includes (1) a translation module that uses an Odds Ratio Preference Alignment Algorithm to accurately translate morphologically rich poetry into English; (2) an image generation module that employs a semantic graph to capture tokens, dependencies, and semantic relationships between metaphors and their meanings, to create visually meaningful representations of Indian poems. Our comprehensive experimental evaluation, including both human and quantitative assessments, demonstrates the superiority of TAI Diffusion in poem image generation tasks, outperforming strong baselines. To further address the scarcity of resources for Indian-language poetry, we introduce the Morphologically Rich Indian Language Poems MorphoVerse Dataset, comprising 1,570 poems across 21 low-resource Indian languages. By addressing the gap in poetry translation and visual comprehension, this work aims to broaden accessibility and enrich the reader’s experience.

PDF Details DOI

TMLR Journal 2026 Journal Article

Harnessing Heterogeneity: Improving Convergence Through Partial Variance Control in Federated Learning

Pranab Sahoo
Ashutosh Tripathi
Sriparna Saha
Samrat Mondal

Federated Learning (FL) has emerged as a promising paradigm for collaborative model training without sharing local data. However, a significant challenge in FL arises from the heterogeneous data distributions across participating clients. This heterogeneity leads to highly variable gradient norms in the model's final layers, resulting in poor generalization, slower convergence, and reduced robustness of the global model. To address these issues, we propose a novel technique that incorporates a gradient penalty term into partial variance control. Our method enables diverse representation learning from heterogeneous client data in the initial layers while modifying standard SGD in the final layers. This approach reduces the variance in the classification layers, aligns the gradients, and mitigates the effects of data heterogeneity. Through theoretical analysis, we establish convergence rate bounds for the proposed algorithm, demonstrating its potential for competitive convergence compared to current FL methods in highly heterogeneous data settings. Empirical evaluations on five benchmark datasets validate our approach, showing enhanced performance and faster convergence over state-of-the-art baselines across various levels of data heterogeneity.

PDF Details

AAAI Conference 2026 Conference Paper

Talk, Snap, Complain: Validation-Aware Multimodal Expert Framework for Fine-Grained Customer Grievances

Rishu Kumar Singh
Navneet Shreya
Sarmistha Das
APOORVA SINGH
Sriparna Saha

Existing approaches to complaint analysis largely rely on unimodal, short-form content such as tweets or product reviews. This work advances the field by leveraging multimodal, multi-turn customer support dialogues—where users often share both textual complaints and visual evidence (e.g., screenshots, product photos)—to enable fine-grained classification of complaint aspects and severity. We introduce VALOR, a Validation-Aware Learner with Expert Routing, tailored for this multimodal setting. It employs a multi-expert reasoning setup using large-scale generative models with Chain-of-Thought (CoT) prompting for nuanced decision-making. To ensure coherence between modalities, a semantic alignment score is computed and integrated into the final classification through a meta-fusion strategy. In alignment with the United Nations Sustainable Development Goals (UN SDGs), the proposed framework supports SDG 9 (Industry, Innovation and Infrastructure) by advancing AI-driven tools for robust, scalable, and context-aware service infrastructure. Further, by enabling structured analysis of complaint narratives and visual context, it contributes to SDG 12 (Responsible Consumption and Production) by promoting more responsive product design and improved accountability in consumer services. We evaluate VALOR on a curated multimodal complaint dataset annotated with fine-grained aspect and severity labels, showing that it consistently outperforms baseline models, especially in complex complaint scenarios where information is distributed across text and images. This study underscores the value of multimodal interaction and expert validation in practical complaint understanding systems.

PDF Details DOI

TMLR Journal 2025 Journal Article

FedDUAL: A Dual-Strategy with Adaptive Loss and Dynamic Aggregation for Mitigating Data Heterogeneity in Federated Learning

Pranab Sahoo
Ashutosh Tripathi
Sriparna Saha
Samrat Mondal

Federated Learning (FL) marks a transformative approach to distributed model training by combining locally optimized models from various clients into a unified global model. While FL preserves data privacy by eliminating centralized storage, it encounters significant challenges such as performance degradation, slower convergence, and reduced robustness of the global model due to the heterogeneity in client data distributions. Among the various forms of data heterogeneity, label skew emerges as a particularly formidable and prevalent issue, especially in domains such as image classification. To address these challenges, we begin with comprehensive experiments to pinpoint the underlying issues in the FL training process, such as gradient instability and the emergence of sharp minima in the global model, both of which contribute to performance inconsistencies. Based on our findings, we introduce an innovative dual-strategy approach designed to effectively resolve these issues. First, we introduce an adaptive loss function for client-side training, meticulously crafted to preserve previously acquired knowledge while maintaining an optimal equilibrium between local optimization and global model coherence. Secondly, we develop a dynamic aggregation strategy for aggregating client models at the server. This approach adapts to each client's unique learning patterns, effectively addressing the challenges of diverse data across the network. Our comprehensive evaluation, conducted across three diverse real-world datasets, coupled with theoretical convergence guarantees, demonstrates the superior efficacy of our method compared to several established state-of-the-art approaches.

PDF Details

JBHI Journal 2024 Journal Article

CAGCL: Predicting Short- and Long-Term Breast Cancer Survival With Cross-Modal Attention and Graph Contrastive Learning

Susmita Palmal
Sriparna Saha
Nikhilanad Arya
Somanath Tripathy

In breast cancer treatment, accurately predicting how long a patient will survive is crucial for decision-making. This information guides treatment choices and supports patients' psychological recovery. To address this challenge, we introduce a novel predictive model to forecast breast cancer prognosis by leveraging diverse data sources, including clinical records, copy number variation, gene expressions, DNA methylation, microRNA (miRSeq) sequencing, and whole slide image data from the TCGA Database. The methodology incorporates graph contrastive learning with cross-modality attention (CAGCL), considering all possible combinations of the six distinct data modalities. Feature embeddings are enhanced through graph contrastive learning, which identifies subtle differences and similarities among samples. Further, to learn the complementary nature of information across multiple data modalities, a cross-attention framework is proposed and applied to the graph contrastive learning-based extracted features from various data sources for breast cancer survival prediction. It performs a binary classification to anticipate the likelihood of short- and long-term breast cancer survivors, delineated by a five-year threshold. The proposed model (CAGCL) showcases superior performance compared to baseline models and other state-of-the-art models. The model attains an accuracy of 0. 932, a sensitivity of 0. 954, a precision of 0. 958, an F1 score of 0. 956, and an AUC of 0. 948, underscoring its effectiveness in predicting breast cancer survival.

Details DOI

AAAI Conference 2024 Conference Paper

CLIPSyntel: CLIP and LLM Synergy for Multimodal Question Summarization in Healthcare

Akash Ghosh
Arkadeep Acharya
Raghav Jain
Sriparna Saha
Aman Chadha
Setu Sinha

In the era of modern healthcare, swiftly generating medical question summaries is crucial for informed and timely patient care. Despite the increasing complexity and volume of medical data, existing studies have focused solely on text-based summarization, neglecting the integration of visual information. Recognizing the untapped potential of combining textual queries with visual representations of medical conditions, we introduce the Multimodal Medical Question Summarization (MMQS) Dataset. This dataset, a major contribution of our work, pairs medical queries with visual aids, facilitating a richer and more nuanced understanding of patient needs. We also propose a framework, utilizing the power of Contrastive Language Image Pretraining(CLIP) and Large Language Models(LLMs), consisting of four modules that identify medical disorders, generate relevant context, filter medical concepts, and craft visually aware summaries. Our comprehensive framework harnesses the power of CLIP, a multimodal foundation model, and various general-purpose LLMs, comprising four main modules: the medical disorder identification module, the relevant context generation module, the context filtration module for distilling relevant medical concepts and knowledge, and finally, a general-purpose LLM to generate visually aware medical question summaries. Leveraging our MMQS dataset, we showcase how visual cues from images enhance the generation of medically nuanced summaries. This multimodal approach not only enhances the decision-making process in healthcare but also fosters a more nuanced understanding of patient queries, laying the groundwork for future research in personalized and responsive medical care. Disclaimer: The article features graphic medical imagery, a result of the subject's inherent requirements.

PDF Details DOI

AAAI Conference 2022 Conference Paper

Sentiment and Emotion-Aware Multi-Modal Complaint Identification

APOORVA SINGH
Soumyodeep Dey
Anamitra Singha
Sriparna Saha

The expression of displeasure on a consumer’s behalf towards an organization, product, or event is denoted via the speech act known as complaint. Customers typically post reviews on retail websites and various social media platforms about the products or services they purchase, and the reviews may include complaints about the products or services. Automatic detection of consumers’ complaints about items or services they buy can be critical for organizations and online merchants since they can use this insight to meet the customers’ requirements, including handling and addressing the complaints. Previous studies on Complaint Identification (CI) are limited to text. Images posted with the reviews can provide cues to identify complaints better, thus emphasizing the importance of incorporating multi-modal inputs into the process. Furthermore, the customer’s emotional state significantly impacts the complaint expression since emotions generally influence any speech act. As a result, the impact of emotion and sentiment on automatic complaint identification must also be investigated. One of the major contributions of this work is the creation of a new dataset- Complaint, Emotion, and Sentiment Annotated Multi-modal Amazon Reviews Dataset (CESAMARD), a collection of opinionated texts (reviews) and images of the products posted on the website of the retail giant Amazon. We present an attentionbased multi-modal, adversarial multi-task deep neural network model for complaint detection to demonstrate the utility of the multi-modal dataset. Experimental results indicate that the multi-modality and multi-tasking complaint identification outperforms uni-modal and single-task variants.

PDF Details

JBHI Journal 2021 Journal Article

MultiPredGO: Deep Multi-Modal Protein Function Prediction by Amalgamating Protein Structure, Sequence, and Interaction Information

Swagarika Jaharlal Giri
Pratik Dutta
Parth Halani
Sriparna Saha

Protein is an essential macro-nutrient for perceiving a wide range of biochemical activities and biological regulations in living cells. In this work, we have presented a novel multi-modal approach, named MultiPredGO, for predicting protein functions by utilizing two different kinds of information, namely protein sequence and the protein secondary structure. Here, our contributions are threefold; firstly, along with the protein sequence, we learn the feature representation from the protein structure. Secondly, we develop two different deep learning models after considering the characteristics of the underlying data patterns of the protein sequence and protein 3D structures. Finally, along with these two modalities, we have also utilized protein interaction information for expediting the efficiency of the proposed model in predicting the protein functions. For extracting features from different modalities, we have utilized various variations of the convolutional neural network. As the protein function classes are dependent on each other, we have used a neuro-symbolic hierarchical classification model, which resembles the structure of Gene Ontology (GO), for effectively predicting the dependent protein functions. Finally, to validate the goodness of our proposed method ( MultiPredGO ), we have compared our results with various uni-modal along with two well-known multi-modal protein function prediction approaches, namely, INGA and DeepGO. Results show that the overall performance of the proposed approach in terms of accuracy, F-measure, precision, and recall metrics are better than those by the state-of-the-art methods. MultiPredGO attains an average 13. 05% and 30. 87% improvements over the best existing comparing approach (DeepGO) for cellular component and molecular functions, respectively.

Details DOI

JBHI Journal 2020 Journal Article

A Multi-View Deep Neural Network Model for Chemical-Disease Relation Extraction From Imbalanced Datasets

Sayantan Mitra
Sriparna Saha
Mohammed Hasanuzzaman

Understanding the chemical-disease relations (CDR) is a crucial task in various biomedical domains. Manual mining of these information from biomedical literature is costly and time-consuming. To address these issues, various researches have been carried out to design an efficient automatic tool. In this paper, we propose a multi-view based deep neural network model for CDR task. Typically, multiple representations (or views) of the datasets are not available for this task. So, we train multiple conceptually different deep neural network models on the dataset to generate different abstract features, treated as different views. A novel loss function, “Penalized LF”, is defined to address the problem of imbalance dataset. The proposed loss function is generic in nature. The model is designed as a combination of Convolution Neural Network (CNN) and Bidirectional Long Short Term Memory (Bi-LSTM) network along with a Multi-Layer Perceptron (MLP). To show the efficacy of our proposed model, we have compared it with six baseline models and other state-of-the-art techniques, on “chemicals-and-disease-DFE” dataset, a free text dataset created by Li et al. from BioCreative V Chemical Disease Relation dataset. Results show that the proposed model attains highest $F1-score$ for individual classes, proving its efficiency in handling class imbalance problem in the dataset. To further demonstrate the efficacy of the proposed model, we have presented results on BioCreative V dataset and two Protein-Protein Interaction Identification (PPI) datasets, viz. , AiMed and BioInfer. All these results are also compared with the state-of-the-art models.

Details DOI

IS Journal 2019 Journal Article

Figure Summarization: A Multiobjective Optimization-Based Approach

Naveen Saini
Sriparna Saha
Vedavikas Potnuru
Rahul Grover
Pushpak Bhattacharyya

In the biomedical domain, figures in the scientific articles attribute significantly in understanding the core concepts. However, these figures are always difficult to interpret by the humans as well as machines and, thus, associated texts in the article are required to summarize the figures. This article proposes an unsupervised automatic summarization system for individual figures present in a scientific biomedical article, where different quality measures capturing relevance of the sentences to the figure are simultaneously optimized using the search capability of a multiobjective optimization technique to obtain a good set of sentences in the summary. A newly designed self-organizing map based genetic operator helping in new solution generation is also introduced in the multiobjective optimization framework. For evaluation of the proposed technique, 94 and 81 figures over two datasets from the biomedical literature are used. Our proposed system, namely MOOFigSum, obtains 5% and 11% improvements in terms of F1-measure metric over the unsupervised technique for both datasets, respectively, while in comparison to supervised techniques, MOOFigSum obtains 9% and 2% improvements over these datasets, respectively.

Details DOI

JBHI Journal 2019 Journal Article

Graph-Based Hub Gene Selection Technique Using Protein Interaction Information: Application to Sample Classification

Pratik Dutta
Sriparna Saha
Saurabh Gulati

Classification of samples of gene expression profile plays a significant role in prediction and diagnosis of diseases. In the task of sample classification, a robust feature selection algorithm is very much essential to identify the important genes from the high dimensional gene expression data. This paper explores the information of protein-protein interaction with a graph mining technique for finding a proper subset of features (genes), which further takes part in sample classification. Here, our contribution for feature selection is three-fold: first, all the genes are grouped into different clusters based on the integrated information of the gene expression values and their protein interactions using a multi-objective optimization based clustering approach. Second, the confidence scores of the protein interactions are incorporated in a popular graph mining algorithm namely Goldberg algorithm to find out the relevant features. These features are the topologically and functionally significant genes, named as hub genes. Finally, these hub genes are identified varying the degrees of the nodes, and those are utilized for the sample classification task. Different machine learning classifiers are exploited for this purpose, and the classification performance is measured with respect to various performance metrics namely accuracy, sensitivity, specificity, precision, F-measure, and Mathews coefficient correlation. Comparative analysis with respect to two baselines and several existing approaches proves the efficiency of the proposed approach. Furthermore, the robustness of the identified hub-gene modules is endorsed using some strong biological significance analysis.

Details DOI

IS Journal 2019 Journal Article

Multitask Representation Learning for Multimodal Estimation of Depression Level

Syed Arbaaz Qureshi
Sriparna Saha
Mohammed Hasanuzzaman
Gael Dias

We propose a novel multitask learning attention -based deep neural network model, which facilitates the fusion of various modalities. In particular, we use this network to both regress and classify the level of depression. Acoustic, textual, and visual modalities have been used to train our proposed network. Various experiments have been carried out on the benchmark dataset, namely, Distress Analysis Interview Corpus -a Wizard of Oz. From the results, we empirically justify that a) multitask learning networks cotrained over regression and classification have better performance compared to single -task networks, and b) the fusion of all the modalities helps in giving the most accurate estimation of depression with respect to regression.

Details DOI

JBHI Journal 2018 Journal Article

Simultaneous Clustering and Feature Weighting Using Multiobjective Optimization for Identifying Functionally Similar miRNAs

Sriparna Saha
Sudipta Acharya
Kavya K
Saisree Miriyala

MicroRNAs (miRNAs) are a type of RNAs, which are responsible for monitoring the gene expression values. Recent research asserts that miRNAs form some clustering on chromosomes. The miRNAs belonging to a particular cluster are highly similar in terms of their activity and they are termed as “coregulated” miRNAs. The current paper presents an approach that simultaneously performs two tasks: i) clustering of miRNAs into different categories based on some similarity measures ii) identification of proper weight values for different time points with respect to which expression values are available. In general, a large number of expression values are available for a given miRNA data set. All these values may not be suitable to be used equally to measure the similarity between two miRNAs. In the current study, the problem of proper selection of weight values for different time points and then determining the proper partitioning from the given miRNA data set utilizing the similarity computed using the new set of weight values is formulated as an optimization problem where several cluster validity indices are optimized as the goodness measures. To that end, a multiobjective differential evolution based optimization technique is utilized. The supremacy of the proposed technique is tested on three miRNA data sets in comparison to some recent approaches in terms of some popular performance measures like Silhouette index and DB-index. The observations are further supported by statistical and biological significance tests. Supplementary information is available at https://www.iitp.ac.in/~sriparna/journals.html.

Details DOI

JBHI Journal 2016 Journal Article

Multiobjective Simulated Annealing-Based Clustering of Tissue Samples for Cancer Diagnosis

Sudipta Acharya
Sriparna Saha
Yamini Thadisina

In the field of pattern recognition, the study of the gene expression profiles of different tissue samples over different experimental conditions has become feasible with the arrival of microarray-based technology. In cancer research, classification of tissue samples is necessary for cancer diagnosis, which can be done with the help of microarray technology. In this paper, we have presented a multiobjective optimization (MOO)-based clustering technique utilizing archived multiobjective simulated annealing(AMOSA) as the underlying optimization strategy for classification of tissue samples from cancer datasets. The presented clustering technique is evaluated for three open source benchmark cancer datasets [Brain tumor dataset, Adult Malignancy, and Small Round Blood Cell Tumors (SRBCT)]. In order to evaluate the quality or goodness of produced clusters, two cluster quality measures viz, adjusted rand index and classification accuracy ( $\% CoA$ ) are calculated. Comparative results of the presented clustering algorithm with ten state-of-the-art existing clustering techniques are shown for three benchmark datasets. Also, we have conducted a statistical significance test called t -test to prove the superiority of our presented MOO-based clustering technique over other clustering techniques. Moreover, significant gene markers have been identified and demonstrated visually from the clustering solutions obtained. In the field of cancer subtype prediction, this study can have important impact.

Details DOI

JBHI Journal 2016 Journal Article

Use of Semisupervised Clustering and Feature-Selection Techniques for Identification of Co-expressed Genes

Sriparna Saha
Abhay Kumar Alok
Asif Ekbal

Studying the patterns hidden in gene-expression data helps to understand the functionality of genes. In general, clustering techniques are widely used for the identification of natural partitionings from the gene expression data. In order to put constraints on dimensionality, feature selection is the key issue because not all features are important from clustering point of view. Moreover some limited amount of supervised information can help to fine tune the obtained clustering solution. In this paper, the problem of simultaneous feature selection and semisupervised clustering is formulated as a multiobjective optimization (MOO) task. A modern simulated annealing-based MOO technique namely AMOSA is utilized as the background optimization methodology. Here, features and cluster centers are represented in the form of a string and the assignment of genes to different clusters is done using a point symmetry-based distance. Six optimization criteria based on several internal and external cluster validity indices are utilized. In order to generate the supervised information, a popular clustering technique, Fuzzy C-mean, is utilized. Appropriate subset of features, proper number of clusters and the proper partitioning are determined using the search capability of AMOSA. The effectiveness of this proposed semisupervised clustering technique, Semi-FeaClustMOO, is demonstrated on five publicly available benchmark gene-expression datasets. Comparison results with the existing techniques for gene-expression data clustering again reveal the superiority of the proposed technique. Statistical and biological significance tests have also been carried out.

Details DOI