Multi-Scale Dynamic Sparse Token Multi-Instance Learning for Pathology Image Classification

In many challenging breast cancer pathology images, the proportion of truly informative tumor regions is extremely limited. The disparity between the essential information required for clinical diagnosis (Tumor area less than 10 $\%$ ) and the vast amount of data within Whole Slide Images (WSIs) makes it exceedingly difficult for pathologists to identify subtle lesions. To address the labor-intensive task imposed by this information gap, this paper proposes a dynamic sparse token based multi-instance learning framework. This framework incorporates a dynamic sparse layer into the transformer architecture, gradually adapting to selectively filter key instances beneficial for the task. Furthermore, to tackle complex scenarios in pathology image tasks, we introduce a weakly supervised cross-scale contrastive learning framework. This framework leverages pathology image features at different scales to perform contrastive learning at the bag-level representation to overcome existing challenges in multi-scale feature fusion in pathology image tasks. To validate the effectiveness and transferability of the model, we conducted various single-scale and multi-scale experiments across four cancer datasets and conducted interpretable analyses. Compared to other state-of-the-art methods, our classification model demonstrates superior performance across six evaluation metrics.

Authors

• Dajiang Lei Chongqing University of Post and Telecommunications
• Yuqi Zhang
• Haodong Wang School of Computer Science, Northwestern Polytechnical University Shaanxi Provincial Key Lab. of Speech and Image Information Processing National Engineering Laboratory for Integrated Aero-Space-Ground-Ocean Big Data Application Technology
• Xiaomin Xiong
• Bo Xu Institute of Automation, Chinese Academy of Sciences
• Guoyin Wang National Center for Applied Mathematics in Chongqing, Chongqing Normal University, Chongqing 401331, China

Keywords

• Pathology
• Feature extraction
• Contrastive learning
• Transformers
• Tumors
• Semantics
• Training
• Technological innovation
• Resource management
• Iterative methods
• Pathological Images
• Multi-instance Learning
• Pathology Image Classification
• Breast Cancer
• Learning Framework
• Feature Fusion
• Multi-scale Features
• Slide Images
• Self-supervised Learning
• Transformer Architecture
• Multi-scale Feature Fusion
• Model Performance
• Primary Tumor
• Positive Samples
• Validation Set
• Negative Samples
• Attention Mechanism
• Global Information
• Lung Squamous Cell Carcinoma
• Clinical Datasets
• Memory Bank
• Weight Allocation
• Multi-scale Network
• Multiple Instance Learning
• Feature Fusion Method
• Positive Instances
• Comparison Of Different Methods
• Semantic Gap
• Multi-scale Fusion
• Feature Pyramid Network
• Multi-scale pathological image analysis
• transformer
• whole slide image
• Humans
• Breast Neoplasms
• Image Interpretation, Computer-Assisted
• Female
• Machine Learning
• Algorithms
• Databases, Factual