A Scalable Graph-Based Framework for Multi-Organ Histology Image Classification

Graph-based approaches are successful for histology image classification tasks but still face many challenges, such as: 1) the lack of nuclei-level labels and the significant variations between histology images make it extremely difficult to extract discriminative high-level nuclei features like nuclei type, texture and micro-environment; 2) graph-based approaches cannot handle large-scale cell graph nodes typically contained in histology images; and 3) graph neural networks (GNNs) struggle to learn the long-range dependency of cell graphs. To address the above challenges, we propose a scalable graph-based framework for multi-organ histology image classification. We develop a two-step masked nuclei patches supervised training approach to extract discriminative high-level nuclei features for histology images without nuclei-level labels. Additionally, we introduce a nuclei sampling strategy to make our graph-based framework scalable for large-scale cell graphs. Furthermore, we propose H ier A rchical T ransformer Graph Neural Net work (HAT-Net+) for cell graph classi- fications. HAT-Net+ adopts Transformer to model the long-range dependency of cell graphs and a parameter-free approach to adaptively fuse different hierarchical graph representations of each layer. We achieved the state-of-the-art results on four public histology image classification datasets: CRC dataset (100%), Extended CRC dataset (98%), UZH dataset (96. 9%) and BACH dataset (88%). Unlike other methods, our approach can be used in various histology image classification tasks, even for images without nuclei-level labels, indicating its potential in cancer diagnosis. The code is available at https://github.com/suyouooooo/HAT-Net.

Authors

• Yu Bai China University of Petroleum (East China)
• Yue Mi
• Yihan Su
• Bo Zhang Northwestern Polytechnical University
• Zheng Zhang
• Jingyun Wu
• Haiwen Huang
• Yongping Xiong
• Xiangyang Gong
• Wendong Wang Beijing University of Posts and Telecommunications

Keywords

• Feature extraction
• Histopathology
• Task analysis
• Transformers
• Cancer
• Training
• Adaptation models
• Image Classification
• Scalable Framework
• Graph-based Framework
• Histology Image Classification
• Colorectal Cancer
• Graphical Representation
• Public Datasets
• Discriminative Features
• High-level Features
• Nodes In The Graph
• Classification Datasets
• Graph Neural Networks
• Long-range Dependencies
• Hierarchical Graph
• Colorectal Cancer Dataset
• Convolutional Neural Network
• Unsupervised Learning
• Grid Cells
• Two-step Approach
• Deep Features
• Images Of Nuclei
• Handcrafted Features
• Node Features
• Grid Cell Size
• Readout Method
• Transformer Layers
• Labeling Of Nuclei
• Gray Level Co-occurrence Matrix
• Segmentation Dataset
• State Of The Art Methods
• Cell-graph
• graph neural network
• transformer