Jun Shi

EAAI Journal 2026 Journal Article

Interpretable prediction and simplified calculation of blast load on structure surface based on machine learning and theoretical model

• Dingkun Yang
• Jian Yang
• Jun Shi

JBHI Journal 2025 Journal Article

A Trustworthy Curriculum Learning Guided Multi-Target Domain Adaptation Network for Autism Spectrum Disorder Classification

• Jiale Dun
• Jun Wang
• Juncheng Li
• Qianhui Yang
• Wenlong Hang
• Xiaofeng Lu
• Shihui Ying
• Jun Shi

Domain adaptation has demonstrated success in classification of multi-center autism spectrum disorder (ASD). However, current domain adaptation methods primarily focus on classifying data in a single target domain with the assistance of one or multiple source domains, lacking the capability to address the clinical scenario of identifying ASD in multiple target domains. In response to this limitation, we propose a Trustworthy Curriculum Learning Guided Multi-Target Domain Adaptation (TCL-MTDA) network for identifying ASD in multiple target domains. To effectively handle varying degrees of data shift in multiple target domains, we propose a trustworthy curriculum learning procedure based on the Dempster-Shafer (D-S) Theory of Evidence. Additionally, a domain-contrastive adaptation method is integrated into the TCL-MTDA process to align data distributions between source and target domains, facilitating the learning of domain-invariant features. The proposed TCL-MTDA method is evaluated on 437 subjects (including 220 ASD patients and 217 NCs) from the Autism Brain Imaging Data Exchange (ABIDE). Experimental results validate the effectiveness of our proposed method in multi-target ASD classification, achieving an average accuracy of 71. 46% (95% CI: 68. 85% - 74. 06%) across four target domains, significantly outperforming most baseline methods (p<0. 05).

JBHI Journal 2025 Journal Article

Partial-Label Contrastive Representation Learning for Fine-Grained Biomarkers Prediction From Histopathology Whole Slide Images

• Yushan Zheng
• Kun Wu
• Jun Li
• Kunming Tang
• Jun Shi
• Haibo Wu
• Zhiguo Jiang
• Wei Wang

In the domain of histopathology analysis, existing representation learning methods for biomarkers prediction from whole slide images (WSIs) face challenges due to the complexity of tissue subtypes and label noise problems. This paper proposed a novel partial-label contrastive representation learning approach to enhance the discrimination of histopathology image representations for fine-grained biomarkers prediction. We designed a partial-label contrastive clustering (PLCC) module for partial-label disambiguation and a dynamic clustering algorithm to sample the most representative features of each category to the clustering queue during the contrastive learning process. We conducted comprehensive experiments on three gene mutation prediction datasets, including USTC-EGFR, BRCA-HER2, and TCGA-EGFR. The results show that our method outperforms 9 existing methods in terms of Accuracy, AUC, and F1 Score. Specifically, our method achieved an AUC of 0. 950 in EGFR mutation subtyping of TCGA-EGFR and an AUC of 0. 853 in HER2 0/1+/2+/3+ grading of BRCA-HER2, which demonstrates its superiority in fine-grained biomarkers prediction from histopathology whole slide images.

AAAI Conference 2025 Conference Paper

Promptable Representation Distribution Learning and Data Augmentation for Gigapixel Histopathology WSI Analysis

• Kunming Tang
• Zhiguo Jiang
• Jun Shi
• Wei Wang
• Haibo Wu
• Yushan Zheng

Gigapixel image analysis, particularly for whole slide images (WSIs), often relies on multiple instance learning (MIL). Under the paradigm of MIL, patch image representations are extracted and then fixed during the training of the MIL classifiers for efficiency consideration. However, the invariance of representations makes it difficult to perform data augmentation for WSI-level model training, which significantly limits the performance of the downstream WSI analysis. The current data augmentation methods for gigapixel images either introduce additional computational costs or result in a loss of semantic information, which is hard to meet the requirements for efficiency and stability needed for WSI model training. In this paper, we propose a Promptable Representation Distribution Learning framework (PRDL) for both patch-level representation learning and WSI-level data augmentation. Meanwhile, we explore the use of prompts to guide data augmentation in feature space, which achieves promptable data augmentation for training robust WSI-level models. The experimental results have demonstrated that the proposed method stably outperforms state-of-the-art methods.

JBHI Journal 2025 Journal Article

Re-Visible Dual-Domain Self-Supervised Deep Unfolding Network for MRI Reconstruction

• Hao Zhang
• Qi Wang
• Jian Sun
• Zhijie Wen
• Jun Shi
• Shihui Ying

Magnetic Resonance Imaging (MRI) is widely used in clinical practice, but suffers from prolonged acquisition time. Although deep learning methods have been proposed to accelerate acquisition and demonstrate promising performance, they rely on high-quality fully-sampled datasets for training in a supervised manner. However, such datasets are time-consuming and expensive-to-collect, which constrains their broader applications. On the other hand, self-supervised methods offer an alternative by enabling learning from under-sampled data alone, but most existing methods rely on further partitioned under-sampled k-space data as model's input for training, which causes an input distribution shift between the the training stage and the inference stage. Additionally, their models have not effectively incorporated comprehensive image priors, leading to degraded reconstruction performance. In this paper, we propose a novel re-visible dual-domain self-supervised deep unfolding network to address these issues when only under-sampled datasets are available. Specifically, by incorporating re-visible dual-domain loss, all under-sampled k-space data are utilized during training to mitigate the input distribution shift caused by further partitioning. This design enables the model to implicitly adapt to all under-sampled k-space data as input. Additionally, we design a Deep Unfolding Network based on Chambolle and Pock Proximal Point Algorithm (DUN-CP-PPA) to achieve end-to-end reconstruction. By employing a Spatial-Frequency Feature Extraction (SFFE) block to capture both global and local representations, the model effectively integrates imaging physics with comprehensive image priors to enhance reconstruction performance. Experiments on both single-coil and multi-coil datasets demonstrate that our method outperforms state-of-the-art approaches in terms of reconstruction performance and generalization capability.

JBHI Journal 2025 Journal Article

Topological GCN Guided Improved Conformer for Detection of Hip Landmarks From Ultrasound Images

• Tianxiang Huang
• Jing Shi
• Ge Jin
• Juncheng Li
• Jun Wang
• Qian Wang
• Jun Du
• Jun Shi

The B-mode ultrasound based computer-aided diagnosis (CAD) has shown its effectiveness for diagnosis of Developmental Dysplasia of the Hip (DDH) in infants within 6 months. Hip landmark detection is a feasible way for the CAD of DDH according to the Graf's method. However, existing landmark detection algorithms mainly focus on designing special models to capture the features from hip ultrasound images, but generally ignore the important spatial relations among different landmarks. To this end, a novel weakly supervised learning-based algorithm, the Topological Graph Convolutional Network (TGCN) guided Improved Conformer (TGCN-ICF), is proposed for detecting landmarks from hip ultrasound images. The TGCN-ICF includes two subnetworks: an Improved Conformer (ICF) subnetwork to generate heatmaps and constraint vectors from ultrasound images, and a TGCN subnetwork to additionally explore topological relations among hip landmarks with the guidance of class labels for further refining and improving the detection accuracy. Moreover, a new Mutual Modulation Fusion (MMF) module is developed to fully exchange and fuse the extracted feature information from the convolutional neural network (CNN) and Transformer branches in ICF. Meanwhile, a novel Mutual Supervision Constraint (MSC) strategy is designed to provide a constraint for detection of each hip landmark. The experimental results on two real-world DDH datasets demonstrate that the TGCN-ICF outperforms all the compared algorithms, suggesting its potential applications.

JBHI Journal 2025 Journal Article

Ψ-Net: Triple-Branch Network with Cross-Branch Alternately Updated Fusion for Diagnosis of Bicuspid Aortic Valve Via Dual-View Echocardiography

• Jiayan Chen
• Zibire Fulati
• Lina Luan
• Xueying Zhou
• Juncheng Li
• Jun Wang
• Haiyan Chen
• Jun Shi

Bicuspid Aortic Valve (BAV) can be diagnosed by Transthoracic Echocardiography (TTE), particularly on the parasternal short‑axis view. In this work, a Triple-Branch Network (named Ψ-Net) is proposed as a Computer-Aided Diagnosis (CAD) model for BAV based on the paired TTE images of aortic valve. This Ψ-shaped triple-branch network effectively learns both the view-common and view-specific features from the paired TTE images for improving feature representation. Moreover, a novel cross-branch alternately updated fusion block is developed by implementing alternately updated clique mechanism cross multiple branches, which maximizes cross-branch feature interaction among the Ψ-Net to enhance multi-view feature fusion. On the other hand, a multi-task self-supervised learning framework is developed to capture inherent properties from limited dual-view TTE samples by integrating the dual-view masked image modelling and Disentangled Representation Learning (DRL) into a unified framework. Specifically, an additional view classification task is designed and embedded into this framework for predicting which view a specific feature belongs to, so as to further promote the disentanglement learning of view-common and view-specific features by DRL. Moreover, the Shapley Value based weight adjustment strategy is designed to automatically assign weights to individual losses in objective function, which can dynamically balance the contribution of each loss term. The experimental results on two BAV TTE datasets demonstrate that Ψ-Net outperforms all the compared algorithms, suggesting its effectiveness in the diagnosis of BAV.

JBHI Journal 2024 Journal Article

Correction to: Multi-View Feature Transformation Based SVM+ for Computer-Aided Diagnosis of Liver Cancers With Ultrasound Image [Mar 23 1512-1523]

• Huili Zhang
• Lehang Guo
• Jun Wang
• Shihui Ying
• Jun Shi

Presents corrections to the paper, Multi-View Feature Transformation Based SVM+ for Computer-Aided Diagnosis of Liver Cancers With Ultrasound Image.

JBHI Journal 2024 Journal Article

Involution Transformer Based U-Net for Landmark Detection in Ultrasound Images for Diagnosis of Infantile DDH

• Tianxiang Huang
• Jing Shi
• Juncheng Li
• Jun Wang
• Jun Du
• Jun Shi

The B-mode ultrasound based computer-aided diagnosis (CAD) has demonstrated its effectiveness for diagnosis of Developmental Dysplasia of the Hip (DDH) in infants, which can conduct the Graf's method by detecting landmarks in hip ultrasound images. However, it is still necessary to explore more valuable information around these landmarks to enhance feature representation for improving detection performance in the detection model. To this end, a novel Involution Transformer based U-Net (IT-UNet) network is proposed for hip landmark detection. The IT-UNet integrates the efficient involution operation into Transformer to develop an Involution Transformer module (ITM), which consists of an involution attention block and a squeeze-and-excitation involution block. The ITM can capture both the spatial-related information and long-range dependencies from hip ultrasound images to effectively improve feature representation. Moreover, an Involution Downsampling block (IDB) is developed to alleviate the issue of feature loss in the encoder modules, which combines involution and convolution for the purpose of downsampling. The experimental results on two DDH ultrasound datasets indicate that the proposed IT-UNet achieves the best landmark detection performance, indicating its potential applications.

IJCAI Conference 2024 Conference Paper

Predictive Accuracy-Based Active Learning for Medical Image Segmentation

• Jun Shi
• Shulan Ruan
• Ziqi Zhu
• Minfan Zhao
• Hong An
• Xudong Xue
• Bing Yan

Active learning is considered a viable solution to alleviate the contradiction between the high dependency of deep learning-based segmentation methods on annotated data and the expensive pixel-level annotation cost of medical images. However, most existing methods suffer from unreliable uncertainty assessment and the struggle to balance diversity and informativeness, leading to poor performance in segmentation tasks. In response, we propose an efficient Predictive Accuracy-based Active Learning (PAAL) method for medical image segmentation, first introducing predictive accuracy to define uncertainty. Specifically, PAAL mainly consists of an Accuracy Predictor (AP) and a Weighted Polling Strategy (WPS). The former is an attached learnable module that can accurately predict the segmentation accuracy of unlabeled samples relative to the target model with the predicted posterior probability. The latter provides an efficient hybrid querying scheme by combining predicted accuracy and feature representation, aiming to ensure the uncertainty and diversity of the acquired samples. Extensive experiment results on multiple datasets demonstrate the superiority of PAAL. PAAL achieves comparable accuracy to fully annotated data while reducing annotation costs by approximately 50% to 80%, showcasing significant potential in clinical applications. The code is available at https: //github. com/shijun18/PAAL-MedSeg.

JBHI Journal 2023 Journal Article

Immunotherapy Efficacy Prediction for Non-Small Cell Lung Cancer Using Multi-View Adaptive Weighted Graph Convolutional Networks

• Qiong Wu
• Jun Wang
• Zongqiong Sun
• Lei Xiao
• Wenhao Ying
• Jun Shi

Immunotherapy is an effective way to treat non-small cell lung cancer (NSCLC). The efficacy of immunotherapy differs from person to person and may cause side effects, making it important to predict the efficacy of immunotherapy before surgery. Radiomics based on machine learning has been successfully used to predict the efficacy of NSCLC immunotherapy. However, most studies only considered the radiomic features of the individual patient, ignoring the inter-patient correlations. Besides, they usually concatenated different features as the input of a single-view model, failing to consider the complex correlation among features of multiple types. To this end, we propose a multi-view adaptive weighted graph convolutional network (MVAW-GCN) for the prediction of NSCLC immunotherapy efficacy. Specifically, we group the radiomic features into several views according to the type of the fitered images they extracted from. We construct a graph in each view based on the radiomic features and phenotypic information. An attention mechanism is introduced to automatically assign weights to each view. Considering the view-shared and view-specific knowledge of radiomic features, we propose separable graph convolution that decomposes the output of the last convolution layer into two components, i. e. , the view-shared and view-specific outputs. We maximize the consistency and enhance the diversity among different views in the learning procedure. The proposed MVAW-GCN is evaluated on 107 NSCLC patients, including 52 patients with valid efficacy and 55 patients with invalid efficacy. Our method achieved an accuracy of 77. 27% and an area under the curve (AUC) of 0. 7780, indicating its effectiveness in NSCLC immunotherapy efficacy prediction.

JBHI Journal 2023 Journal Article

Multi-Scale Efficient Graph-Transformer for Whole Slide Image Classification

• Saisai Ding
• Juncheng Li
• Jun Wang
• Shihui Ying
• Jun Shi

The multi-scale information among the whole slide images (WSIs) is essential for cancer diagnosis. Although the existing multi-scale vision Transformer has shown its effectiveness for learning multi-scale image representation, it still cannot work well on the gigapixel WSIs due to their extremely large image sizes. To this end, we propose a novel Multi-scale Efficient Graph-Transformer (MEGT) framework for WSI classification. The key idea of MEGT is to adopt two independent efficient Graph-based Transformer (EGT) branches to process the low-resolution and high-resolution patch embeddings (i. e. , tokens in a Transformer) of WSIs, respectively, and then fuse these tokens via a multi-scale feature fusion module (MFFM). Specifically, we design an EGT to efficiently learn the local-global information of patch tokens, which integrates the graph representation into Transformer to capture spatial-related information of WSIs. Meanwhile, we propose a novel MFFM to alleviate the semantic gap among different resolution patches during feature fusion, which creates a non-patch token for each branch as an agent to exchange information with another branch by cross-attention mechanism. In addition, to expedite network training, a new token pruning module is developed in EGT to reduce the redundant tokens. Extensive experiments on both TCGA-RCC and CAMELYON16 datasets demonstrate the effectiveness of the proposed MEGT.

JBHI Journal 2023 Journal Article

Multi-View Feature Transformation Based SVM+ for Computer-Aided Diagnosis of Liver Cancers With Ultrasound Images

• Huili Zhang
• Lehang Guo
• Jun Wang
• Shihui Ying
• Jun Shi

It is feasible to improve the performance of B-mode ultrasound (BUS) based computer-aided diagnosis (CAD) for liver cancers by transferring knowledge from contrast-enhanced ultrasound (CEUS) images. In this work, we propose a novel feature transformation based support vector machine plus (SVM+) algorithm for this transfer learning task by introducing feature transformation into the SVM+ framework (named FSVM+). Specifically, the transformation matrix in FSVM+ is learned to minimize the radius of the enclosing ball of all samples, while the SVM+ is used to maximize the margin between two classes. Moreover, to capture more transferable information from multiple CEUS phase images, a multi-view FSVM+ (MFSVM+) is further developed, which transfers knowledge from three CEUS images from three phases, i. e. , arterial phase, portal venous phase, and delayed phase, to the BUS-based CAD model. MFSVM+ innovatively assigns appropriate weights for each CEUS image by calculating the maximum mean discrepancy between a pair of BUS and CEUS images, which can capture the relationship between source and target domains. The experimental results on a bi-modal ultrasound liver cancer dataset demonstrate that MFSVM+ achieves the best classification accuracy of 88. 24±1. 28%, sensitivity of 88. 32±2. 88%, specificity of 88. 17±2. 91%, suggesting its effectiveness in promoting the diagnostic accuracy of BUS-based CAD.

JBHI Journal 2023 Journal Article

Reconstruction of Quantitative Susceptibility Mapping from Total Field Maps with Local Field Maps Guided UU-Net

• Zheng Li
• Shihui Ying
• Jun Wang
• Hongjian He
• Jun Shi

Quantitative susceptibility mapping (QSM) is an emerging computational technique based on the magnetic resonance imaging (MRI) phase signal, which can provide magnetic susceptibility values of tissues. The existing deep learning-based models mainly reconstruct QSM from local field maps. However, the complicated inconsecutive reconstruction steps not only accumulate errors for inaccurate estimation, but also are inefficient in clinical practice. To this end, a novel local field maps guided UU-Net with Self- and Cross-Guided Transformer (LGUU-SCT-Net) is proposed to reconstruct QSM directly from the total field maps. Specifically, we propose to additionally generate the local field maps as the auxiliary supervision during the training stage. This strategy decomposes the more complicated mapping from total maps to QSM into two relatively easier ones, effectively alleviating the difficulty of direct mapping. Meanwhile, an improved U-Net model, named LGUU-SCT-Net, is further designed to promote the nonlinear mapping ability. The long-range connections are designed between two sequentially stacked U-Nets to bring more feature fusions and facilitate the information flow. The Self- and Cross-Guided Transformer integrated into these connections further captures multi-scale channel-wise correlations and guides the fusion of multi-scale transferred features, assisting in the more accurate reconstruction. The experimental results on an in-vivo dataset demonstrate the superior reconstruction results of our proposed algorithm.

JBHI Journal 2023 Journal Article

Two-Stage Self-Supervised Cycle-Consistency Transformer Network for Reducing Slice Gap in MR Images

• Zhiyang Lu
• Jian Wang
• Zheng Li
• Shihui Ying
• Jun Wang
• Jun Shi
• Dinggang Shen

Magnetic resonance (MR) images are usually acquired with large slice gap in clinical practice, i. e. , low resolution (LR) along the through-plane direction. It is feasible to reduce the slice gap and reconstruct high-resolution (HR) images with the deep learning (DL) methods. To this end, the paired LR and HR images are generally required to train a DL model in a popular fully supervised manner. However, since the HR images are hardly acquired in clinical routine, it is difficult to get sufficient paired samples to train a robust model. Moreover, the widely used convolutional Neural Network (CNN) still cannot capture long-range image dependencies to combine useful information of similar contents, which are often spatially far away from each other across neighboring slices. To this end, a Two-stage Self-supervised Cycle-consistency Transformer Network (TSCTNet) is proposed to reduce the slice gap for MR images in this work. A novel self-supervised learning (SSL) strategy is designed with two stages respectively for robust network pre-training and specialized network refinement based on a cycle-consistency constraint. A hybrid Transformer and CNN structure is utilized to build an interpolation model, which explores both local and global slice representations. The experimental results on two public MR image datasets indicate that TSCTNet achieves superior performance over other compared SSL-based algorithms.

JBHI Journal 2022 Journal Article

A Convolutional Neural Network and Graph Convolutional Network Based Framework for Classification of Breast Histopathological Images

• Zhiyang Gao
• Zhiyang Lu
• Jun Wang
• Shihui Ying
• Jun Shi

The spatial correlation among different tissue components is an essential characteristic for diagnosis of breast cancers based on histopathological images. Graph convolutional network (GCN) can effectively capture this spatial feature representation, and has been successfully applied to the histopathological image based computer-aided diagnosis (CAD). However, the current GCN-based approaches need complicated image preprocessing for graph construction. In this work, we propose a novel CAD framework for classification of breast histopathological images, which integrates both convolutional neural network (CNN) and GCN (named CNN-GCN) into a unified framework, where CNN learns high-level features from histopathological images for further adaptive graph construction, and the generated graph is then fed to GCN to learn the spatial features of histopathological images for the classification task. In particular, a novel clique GCN (cGCN) is proposed to learn more effective graph representation, which can arrange both forward and backward connections between any two graph convolution layers. Moreover, a new group graph convolution is further developed to replace the classical graph convolution of each layer in cGCN, so as to reduce redundant information and implicitly select superior fused feature representation. The proposed clique group GCN (cgGCN) is then embedded in the CNN-GCN framework (named CNN-cgGCN) to promote the learned spatial representation for diagnosis of breast cancers. The experimental results on two public breast histopathological image datasets indicate the effectiveness of the proposed CNN-cgGCN with superior performance to all the compared algorithms.

JBHI Journal 2022 Journal Article

Diagnosis of Infantile Hip Dysplasia With B-Mode Ultrasound via Two-Stage Meta-Learning Based Deep Exclusivity Regularized Machine

• Bangming Gong
• Jing Shi
• Xiangmin Han
• Huan Zhang
• Yuemin Huang
• Liwei Hu
• Jun Wang
• Jun Du

The B-mode ultrasound (BUS) based computer-aided diagnosis (CAD) has shown its effectiveness for developmental dysplasia of the hip (DDH) in infants. In this work, a two-stage meta-learning based deep exclusivity regularized machine (TML-DERM) is proposed for the BUS-based CAD of DDH. TML-DERM integrates deep neural network (DNN) and exclusivity regularized machine into a unified framework to simultaneously improve the feature representation and classification performance. Moreover, the first-stage meta-learning is mainly conducted on the DNN module to alleviate the overfitting issue caused by the significantly increased parameters in DNN, and a random sampling strategy is adopted to self-generate the meta-tasks; while the second-stage meta-learning mainly learns the combination of multiple weak classifiers by a weight vector to improve the classification performance, and also optimizes the unified framework again. The experimental results on a DDH ultrasound dataset show the proposed TML-DERM algorithm achieves the superior classification performance with the mean accuracy of 85. 89%, sensitivity of 86. 54%, and specificity of 85. 23%.

JBHI Journal 2022 Journal Article

Joint Localization and Classification of Breast Cancer in B-Mode Ultrasound Imaging via Collaborative Learning With Elastography

• Weichang Ding
• Jun Wang
• Weijun Zhou
• Shichong Zhou
• Cai Chang
• Jun Shi

Convolutional neural networks (CNNs) have been successfully applied in the computer-aided ultrasound diagnosis for breast cancer. Up to now, several CNN-based methods have been proposed. However, most of them consider tumor localization and classification as two separate steps, rather than performing them simultaneously. Besides, they suffer from the limited diagnosis information in the B-mode ultrasound (BUS) images. In this study, we develop a novel network ResNet-GAP that incorporates both localization and classification into a unified procedure. To enhance the performance of ResNet-GAP, we leverage stiffness information in the elastography ultrasound (EUS) modality by collaborative learning in the training stage. Specifically, a dual-channel ResNet-GAP network is developed, one channel for BUS and the other for EUS. In each channel, multiple class activity maps (CAMs) are generated using a series of convolutional kernels of different sizes. The multi-scale consistency of the CAMs in both channels are further considered in network optimization. Experiments on 264 patients in this study show that the newly developed ResNet-GAP achieves an accuracy of 88. 6%, a sensitivity of 95. 3%, a specificity of 84. 6%, and an AUC of 93. 6% on the classification task, and a 1. 0NLF of 87. 9% on the localization task, which is better than some state-of-the-art approaches.

JBHI Journal 2022 Journal Article

S2Q-Net: Mining the High-Pass Filtered Phase Data in Susceptibility Weighted Imaging for Quantitative Susceptibility Mapping

• Zhiyang Lu
• Jun Li
• Chaoyue Wang
• Rongjun Ge
• Lili Chen
• Hongjian He
• Jun Shi

Susceptibility weighted imaging (SWI) is a routine magnetic resonance imaging (MRI) sequence that combines the magnitude and high-pass filtered phase images to qualitatively enhance the image contrasts related to tissue susceptibility. Tremendous amounts of the high-pass filtered phase data with low signal to noise ratio and incomplete background field removal have thus been collected under default clinical settings. Since SWI cannot quantitatively estimate the susceptibility, it is thus non-trivial to derive quantitative susceptibility mapping (QSM) directly from these redundant phase data, which effectively promotes the mining of the SWI data collected previously. To this end, a novel deep learning based SWI-to-QSM-Net (S2Q-Net) is proposed for QSM reconstruction from SWI high-pass filtered phase data. S2Q-Net firstly estimates the edge maps of QSM to integrate edge prior into features, which benefits the network to reconstruct QSM with realistic and clear tissue boundaries. Furthermore, a novel Second-order Cross Dense Block is proposed in S2Q-Net, which can capture rich inter-region interactions to provide more non-local phase information related to local tissue susceptibility. Experimental results on both simulated and in-vivo data indicate its superiority over all the compared deep learning based QSM reconstruction methods.

JBHI Journal 2022 Journal Article

Self-Supervised Bi-Channel Transformer Networks for Computer-Aided Diagnosis

• Ronglin Gong
• Xiangmin Han
• Jun Wang
• Shihui Ying
• Jun Shi

Self-supervised learning (SSL) can alleviate the issue of small sample size, which has shown its effectiveness for the computer-aided diagnosis (CAD) models. However, since the conventional SSL methods share the identical backbone in both the pretext and downstream tasks, the pretext network generally cannot be well trained in the pre-training stage, if the pretext task is totally different from the downstream one. In this work, we propose a novel task-driven SSL method, namely Self-Supervised Bi-channel Transformer Networks (SSBTN), to improve the diagnostic accuracy of a CAD model by enhancing SSL flexibility. In SSBTN, we innovatively integrate two different networks for the pretext and downstream tasks, respectively, into a unified framework. Consequently, the pretext task can be flexibly designed based on the data characteristics, and the corresponding designed pretext network thus learns more effective feature representation to be transferred to the downstream network. Furthermore, a transformer-based transfer module is developed to efficiently enhance knowledge transfer by conducting feature alignment between two different networks. The proposed SSBTN is evaluated on two publicly available datasets, namely the full-field digital mammography INbreast dataset and the wireless video capsule CrohnIPI dataset. The experimental results indicate that the proposed SSBTN outperforms all the compared algorithms.

JBHI Journal 2021 Journal Article

Multi-Source Transfer Learning Via Multi-Kernel Support Vector Machine Plus for B-Mode Ultrasound-Based Computer-Aided Diagnosis of Liver Cancers

• Huili Zhang
• Lehang Guo
• Dan Wang
• Jun Wang
• Lili Bao
• Shihui Ying
• Huixiong Xu
• Jun Shi

B-mode ultrasound (BUS) imaging is a routine tool for diagnosis of liver cancers, while contrast-enhanced ultrasound (CEUS) provides additional information to BUS on the local tissue vascularization and perfusion to promote diagnostic accuracy. In this work, we propose to improve the BUS-based computer aided diagnosis for liver cancers by transferring knowledge from the multi-view CEUS images, including the arterial phase, portal venous phase, and delayed phase, respectively. To make full use of the shared labels of paired of BUS and CEUS images to guide knowledge transfer, support vector machine plus (SVM+), a specifically designed transfer learning (TL) classifier for paired data with shared labels, is adopted for this supervised TL. A nonparallel hyperplane based SVM+ (NHSVM+) is first proposed to improve the TL performance by transferring the per-class knowledge from source domain to the corresponding target domain. Moreover, to handle the issue of multi-source TL, a multi-kernel learning based NHSVM+ (MKL-NHSVM+) algorithm is further developed to effectively transfer multi-source knowledge from multi-view CEUS images. The experimental results indicate that the proposed MKL-NHSVM+ outperforms all the compared algorithms for diagnosis of liver cancers, whose mean classification accuracy, sensitivity, and specificity are 88. 18 ± 3. 16 %, 86. 98 ± 4. 77 %, and 89. 42±3. 77%, respectively.

JBHI Journal 2021 Journal Article

Stain Standardization Capsule for Application-Driven Histopathological Image Normalization

• Yushan Zheng
• Zhiguo Jiang
• Haopeng Zhang
• Fengying Xie
• Dingyi Hu
• Shujiao Sun
• Jun Shi
• Chenghai Xue

Color consistency is crucial to developing robust deep learning methods for histopathological image analysis. With the increasing application of digital histopathological slides, the deep learning methods are probably developed based on the data from multiple medical centers. This requirement makes it a challenging task to normalize the color variance of histopathological images from different medical centers. In this paper, we propose a novel color standardization module named stain standardization capsule based on the capsule network and the corresponding dynamic routing algorithm. The proposed module can learn and generate uniform stain separation outputs for histopathological images in various color appearance without the reference to manually selected template images. The proposed module is light and can be jointly trained with the application-driven CNN model. The proposed method was validated on three histopathology datasets and a cytology dataset, and was compared with state-of-the-art methods. The experimental results have demonstrated that the SSC module is effective in improving the performance of histopathological image analysis and has achieved the best performance in the compared methods.

JBHI Journal 2019 Journal Article

MR Image Super-Resolution via Wide Residual Networks With Fixed Skip Connection

• Jun Shi
• Zheng Li
• Shihui Ying
• Chaofeng Wang
• Qingping Liu
• Qi Zhang
• Pingkun Yan

Spatial resolution is a critical imaging parameter in magnetic resonance imaging. The image super-resolution (SR) is an effective and cost efficient alternative technique to improve the spatial resolution of MR images. Over the past several years, the convolutional neural networks (CNN)-based SR methods have achieved state-of-the-art performance. However, CNNs with very deep network structures usually suffer from the problems of degradation and diminishing feature reuse, which add difficulty to network training and degenerate the transmission capability of details for SR. To address these problems, in this work, a progressive wide residual network with a fixed skip connection (named FSCWRN) based SR algorithm is proposed to reconstruct MR images, which combines the global residual learning and the shallow network based local residual learning. The strategy of progressive wide networks is adopted to replace deeper networks, which can partially relax the above-mentioned problems, while a fixed skip connection helps provide rich local details at high frequencies from a fixed shallow layer network to subsequent networks. The experimental results on one simulated MR image database and three real MR image databases show the effectiveness of the proposed FSCWRN SR algorithm, which achieves improved reconstruction performance compared with other algorithms.

JBHI Journal 2018 Journal Article

Multimodal Neuroimaging Feature Learning With Multimodal Stacked Deep Polynomial Networks for Diagnosis of Alzheimer's Disease

• Jun Shi
• Xiao Zheng
• Yan Li
• Qi Zhang
• Shihui Ying

The accurate diagnosis of Alzheimer's disease (AD) and its early stage, i. e. , mild cognitive impairment, is essential for timely treatment and possible delay of AD. Fusion of multimodal neuroimaging data, such as magnetic resonance imaging (MRI) and positron emission tomography (PET), has shown its effectiveness for AD diagnosis. The deep polynomial networks (DPN) is a recently proposed deep learning algorithm, which performs well on both large-scale and small-size datasets. In this study, a multimodal stacked DPN (MM-SDPN) algorithm, which MM-SDPN consists of two-stage SDPNs, is proposed to fuse and learn feature representation from multimodal neuroimaging data for AD diagnosis. Specifically speaking, two SDPNs are first used to learn high-level features of MRI and PET, respectively, which are then fed to another SDPN to fuse multimodal neuroimaging information. The proposed MM-SDPN algorithm is applied to the ADNI dataset to conduct both binary classification and multiclass classification tasks. Experimental results indicate that MM-SDPN is superior over the state-of-the-art multimodal feature-learning-based algorithms for AD diagnosis.

JBHI Journal 2017 Journal Article

Histopathological Image Classification With Color Pattern Random Binary Hashing-Based PCANet and Matrix-Form Classifier

• Jun Shi
• Jinjie Wu
• Yan Li
• Qi Zhang
• Shihui Ying

The computer-aided diagnosis for histopathological images has attracted considerable attention. Principal component analysis network (PCANet) is a novel deep learning algorithm for feature learning with the simple network architecture and parameters. In this study, a color pattern random binary hashing-based PCANet (C-RBH-PCANet) algorithm is proposed to learn an effective feature representation from color histopathological images. The color norm pattern and angular pattern are extracted from the principal component images of R, G, and B color channels after cascaded PCA networks. The random binary encoding is then performed on both color norm pattern images and angular pattern images to generate multiple binary images. Moreover, we rearrange the pooled local histogram features by spatial pyramid pooling to a matrix-form for reducing the dimension of feature and preserving spatial information. Therefore, a C-RBH-PCANet and matrix-form classifier-based feature learning and classification framework is proposed for diagnosis of color histopathological images. The experimental results on three color histopathological image datasets show that the proposed C-RBH-PCANet algorithm is superior to the original PCANet and other conventional unsupervised deep learning algorithms, while the best performance is achieved by the proposed feature learning and classification framework that combines C-RBH-PCANet and matrix-form classifier.