Author name cluster

Yun Gu

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

16 papers

2 author rows

JBHI Journal 2026 Journal Article

MedSegAgent: A Universal and Scalable Multi-Agent System for Instructive Medical Image Segmentation

Ziyan Huang
Haoyu Wang
Jin Ye
Yuanfeng Ji
Xiaowei Hu
Lihao Liu
Zhikai Yang
Wei Li

Medical image segmentation is vital for clinical diagnosis and treatment; however, current solutions face three major limitations: (1) the lack of a universal framework capable of handling diverse modalities and anatomical targets, (2) the limited scalability to adapt to evolving clinical needs and new datasets, and (3) the lack of instructive interfaces that make models usable for non-expert users. To address these challenges, this paper presents MedSegAgent, a universal and scalable multi-agent system for instructive medical image segmentation. Specifically, MedSegAgent comprises five agents: one query parsing agent that processes natural language requests, three coarse-to-fine filtering agents (modality filtering, anatomical filtering, and label selection) for identifying relevant datasets and label values, and one execution agent responsible for model inference and result integration. Based on this framework, MedSegAgent utilizes 23 diverse datasets and pre-trained models to perform 343 types of segmentation across various modalities and anatomical targets. Experimental results demonstrate that MedSegAgent simplifies model selection while maintaining high performance, accurately identifying matching datasets and labels in 94. 27% of queries and locating at least one suitable match in 99. 03% of queries. MedSegAgent offers a universal and scalable solution for diverse medical image segmentation tasks, bridging the gap between user-friendly queries and the complexities of model selection and deployment. Our code is publicly available at https://github.com/uni-medical/MedSegAgent.

Details DOI

JBHI Journal 2025 Journal Article

Bridging Brain and Machine: A Neurocognitive Framework for Early Alzheimer's Detection Through Hierarchical Speech Processing

Xiaohong Lyu
Jiayuan Bai
Pengfei Li
Fan Zhang
Xianglin Zhang
Yun Gu
Ning Liu

Alzheimer's disease progressively disrupts temporal-frontal language networks, manifesting through degraded speech production detectable years before clinical dementia diagnosis. Traditional automated detection systems rely on handcrafted acoustic features, including mel-frequency cepstral coefficients, that may discard diagnostically relevant temporal dynamics through fixed spectral transformations. Drawing inspiration from cognitive neuroscience models of hierarchical speech processing, we propose the neuro residual acoustic learning framework, an end-to-end system operating directly on raw waveforms while incorporating architectural principles mirroring biological auditory pathways. The framework implements acoustic-symbolic transformation bridging continuous signals and discrete phonetic-acoustic categories analogous to primary auditory cortex processing, hierarchical temporal encoding through residual blocks with dilated convolutions capturing multi-scale patterns across phoneme to utterance timescales, and squeeze-and-excitation attention mechanisms modeling prefrontal modulation of temporal representations. Evaluation on NCMMSC2021 Mandarin speech demonstrates 86. 55% accuracy on extended utterances and 81. 35% on brief segments, surpassing contemporary methods, including multimodal architectures and large language model hybrids, by 2. 88% and 2. 43% respectively. Cross-linguistic validation on INTERSPEECH2020 ADReSS English speech achieves 70. 83% accuracy, exceeding baselines by 2. 08%. Confusion matrix analysis reveals 93. 3% healthy control recall and asymmetric Alzheimer's disease versus mild cognitive impairment misclassification patterns consistent with prodromal acoustic biomarkers.

Details DOI

IROS Conference 2025 Conference Paper

R2Nav: Robust, Real-time Test Time Adaptation for Robot Assisted Endoluminal Navigation

Junyang Wu
Yimin Chu
Haixia Peng
Yun Gu
Guang-Zhong Yang

Robot assisted endoluminal intervention is an emerging tool for treating luminal lesions. Vision-based endoluminal navigation, particularly through video-CT registration, is a tangible way of obtaining absolute camera position information. By using pre-operative CT data, accurate endoscope localization can be achieved, without the need of additional tracking hardware intraoperatively. However, aligning preoperative CT with intraoperative domain remains a challenge. Although approaches such as style transfer have been explored, patient-specific textures and intra-operative artifacts can significantly complicate the task. To overcome these challenges, we propose R2Nav, a robust, real-time test time adaptation method for endoluminal navigation. R2Nav constructs a confidence buffer during the testing phase, refining the model only for frames with high uncertainty. We introduce a registration-augmented model refinement strategy, which enhances both accuracy and efficiency of the system by selecting relevant training samples from the virtual gallery. Additionally, we propose a novel warm-up strategy for the registration encoder during the initial testing phase, enabling the extraction of more robust features when the model is suboptimal. Extensive validation demonstrates that R2Nav outperforms the current state-of-the-art methods, offering significant advantages for real-time, intra-operative endoluminal navigation. Code is at: https://github.com/EndoluminalSurgicalVision-IMR/R2Nav.

Details

ICRA Conference 2025 Conference Paper

Sim2real Within 5 Minutes: Efficient Domain Transfer with Stylized Gaussian Splatting for Endoscopic Images

Junyang Wu
Yun Gu
Guang-Zhong Yang

Robot assisted endoluminal intervention is an emerging technique for both benign and malignant luminal lesions. With vision-based navigation, when combined with pre-operative imaging data as priors, it is possible to recover position and pose of the endoscope without the need of additional sensors. In practice, however, aligning pre-operative and intra-operative domains is complicated by significant texture differences. Although methods such as style transfer can be used to address this issue, they require large datasets from both source and target domains with prolonged training times. This paper proposes an efficient domain transfer method based on stylized Gaussian splatting, only requiring a few of real images (10 images) with very fast training time. Specifically, the transfer process includes two phases. In the first phase, the 3D models reconstructed from CT scans are represented as differential Gaussian point clouds. In the second phase, only color appearance related parameters are optimized to transfer the style and preserve the visual content. A novel structure consistency loss is applied to latent features and depth levels to enhance the stability of the transferred images. Detailed validation was performed to demonstrate the performance advantages of the proposed method compared to that of the current state-of-the-art, highlighting the potential for intra-operative surgical navigation.

Details

JBHI Journal 2025 Journal Article

SLoRD: Structural Low-Rank Descriptors for Shape Consistency in Vertebrae Segmentation

Xin You
Yixin Lou
Minghui Zhang
Jie Yang
Yun Gu

Automatic and precise multi-class vertebrae segmentation from CT images is crucial for various clinical applications. However, due to similar appearances between adjacent vertebrae and the existence of various pathologies, existing single-stage and multi-stage methods suffer from imprecise vertebrae segmentation. Essentially, these methods fail to explicitly impose both contour precision and intra-vertebrae voxel consistency constraints synchronously, resulting in the intra-vertebrae segmentation inconsistency, which refers to multiple label predictions inside a singular vertebra. In this work, we intend to label complete binary masks with sequential indices to address that challenge. Specifically, a contour generation network is proposed based on Structural Low-Rank Descriptors for shape consistency, termed SLoRD. For a structural representation of vertebral contours, we adopt the spherical coordinate system and devise the spherical centroid to calculate contour descriptors. Due to vertebrae’s similar appearances, basic contour descriptors can be acquired offline to restore original contours. Therefore, SLoRD leverages these contour priors and explicit shape constraints to facilitate regressed contour points close to vertebral surfaces. Quantitative and qualitative evaluations on VerSe 2019 and 2020 demonstrate the superior performance of our framework over other single-stage and multi-stage state-of-the-art (SOTA) methods. Further, SLoRD is a plug-and-play framework to refine the segmentation inconsistency existing in coarse predictions from other approaches.

Details DOI

TMLR Journal 2025 Journal Article

Unleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Models

Yulei Qin
Yuncheng Yang
Pengcheng Guo
Gang Li
Hang Shao
Yuchen Shi
Zihan Xu
Yun Gu

Instruction tuning plays a critical role in aligning large language models (LLMs) with human preference. Despite the vast amount of open instruction datasets, naively training a LLM on all existing instructions may not be optimal and practical. To pinpoint the most beneficial datapoints, data assessment and selection methods have been proposed in the fields of natural language processing (NLP) and deep learning. However, under the context of instruction tuning, there still exists a gap in knowledge on what kind of data evaluation metrics can be employed and how they can be integrated into the selection mechanism. To bridge this gap, we present a comprehensive review on existing literature of data assessment and selection especially for instruction tuning of LLMs. We systematically categorize all applicable methods into quality-based, diversity-based, and importance-based ones where a unified, fine-grained taxonomy is structured. For each category, representative methods are elaborated to describe the landscape of relevant research. In addition, comparison between the latest methods is conducted on their officially reported results to provide in-depth discussions on their limitations. Finally, we summarize the open challenges and propose the promosing avenues for future studies. All related contents are available at https://github.com/yuleiqin/fantastic-data-engineering.

PDF Details

JBHI Journal 2024 Journal Article

Towards Connectivity-Aware Pulmonary Airway Segmentation

Minghui Zhang
Yun Gu

Detailed pulmonary airway segmentation is a clinically important task for endobronchial intervention and treatment of peripheral pulmonary lesions. Convolutional Neural Networks (CNNs) are promising for automated analysis of medical imaging, which however performs poorly on airway segmentation. Specifically, breakage of small bronchi distals cannot be effectively eliminated in the prediction results of CNNs, which is detrimental to use as a reference for bronchoscopic-assisted surgery. In this article, we proposed a connectivity-aware segmentation framework to improve the performance of airway segmentation. A Connectivity-Aware Surrogate (CAS) module is first proposed to balance the training progress within-class distribution. Furthermore, a Local-Sensitive Distance (LSD) module is designed to identify the breakage and minimize the variation of the distance map between the prediction and ground-truth. The proposed method is validated with the publically available reference airway segmentation datasets. The detected rate of branch and length on public EXACT'09 and BAS datasets are 82. 1%/79. 6% and 96. 5%/91. 5% respectively, demonstrating the effectiveness of the method in terms of improving the connectedness of the segmentation performance.

Details DOI

JBHI Journal 2023 Journal Article

A Domain Generative Graph Network for EEG-Based Emotion Recognition

Yun Gu
Xinyue Zhong
Cheng Qu
Chuanjun Liu
Bin Chen

Emotion is a human attitude experience and corresponding behavioral response to objective things. Effective emotion recognition is important for the intelligence and humanization of brain-computer interface (BCI). Although deep learning has been widely used in emotion recognition in recent years, emotion recognition based on electroencephalography (EEG) is still a challenging task in practical applications. Herein, we proposed a novel hybrid model that employs generative adversarial networks to generate potential representations of EEG signals while combining graph convolutional neural networks and long short-term memory networks to recognize emotions from EEG signals. Experimental results on DEAP and SEED datasets show that the proposed model achieved the promising emotion classification performance compared with the state-of-the-art methods.

Details DOI

NeurIPS Conference 2023 Conference Paper

CAPro: Webly Supervised Learning with Cross-modality Aligned Prototypes

Yulei Qin
Xingyu Chen
Yunhang Shen
Chaoyou Fu
Yun Gu
Ke Li
Xing Sun
Rongrong Ji

Webly supervised learning has attracted increasing attention for its effectiveness in exploring publicly accessible data at scale without manual annotation. However, most existing methods of learning with web datasets are faced with challenges from label noise, and they have limited assumptions on clean samples under various noise. For instance, web images retrieved with queries of ”tiger cat“ (a cat species) and ”drumstick“ (a musical instrument) are almost dominated by images of tigers and chickens, which exacerbates the challenge of fine-grained visual concept learning. In this case, exploiting both web images and their associated texts is a requisite solution to combat real-world noise. In this paper, we propose Cross-modality Aligned Prototypes (CAPro), a unified prototypical contrastive learning framework to learn visual representations with correct semantics. For one thing, we leverage textual prototypes, which stem from the distinct concept definition of classes, to select clean images by text matching and thus disambiguate the formation of visual prototypes. For another, to handle missing and mismatched noisy texts, we resort to the visual feature space to complete and enhance individual texts and thereafter improve text matching. Such semantically aligned visual prototypes are further polished up with high-quality samples, and engaged in both cluster regularization and noise removal. Besides, we propose collective bootstrapping to encourage smoother and wiser label reference from appearance-similar instances in a manner of dictionary look-up. Extensive experiments on WebVision1k and NUS-WIDE (Web) demonstrate that CAPro well handles realistic noise under both single-label and multi-label scenarios. CAPro achieves new state-of-the-art performance and exhibits robustness to open-set recognition. Codes are available at https: //github. com/yuleiqin/capro.

PDF Details

ICRA Conference 2023 Conference Paper

CDFI: Cross Domain Feature Interaction for Robust Bronchi Lumen Detection

Jiasheng Xu
Tianyi Zhang
Yangqian Wu
Jie Yang 0002
Guang-Zhong Yang
Yun Gu

Endobronchial intervention is increasingly used as a minimally invasive means for the treatment of pulmonary diseases. In order to reduce the difficulty of manipulation in complex airway networks, robust lumen detection is essential for intraoperative guidance. However, these methods are sensitive to visual artifacts which are inevitable during the surgery. In this work, a cross domain feature interaction (CDFI) network is proposed to extract the structural features of lumens, as well as to provide artifact cues to characterize the visual features. To effectively extract the structural and artifact features, the Quadruple Feature Constraints (QFC) module is designed to constrain the intrinsic connections of samples with various imaging-quality. Furthermore, we design a Guided Feature Fusion (GFF) module to supervise the model for adaptive feature fusion based on different types of artifacts. Results show that the features extracted by the proposed method can preserve the structural information of lumen in the presence of large visual variations, bringing much-improved lumen detection accuracy.

Details

JBHI Journal 2023 Journal Article

Contrastive Adversarial Learning for Endomicroscopy Imaging Super-Resolution

Chuyan Zhang
Yun Gu
Guang-Zhong Yang

Endomicroscopy is an emerging imaging modality for real-time optical biopsy. One limitation of existing endomicroscopy based on coherent fibre bundles is that the image resolution is intrinsically limited by the number of fibres that can be practically integrated within the small imaging probe. To improve the image resolution, Super-Resolution (SR) techniques combined with image priors can enhance the clinical utility of endomicroscopy whereas existing SR algorithms suffer from the lack of explicit guidance from ground truth high-resolution (HR) images. In this article, we propose an unsupervised SR pipeline to allow stable offline and kernel-generic learning. Our method takes advantage of both internal statistics and external cross-modality priors. To improve the joint learning process, we present a Sharpness-aware Contrastive Generative Adversarial Network (SCGAN) with two dedicated modules, a sharpness-aware generator and a contrastive-learning discriminator. In the generator, an auxiliary task of sharpness discrimination is formulated to facilitate internal learning by considering the rankings of training instances in various sharpness levels. In the discriminator, we design a contrastive-learning module to mitigate the ill-posed nature of SR tasks via constraints from both positive and negative images. Experiments on multiple datasets demonstrate that SCGAN reduces the performance gap between previous unsupervised approaches and the upper bounds defined in supervised settings by more than 50%, delivering a new state-of-the-art performance score for endomicroscopy super-resolution. Further application on a realistic Voronoi-based pCLE downsampling kernel proves that SCGAN attains PSNR of 35. 851 dB, improving 5. 23 dB compared with the traditional Delaunay interpolation.

Details DOI

AAAI Conference 2023 Conference Paper

FoPro: Few-Shot Guided Robust Webly-Supervised Prototypical Learning

Yulei Qin
Xingyu Chen
Chao Chen
Yunhang Shen
Bo Ren
Yun Gu
Jie Yang
Chunhua Shen

Recently, webly supervised learning (WSL) has been studied to leverage numerous and accessible data from the Internet. Most existing methods focus on learning noise-robust models from web images while neglecting the performance drop caused by the differences between web domain and real-world domain. However, only by tackling the performance gap above can we fully exploit the practical value of web datasets. To this end, we propose a Few-shot guided Prototypical (FoPro) representation learning method, which only needs a few labeled examples from reality and can significantly improve the performance in the real-world domain. Specifically, we initialize each class center with few-shot real-world data as the ``realistic" prototype. Then, the intra-class distance between web instances and ``realistic" prototypes is narrowed by contrastive learning. Finally, we measure image-prototype distance with a learnable metric. Prototypes are polished by adjacent high-quality web images and involved in removing distant out-of-distribution samples. In experiments, FoPro is trained on web datasets with a few real-world examples guided and evaluated on real-world datasets. Our method achieves the state-of-the-art performance on three fine-grained datasets and two large-scale datasets. Compared with existing WSL methods under the same few-shot settings, FoPro still excels in real-world generalization. Code is available at https://github.com/yuleiqin/fopro.

PDF Details DOI

ICRA Conference 2021 Conference Paper

Discriminative Asymmetric Learning for Efficient Surgical Instrument Parsing

Jiaqi Liu
Yu Qiao 0003
Jie Yang 0002
Guang-Zhong Yang
Yun Gu

Semantic segmentation of surgical instruments provides essential priors for autonomous surgery. This task is however challenging since the fine-structure of surgical instruments requires the accurate segmentation of detailed regions in images. As the visual guidance for autonomous surgery, the algorithm should also be real-time and friendly to embedded systems. In this paper, a discriminative asymmetric learning framework is proposed to balance the efficiency and effectiveness of surgical instrument segmentation. Two convolutional neural networks with specific designs are deployed to extract the detail and semantic features of instruments. To reduce the redundancy of visual representation, the aggregator-discriminator mechanism is proposed to distinguish the features learned from different levels. Experiments demonstrate that the proposed method contributes to competitive segmentation accuracy and a higher efficiency compared to existing methods.

Details

JBHI Journal 2019 Journal Article

Densely-Connected Multi-Magnification Hashing for Histopathological Image Retrieval

Yun Gu
Jie Yang

Content-based medical image retrieval is an important computer-aided diagnosis technique providing the clinicians with interpretative references based on visual similarity. In this paper, we focus on the tasks of histopathological image retrieval for breast cancer diagnosis. The densely-connected multi-magnification (DCMMH) framework is proposed to generate the discriminative binary codes by exploiting the histopathological images with multiple magnification factors. The low-magnification images are boosted by the accumulated similarity based on local patches that also regularize the feature learning of high-magnification images. In order to fully utilize the information across different magnification levels, a densely-connected architecture is finally deployed for high-low magnification pairs of datasets. Experiments on BreakHis dataset demonstrate that, DCMMH outperforms the previous hashing methods on histopathological image retrieval.

Details DOI

IROS Conference 2018 Conference Paper

Cross-Scene Suture Thread Parsing for Robot Assisted Anastomosis based on Joint Feature Learning

Yun Gu
Yang Hu 0011
Lin Zhang 0021
Jie Yang 0002
Guang-Zhong Yang

Task autonomy is an important consideration for the development of future surgical robots. For robot-assisted anastomosis, suture thread detection is a prerequisite for subsequent robot manipulation. Previous works on automatic thread detection are focused on the learning of the models with specific surgical settings that are poorly generalisable to generic settings. In this paper, we propose a joint feature learning framework that caters for the foreground and background adaptation for surgical suture thread detection. The proposed method is developed in the context of semi-supervised and unsupervised domain adaptation, leveraging the labelled training data from the source domain to learn the detection model for unlabelled or partially labelled target domain, which can also be from different types of threads or organs. Based on adversarial learning, we further preserve the semantic identity and introduce curriculum adaptation to generate synthetic data. Experiments on four domain adaptation tasks for suture thread detection demonstrate the strength of the proposed method being able to generate good quality synthetic data and transfer between specific domains with limited or even no labelled data of the target domain.

Details

ICRA Conference 2018 Conference Paper

Multi-Stage Suture Detection for Robot Assisted Anastomosis Based on Deep Learning

Yang Hu 0011
Yun Gu
Jie Yang 0002
Guang-Zhong Yang

The technique of robust suture detection is vital in many applications including trainee suturing skill evaluation, suture augmentation in robotic-assisted surgery and suture recognition for automatic suturing. Due to the complicated environment of surgery, the detection of a suture is challenged by high deformation and frequent occlusion. In this paper, we propose a deep multi-stage framework for suture detection. The fully convolutional neural networks are firstly used to predict a gradient map which not only serves as a segmentation mask, but also provides useful structure information for the following thread centerline reconstruction. An overlapping map is also predicted to improve the quality of the gradient map in self-intersection area. Based on the gradient map, multiple segments of the thread are extracted and linked to form the whole thread using a curvilinear structure detector. Experiments on two types of threads demonstrate that the proposed method is able to detect the thread with human level performance when the thread is no occlusion or under finite self-intersection.

Details