Author name cluster

Meng Wei

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

6 papers

1 author row

AAAI Conference 2026 Conference Paper

Where It Moves, It Matters: Referring Surgical Instrument Segmentation via Motion

Meng Wei
Kun Yuan
Shi Li
Yue Zhou
Long Bai
Nassir Navab
Hongliang Ren
Hong Joo Lee

Enabling intuitive, language-driven interaction with surgical scenes is a critical step toward intelligent operating rooms and autonomous surgical robotic assistance. However, the task of referring segmentation, localizing surgical instruments based on natural language descriptions, remains underexplored in surgical videos, with existing approaches struggling to generalize due to reliance on static visual cues and predefined instrument names. In this work, we introduce SurgRef, a novel motion-guided framework that grounds free-form language expressions in instrument motion, capturing how tools move and interact across time, rather than what they look like. This allows models to understand and segment instruments even under occlusion, ambiguity, or unfamiliar terminology. To train and evaluate SurgRef, we present Ref-IMotion, a diverse, multi-institutional video dataset with dense spatiotemporal masks and rich motion-centric expressions. SurgRef achieves state-of-the-art accuracy and generalization across surgical procedures, setting a new benchmark for robust, language-driven surgical video segmentation.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

Normal-GS: 3D Gaussian Splatting with Normal-Involved Rendering

Meng Wei
Qianyi Wu
Jianmin Zheng
Hamid Rezatofighi
Jianfei Cai

Rendering and reconstruction are long-standing topics in computer vision and graphics. Achieving both high rendering quality and accurate geometry is a challenge. Recent advancements in 3D Gaussian Splatting (3DGS) have enabled high-fidelity novel view synthesis at real-time speeds. However, the noisy and discrete nature of 3D Gaussian primitives hinders accurate surface estimation. Previous attempts to regularize 3D Gaussian normals often degrade rendering quality due to the fundamental disconnect between normal vectors and the rendering pipeline in 3DGS-based methods. Therefore, we introduce Normal-GS, a novel approach that integrates normal vectors into the 3DGS rendering pipeline. The core idea is to model the interaction between normals and incident lighting using the physically-based rendering equation. Our approach re-parameterizes surface colors as the product of normals and a designed Integrated Directional Illumination Vector (IDIV). To optimize memory usage and simplify optimization, we employ an anchor-based 3DGS to implicitly encode locally-shared IDIVs. Additionally, Normal-GS leverages optimized normals and Integrated Directional Encoding (IDE) to accurately model specular effects, enhancing both rendering quality and surface normal precision. Extensive experiments demonstrate that Normal-GS achieves near state-of-the-art visual quality while obtaining accurate surface normals and preserving real-time rendering performance.

PDF Details DOI

NeurIPS Conference 2023 Conference Paper

OV-PARTS: Towards Open-Vocabulary Part Segmentation

Meng Wei
Xiaoyu Yue
Wenwei Zhang
Shu Kong
Xihui Liu
Jiangmiao Pang

Segmenting and recognizing diverse object parts is a crucial ability in applications spanning various computer vision and robotic tasks. While significant progress has been made in object-level Open-Vocabulary Semantic Segmentation (OVSS), i. e. , segmenting objects with arbitrary text, the corresponding part-level research poses additional challenges. Firstly, part segmentation inherently involves intricate boundaries, while limited annotated data compounds the challenge. Secondly, part segmentation introduces an open granularity challenge due to the diverse and often ambiguous definitions of parts in the open world. Furthermore, the large-scale vision and language models, which play a key role in the open vocabulary setting, struggle to recognize parts as effectively as objects. To comprehensively investigate and tackle these challenges, we propose an Open-Vocabulary Part Segmentation (OV-PARTS) benchmark. OV-PARTS includes refined versions of two publicly available datasets: Pascal-Part-116 and ADE20K-Part-234. And it covers three specific tasks: Generalized Zero-Shot Part Segmentation, Cross-Dataset Part Segmentation, and Few-Shot Part Segmentation, providing insights into analogical reasoning, open granularity and few-shot adapting abilities of models. Moreover, we analyze and adapt two prevailing paradigms of existing object-level OVSS methods for OV-PARTS. Extensive experimental analysis is conducted to inspire future research in leveraging foundational models for OV-PARTS. The code and dataset are available at https: //github. com/kellyiss/OV_PARTS.

PDF Details

NeurIPS Conference 2023 Conference Paper

Text Promptable Surgical Instrument Segmentation with Vision-Language Models

Zijian Zhou
Oluwatosin Alabi
Meng Wei
Tom Vercauteren
Miaojing Shi

In this paper, we propose a novel text promptable surgical instrument segmentation approach to overcome challenges associated with diversity and differentiation of surgical instruments in minimally invasive surgeries. We redefine the task as text promptable, thereby enabling a more nuanced comprehension of surgical instruments and adaptability to new instrument types. Inspired by recent advancements in vision-language models, we leverage pretrained image and text encoders as our model backbone and design a text promptable mask decoder consisting of attention- and convolution-based prompting schemes for surgical instrument segmentation prediction. Our model leverages multiple text prompts for each surgical instrument through a new mixture of prompts mechanism, resulting in enhanced segmentation performance. Additionally, we introduce a hard instrument area reinforcement module to improve image feature comprehension and segmentation precision. Extensive experiments on several surgical instrument segmentation datasets demonstrate our model's superior performance and promising generalization capability. To our knowledge, this is the first implementation of a promptable approach to surgical instrument segmentation, offering significant potential for practical application in the field of robotic-assisted surgery. Code is available at https: //github. com/franciszzj/TP-SIS.

PDF Details

AAAI Conference 2022 Conference Paper

Rethinking the Two-Stage Framework for Grounded Situation Recognition

Meng Wei
Long Chen
Wei Ji
Xiaoyu Yue
Tat-Seng Chua

Grounded Situation Recognition (GSR), i. e. , recognizing the salient activity (or verb) category in an image (e. g. , buying) and detecting all corresponding semantic roles (e. g. , agent and goods), is an essential step towards “human-like” event understanding. Since each verb is associated with a specific set of semantic roles, all existing GSR methods resort to a two-stage framework: predicting the verb in the first stage and detecting the semantic roles in the second stage. However, there are obvious drawbacks in both stages: 1) The widely-used cross-entropy (XE) loss for object recognition is insufficient in verb classification due to the large intraclass variation and high inter-class similarity among daily activities. 2) All semantic roles are detected in an autoregressive manner, which fails to model the complex semantic relations between different roles. To this end, we propose a novel SituFormer for GSR which consists of a Coarse-to- Fine Verb Model (CFVM) and a Transformer-based Noun Model (TNM). CFVM is a two-step verb prediction model: a coarse-grained model trained with XE loss first proposes a set of verb candidates, and then a fine-grained model trained with triplet loss re-ranks these candidates with enhanced verb features (not only separable but also discriminative). TNM is a transformer-based semantic role detection model, which detects all roles parallelly. Owing to the global relation modeling ability and flexibility of the transformer decoder, TNM can fully explore the statistical dependency of the roles. Extensive validations on the challenging SWiG benchmark show that SituFormer achieves a new state-of-the-art performance with significant gains under various metrics. Code is available at https: //github. com/kellyiss/SituFormer.

PDF Details

JBHI Journal 2021 Journal Article

Deep Learning for Hemorrhagic Lesion Detection and Segmentation on Brain CT Images

Lu Li
Meng Wei
Bo Liu
Kunakorn Atchaneeyasakul
Fugen Zhou
Zehao Pan
Shimran A. Kumar
Jason Y. Zhang

Stroke is an acute cerebral vascular disease that is likely to cause long-term disabilities and death. Immediate emergency care with accurate diagnosis of computed tomographic (CT) images is crucial for dealing with a hemorrhagic stroke. However, due to the high variability of a stroke's location, contrast, and shape, it is challenging and time-consuming even for experienced radiologists to locate them. In this paper, we propose a U-net based deep learning framework to automatically detect and segment hemorrhage strokes in CT brain images. The input of the network is built by concatenating the flipped image with the original CT slice which introduces symmetry constraints of the brain images into the proposed model. This enhances the contrast between hemorrhagic area and normal brain tissue. Various Deep Learning topologies are compared by varying the layers, batch normalization, dilation rates, and pre-train models. This could increase the respective filed and preserves more information on lesion characteristics. Besides, the adversarial training is also adopted in the proposed network to improve the accuracy of the segmentation. The proposed model is trained and evaluated on two different datasets, which achieve the competitive performance with human experts with the highest location accuracy 0. 9859 for detection, 0. 8033 Dice score, and 0. 6919 IoU for segmentation. The results demonstrate the effectiveness, robustness, and advantages of the proposed deep learning model in automatically hemorrhage lesion diagnosis, which make it possible to be a clinical decision support tool in stroke diagnosis.

Details DOI