Author name cluster

Yuming Chen

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

3 papers

2 author rows

NeurIPS Conference 2025 Conference Paper

OmniSegmentor: A Flexible Multi-Modal Learning Framework for Semantic Segmentation

Bo-Wen Yin
Jiao-Long Cao
Xuying Zhang
Yuming Chen
Ming-Ming Cheng
Qibin Hou

Recent research on representation learning has proved the merits of multi-modal clues for robust semantic segmentation. Nevertheless, a flexible pretrain-and-finetune pipeline for multiple visual modalities remains unexplored. In this paper, we propose a novel multi-modal learning framework, termed OmniSegmentor. It has two key innovations: 1) Based on ImageNet, we assemble a large-scale dataset for multi-modal pretraining, called OmniSegmentor, which contains five popular visual modalities; 2) We provide an efficient pretraining manner to endow the model with the capacity to encode different modality information in the OmniSegmentor. For the first time, we introduce a universal multi-modal pretraining framework that consistently amplifies the model's perceptual capabilities across various scenarios, regardless of the arbitrary combination of the involved modalities. Remarkably, our OmniSegmentor achieves new state-of-the-art records on a wide range of multi-modal semantic segmentation datasets, including NYU Depthv2, EventScape, MFNet, DeLiVER, SUNRGBD, and KITTI-360. Data, model checkpoints, and source code will be made publicly available: https: //github. com/VCIP-RGBD/DFormer.

PDF Details

ICLR Conference 2025 Conference Paper

Re-Aligning Language to Visual Objects with an Agentic Workflow

Yuming Chen
Jiangyan Feng
Haodong Zhang
Lijun Gong
Feng Zhu 0006
Rui Zhao 0001
Qibin Hou
Ming-Ming Cheng

Language-based object detection (LOD) aims to align visual objects with language expressions. A large amount of paired data is utilized to improve LOD model generalizations. During the training process, recent studies leverage vision-language models (VLMs) to automatically generate human-like expressions for visual objects, facilitating training data scaling up. In this process, we observe that VLM hallucinations bring inaccurate object descriptions (e.g., object name, color, and shape) to deteriorate VL alignment quality. To reduce VLM hallucinations, we propose an agentic workflow controlled by an LLM to re-align language to visual objects via adaptively adjusting image and text prompts. We name this workflow Real-LOD, which includes planning, tool use, and reflection steps. Given an image with detected objects and VLM raw language expressions, Real-LOD reasons its state automatically and arranges action based on our neural symbolic designs (i.e., planning). The action will adaptively adjust the image and text prompts and send them to VLMs for object re-description (i.e., tool use). Then, we use another LLM to analyze these refined expressions for feedback (i.e., reflection). These steps are conducted in a cyclic form to gradually improve language descriptions for re-aligning to visual objects. We construct a dataset that contains a tiny amount of 0.18M images with re-aligned language expression and train a prevalent LOD model to surpass existing LOD methods by around 50% on the standard benchmarks. Our Real-LOD workflow, with automatic VL refinement, reveals a potential to preserve data quality along with scaling up data quantity, which further improves LOD performance from a data-alignment perspective.

Details

ECAI Conference 2025 Conference Paper

Spatial Relation Extraction on Null-Role and Multi-Mention Relation Templates

Yuming Chen
Peifeng Li
Qiaoming Zhu

Spatial relation extraction is to identify the spatial positions and dynamic relations between geographical entities within a given text. Previous research has been hindered by two significant challenges: extracting null-role and multi-mention relations. Null-role relations frequently lack the requisite spatial elements to fully express their semantics, while multi-mention relations may confuse models due to the presence of non-coreferent repeated elements within a sentence. To address the two above issues, we propose a prompt-based method that integrates null-role-specific templates and element markers to enhance the encoding process and subsequently improve the model’s discriminative ability. The experimental results on the SpaceEval dataset demonstrate that our proposed model outperforms the SOTA baselines significantly, particularly in terms of its ability to handle null-role and multi-mention relations.

Details