Author name cluster

Yang Fu

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

13 papers

2 author rows

EAAI Journal 2026 Journal Article

Research on cable terminal interface defect state detection based on electric field characteristics and multi-core improved support vector machine

Yujing Tang
Yang Fu
Qin Cai
Jieping Wu
Qi Wang
Guoqiang Gao

As key equipment for high-speed rail power transmission and the connection of high-voltage systems, the cable terminals are crucial to ensuring the stable operation of the railway system. However, the existing detection methods for cable terminals are easily affected by on-site noise and have low detection accuracy. Therefore, this paper proposes a method for detecting interface defect status of high-speed cable terminals based on the electric field strength feature set and multi-kernel support vector machine (MK-SVM). Firstly, a spatial electric field detection platform was built to extract the electric field intensity of the prefabricated defective cable terminals of different lengths. Secondly, the optimization of the characteristic parameters of electric field strength of defective cable terminals was realized based on the Pearson coefficient method. In order to improve the recognition effect and model generalization ability, a MK-SVM combining linear kernel function and radial basis kernel function was proposed. Finally, a comparative study was conducted on the optimization effects of particle swarm algorithm, firefly algorithm, simulated annealing algorithm and genetic algorithm on MK-SVM. Research has shown that using genetic algorithm for parameter optimization of multi-core SVM has the best performance, with recognition accuracy, average precision, average recall, and average F1 score of 95. 6 %, 96 %, 95. 6 %, and 0. 96, respectively. Compared with the unoptimized SVM, the four feature parameters increased by 8. 9 %, 7. 9 %, 8. 9 %, and 9. 6 %, respectively.

Details DOI

IROS Conference 2025 Conference Paper

Learning Generalizable Feature Fields for Mobile Manipulation

Ri-Zhao Qiu
Yafei Hu
Yuchen Song
Ge Yang
Yang Fu
Jianglong Ye
Jiteng Mu
Ruihan Yang

An open problem in mobile manipulation is how to represent objects and scenes in a unified manner so that robots can use both for navigation and manipulation. The latter requires capturing intricate geometry while understanding fine-grained semantics, whereas the former involves capturing the complexity inherent at an expansive physical scale. In this work, we present GeFF (Generalizable Feature Fields), a scene-level generalizable neural feature field that acts as a unified representation for both navigation and manipulation that performs in real-time. To do so, we treat generative novel view synthesis as a pre-training task, and then align the resulting rich scene priors with natural language via CLIP feature distillation. We demonstrate the effectiveness of this approach by deploying GeFF on a quadrupedal robot equipped with a manipulator. We quantitatively evaluate GeFF’s ability for open-vocabulary object-/part-level manipulation and show that GeFF outperforms point-based baselines in runtime and storage-accuracy trade-offs, with qualitative examples of semantics-aware navigation and articulated object manipulation.

Details

EAAI Journal 2025 Journal Article

Multi-scale dynamic spatio-temporal graph network for anomaly detection of wind turbine main bearing under time-varying conditions

Yang Fu
Jiachen Ma
Deqiang He
Zhenzhen Jin
Hongrui Cao
Bin Yu

Spatio-temporal analysis for Supervisory Control and Data Acquisition (SCADA)-based condition monitoring of wind turbines has attracted increasing attention in recent years. However, many existing methods insufficiently address time-varying operating conditions, limiting their applicability in complex real-world scenarios. This study develops an artificial intelligence (AI) approach for wind turbine condition monitoring: a fault detection method for main bearings based on a multiscale dynamic spatio-temporal graph network (MS-DSTGN). First, a dynamic topology structure is constructed to model sensor relationships, with spatial features extracted using a graph attention network (GAT). Temporal dependencies are captured through convolutional neural networks (CNN) and long short-term memory (LSTM) networks, reflecting short- and long-term operational trends at different time scales. These features are then fused via a multi-head attention mechanism. The design combines per-window dynamic graph construction with attention-based fusion of short- and long-term temporal branches, enabling multi-scale adaptation to time-varying operating regimes beyond standard spatio-temporal formulations. Finally, anomaly detection is performed using an autoencoder trained exclusively on normal samples, enhancing its real-world applicability for wind turbine monitoring. Experimental results on two real-world wind turbine datasets demonstrate that the proposed method consistently outperforms existing approaches in both accuracy and robustness, showing strong potential for practical applications in wind turbine fault detection under time-varying conditions.

Details DOI

NeurIPS Conference 2025 Conference Paper

TalkCuts: A Large-Scale Dataset for Multi-Shot Human Speech Video Generation

Jiaben Chen
Zixin Wang
Ailing Zeng
Yang Fu
Xueyang Yu
Siyuan Cen
Julian Tanke
Yihang Chen

In this work, we present TalkCuts, a large-scale dataset designed to facilitate the study of multi-shot human speech video generation. Unlike existing datasets that focus on single-shot, static viewpoints, TalkCuts offers 164k clips totaling over 500 hours of high-quality 1080P human speech videos with diverse camera shots, including close-up, half-body, and full-body views. The dataset includes detailed textual descriptions, 2D keypoints and 3D SMPL-X motion annotations, covering over 10k identities, enabling multimodal learning and evaluation. As a first attempt to showcase the value of the dataset, we present Orator, an LLM-guided multi-modal generation framework as a simple baseline, where the language model functions as a multi-faceted director, orchestrating detailed specifications for camera transitions, speaker gesticulations, and vocal modulation. This architecture enables the synthesis of coherent long-form videos through our integrated multi-modal video generation module. Extensive experiments in both pose-guided and audio-driven settings show that training on TalkCuts significantly enhances the cinematographic coherence and visual appeal of generated multi-shot speech videos. We believe TalkCuts provides a strong foundation for future work in controllable, multi-shot speech video generation and broader multimodal learning.

PDF Details

ICLR Conference 2024 Conference Paper

3D Reconstruction with Generalizable Neural Fields using Scene Priors

Yang Fu
Shalini De Mello
Xueting Li
Amey Kulkarni
Jan Kautz
Xiaolong Wang 0004
Sifei Liu

High-fidelity 3D scene reconstruction has been substantially advanced by recent progress in neural fields. However, most existing methods train a separate network from scratch for each individual scene. This is not scalable, inefficient, and unable to yield good results given limited views. While learning-based multi-view stereo methods alleviate this issue to some extent, their multi-view setting makes it less flexible to scale up and to broad applications. Instead, we introduce training generalizable Neural Fields incorporating scene Priors (NFPs). The NFP network maps any single-view RGB-D image into signed distance and radiance values. A complete scene can be reconstructed by merging individual frames in the volumetric space WITHOUT a fusion module, which provides better flexibility. The scene priors can be trained on large-scale datasets, allowing for fast adaptation to the reconstruction of a new scene with fewer views. NFP not only demonstrates SOTA scene reconstruction performance and efficiency, but it also supports single-image novel-view synthesis, which is under-explored in neural fields. More qualitative results are available at: https://oasisyang.github.io/neural-prior.

Details

NeurIPS Conference 2024 Conference Paper

SpatialRGPT: Grounded Spatial Reasoning in Vision-Language Models

An-Chieh Cheng
Hongxu Yin
Yang Fu
Qiushan Guo
Ruihan Yang
Jan Kautz
Xiaolong Wang
Sifei Liu

Vision Language Models (VLMs) have demonstrated remarkable performance in 2D vision and language tasks. However, their ability to reason about spatial arrangements remains limited. In this work, we introduce Spatial Region GPT (SpatialRGPT) to enhance VLMs’ spatial perception and reasoning capabilities. SpatialRGPT advances VLMs’ spatial understanding through two key innovations: (i) a data curation pipeline that enables effective learning of regional representation from 3D scene graphs, and (ii) a flexible ``plugin'' module for integrating depth information into the visual encoder of existing VLMs. During inference, when provided with user-specified region proposals, SpatialRGPT can accurately perceive their relative directions and distances. Additionally, we propose SpatialRGBT-Bench, a benchmark with ground-truth 3D annotations encompassing indoor, outdoor, and simulated environments, for evaluating 3D spatial cognition in Vision-Language Models (VLMs). Our results demonstrate that SpatialRGPT significantly enhances performance in spatial reasoning tasks, both with and without local region prompts. The model also exhibits strong generalization capabilities, effectively reasoning about complex spatial relations and functioning as a region-aware dense reward annotator for robotic tasks. Code, dataset, and benchmark are released at https: //www. anjiecheng. me/SpatialRGPT.

PDF Details DOI

ICML Conference 2023 Conference Paper

MonoNeRF: Learning Generalizable NeRFs from Monocular Videos without Camera Poses

Yang Fu
Ishan Misra
Xiaolong Wang 0004

We propose a generalizable neural radiance fields - MonoNeRF, that can be trained on large-scale monocular videos of moving in static scenes without any ground-truth annotations of depth and camera poses. MonoNeRF follows an Autoencoder-based architecture, where the encoder estimates the monocular depth and the camera pose, and the decoder constructs a Multiplane NeRF representation based on the depth encoder feature, and renders the input frames with the estimated camera. The learning is supervised by the reconstruction error. Once the model is learned, it can be applied to multiple applications including depth estimation, camera pose estimation, and single-image novel view synthesis. More qualitative results are available at: https: //oasisyang. github. io/mononerf.

Details

ICLR Conference 2023 Conference Paper

Self-Supervised Geometric Correspondence for Category-Level 6D Object Pose Estimation in the Wild

Kaifeng Zhang
Yang Fu
Shubhankar Borse
Hong Cai
Fatih Porikli
Xiaolong Wang 0004

While 6D object pose estimation has wide applications across computer vision and robotics, it remains far from being solved due to the lack of annotations. The problem becomes even more challenging when moving to category-level 6D pose, which requires generalization to unseen instances. Current approaches are restricted by leveraging annotations from simulation or collected from humans. In this paper, we overcome this barrier by introducing a self-supervised learning approach trained directly on large-scale real-world object videos for category-level 6D pose estimation in the wild. Our framework reconstructs the canonical 3D shape of an object category and learns dense correspondences between input images and the canonical shape via surface embedding. For training, we propose novel geometrical cycle-consistency losses which construct cycles across 2D-3D spaces, across different instances and different time steps. The learned correspondence can be applied for 6D pose estimation and other downstream tasks such as keypoint transfer. Surprisingly, our method, without any human annotations or simulators, can achieve on-par or even better performance than previous supervised or semi-supervised methods on in-the-wild images. Code and videos are available at https://kywind.github.io/self-pose.

Details

NeurIPS Conference 2022 Conference Paper

Category-Level 6D Object Pose Estimation in the Wild: A Semi-Supervised Learning Approach and A New Dataset

Yang Fu
Xiaolong Wang

6D object pose estimation is one of the fundamental problems in computer vision and robotics research. While a lot of recent efforts have been made on generalizing pose estimation to novel object instances within the same category, namely category-level 6D pose estimation, it is still restricted in constrained environments given the limited number of annotated data. In this paper, we collect Wild6D, a new unlabeled RGBD object video dataset with diverse instances and backgrounds. We utilize this data to generalize category-level 6D object pose estimation in the wild with semi-supervised learning. We propose a new model, called Rendering for Pose estimation network RePoNet), that is jointly trained using the free ground-truths with the synthetic data, and a silhouette matching objective function on the real-world data. Without using any 3D annotations on real data, our method outperforms state-of-the-art methods on the previous dataset and our Wild6D test set (with manual annotations for evaluation) by a large margin. Project page with Wild6D data: \url{https: //oasisyang. github. io/semi-pose/}.

PDF Details

AAAI Conference 2021 Conference Paper

CompFeat: Comprehensive Feature Aggregation for Video Instance Segmentation

Yang Fu
Linjie Yang
Ding Liu
Thomas S. Huang
Humphrey Shi

Video instance segmentation is a complex task in which we need to detect, segment, and track each object for any given video. Previous approaches only utilize single-frame features for the detection, segmentation, and tracking of objects and they suffer in the video scenario due to several distinct challenges such as motion blur and drastic appearance change. To eliminate ambiguities introduced by only using single-frame features, we propose a novel comprehensive feature aggregation approach (CompFeat) to refine features at both framelevel and object-level with temporal and spatial context information. The aggregation process is carefully designed with a new attention mechanism which significantly increases the discriminative power of the learned features. We further improve the tracking capability of our model through a siamese design by incorporating both feature similarities and spatial similarities. Experiments conducted on the YouTube-VIS dataset validate the effectiveness of proposed CompFeat.

PDF Details

AAAI Conference 2019 Conference Paper

Horizontal Pyramid Matching for Person Re-Identification

Yang Fu
Yunchao Wei
Yuqian Zhou
Honghui Shi
Gao Huang
Xinchao Wang
Zhiqiang Yao
Thomas Huang

Despite the remarkable progress in person re-identification (Re-ID), such approaches still suffer from the failure cases where the discriminative body parts are missing. To mitigate this type of failure, we propose a simple yet effective Horizontal Pyramid Matching (HPM) approach to fully exploit various partial information of a given person, so that correct person candidates can be identified even if some key parts are missing. With HPM, we make the following contributions to produce more robust feature representations for the Re-ID task: 1) we learn to classify using partial feature representations at different horizontal pyramid scales, which successfully enhance the discriminative capabilities of various person parts; 2) we exploit average and max pooling strategies to account for person-specific discriminative information in a global-local manner. To validate the effectiveness of our proposed HPM method, extensive experiments are conducted on three popular datasets including Market-1501, DukeMTMC- ReID and CUHK03. Respectively, we achieve mAP scores of 83. 1%, 74. 5% and 59. 7% on these challenging benchmarks, which are the new state-of-the-arts.

PDF Details

AAAI Conference 2019 Conference Paper

STA: Spatial-Temporal Attention for Large-Scale Video-Based Person Re-Identification

Yang Fu
Xiaoyang Wang
Yunchao Wei
Thomas Huang

In this work, we propose a novel Spatial-Temporal Attention (STA) approach to tackle the large-scale person reidentification task in videos. Different from the most existing methods, which simply compute representations of video clips using frame-level aggregation (e. g. average pooling), the proposed STA adopts a more effective way for producing robust clip-level feature representation. Concretely, our STA fully exploits those discriminative parts of one target person in both spatial and temporal dimensions, which results in a 2-D attention score matrix via inter-frame regularization to measure the importances of spatial parts across different frames. Thus, a more robust clip-level feature representation can be generated according to a weighted sum operation guided by the mined 2-D attention score matrix. In this way, the challenging cases for video-based person re-identification such as pose variation and partial occlusion can be well tackled by the STA. We conduct extensive experiments on two large-scale benchmarks, i. e. MARS and DukeMTMC- VideoReID. In particular, the mAP reaches 87. 7% on MARS, which significantly outperforms the state-of-the-arts with a large margin of more than 11. 6%.

PDF Details

EAAI Journal 2017 Journal Article

Machining vibration states monitoring based on image representation using convolutional neural networks

Yang Fu
Yun Zhang
Yuan Gao
Huang Gao
Ting Mao
Huamin Zhou
Dequn Li

Measured signals are usually fed into filters or signal decomposers to extract useful features to assist making identification in state monitoring or fault diagnosis. But what is routinely ignored is that an experienced expert can realize what is happening just by watching the signals presented on the oscilloscope even without the analyzing report. The vision image input and the experience feedback are the two keys in this identification process by the brain. The experience can be easily quantified, like 1 for “good” and 0 for “bad”, and used for identification model construction, while there has been no attempt to use pictured signal as the model input. For closed-loop control system, it is necessary to acquire signal feedback point by point to adjust the system in real time. But for state monitoring and fault diagnosis, the pattern hiding among the signal points is usually more important, which is exactly one of the special fields of image representation to indicate complex interrelationship. Taking machining state monitoring as example, this paper explore the possibility to use the pictured signals as input to construct identification model without traditional feature engineering based on signal analysis. Convolutional neural networks (CNN) is introduced to connect pictured signals to different vibration states with experience feedback. Results validate the proposed method with excellent modeling performance. Time complexity analysis proves this pictured signal image representation based CNN method to be capable to be real-time. Two dimensional image representation is a powerful way to exhibit and fuse information. With high flexibility, the proposed method may be a promising framework for monitoring or fault diagnosis tasks.

Details DOI