Author name cluster

Zhenyu Tang

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

11 papers

2 author rows

AAAI Conference 2026 Conference Paper

NeuralGS: Bridging Neural Fields and 3D Gaussian Splatting for Compact 3D Representations

Zhenyu Tang
Chaoran Feng
Xinhua Cheng
Wangbo Yu
Junwu Zhang
Yuan Liu
Xiao-Xiao Long
Wenping Wang

3D Gaussian Splatting (3DGS) achieves impressive quality and rendering speed, but with millions of 3D Gaussians and significant storage and transmission costs. In this paper, we aim to develop a simple yet effective method called NeuralGS that compresses the original 3DGS into a compact representation. Our observation is that neural fields like NeRF can represent complex 3D scenes with Multi-Layer Perceptron (MLP) neural networks using only a few megabytes. Thus, NeuralGS effectively adopts the neural field representation to encode the attributes of 3D Gaussians with MLPs, only requiring a small storage size even for a large-scale scene. To achieve this, we adopt a clustering strategy and fit the Gaussians within each cluster using different tiny MLPs, based on importance scores of Gaussians as fitting weights. We experiment on multiple datasets, achieving a 91$\times$ average model size reduction without harming the visual quality.

PDF Details DOI

AAAI Conference 2026 Conference Paper

Next Patch Prediction for AutoRegressive Visual Generation

Yatian Pang
Peng Jin
Shuo Yang
Bin Zhu
Bin Lin
Chaoran Feng
Zhenyu Tang
Liuhan Chen

Autoregressive models, built based on the Next Token Prediction (NTP) paradigm, show great potential in developing a unified framework that integrates both language and vision tasks. Pioneering works introduce NTP to autoregressive visual generation tasks. In this work, we rethink the NTP for autoregressive image generation and extend it to a novel Next Patch Prediction (NPP) paradigm. Our key idea is to group and aggregate image tokens into patch tokens with higher information density. By using patch tokens as a more compact input sequence, the autoregressive model is trained to predict the next patch, significantly reducing computational costs. To further exploit the natural hierarchical structure of image data, we propose a multi-scale coarse-to-fine patch grouping strategy. With this strategy, the training process begins with a large patch size and ends with vanilla NTP where the patch size is 1x1, thus maintaining the original inference process without modifications. Extensive experiments across a diverse range of model sizes demonstrate that NPP could reduce the training cost to around 0.6 times while improving image generation quality by up to 1.0 FID score on the ImageNet 256x256 generation benchmark. Notably, our method retains the original autoregressive model architecture without introducing additional trainable parameters or specifically designing a custom image tokenizer, offering a flexible and plug-and-play solution for enhancing autoregressive visual generation.

PDF Details DOI

AAAI Conference 2025 Conference Paper

AE-NeRF: Augmenting Event-Based Neural Radiance Fields for Non-ideal Conditions and Larger Scenes

Chaoran Feng
Wangbo Yu
Xinhua Cheng
Zhenyu Tang
Junwu Zhang
Li Yuan
Yonghong Tian

Compared to frame-based methods, computational neuromorphic imaging using event cameras offers significant advantages, such as minimal motion blur, enhanced temporal resolution, and high dynamic range. The multi-view consistency of Neural Radiance Fields combined with the unique benefits of event cameras, has spurred recent research into reconstructing NeRF from data captured by moving event cameras. While showing impressive performance, existing methods rely on ideal conditions with the availability of uniform and high-quality event sequences and accurate camera poses, and mainly focus on the object level reconstruction, thus limiting their practical applications. In this work, we propose AE-NeRF to address the challenges of learning event-based NeRF from non-ideal conditions, including non-uniform event sequences, noisy poses, and various scales of scenes. Our method exploits the density of event streams and jointly learn a pose correction module with an event-based NeRF (e-NeRF) framework for robust 3D reconstruction from inaccurate camera poses. To generalize to larger scenes, we propose hierarchical event distillation with a proposal e-NeRF network and a vanilla e-NeRF network to resample and refine the reconstruction process. We further propose an event reconstruction loss and a temporal loss to improve the view consistency of the reconstructed scene. We established a comprehensive benchmark that includes large-scale scenes to simulate practical non-ideal conditions, incorporating both synthetic and challenging real-world event datasets. The experimental results show that our method achieves a new state-of-the-art in event-based 3D reconstruction.

PDF Details DOI

AAAI Conference 2025 Conference Paper

Cycle3D: High-quality and Consistent Image-to-3D Generation via Generation-Reconstruction Cycle

Zhenyu Tang
Junwu Zhang
Xinhua Cheng
Wangbo Yu
Chaoran Feng
Yatian Pang
Bin Lin
Li Yuan

Recent 3D large reconstruction models typically employ a two-stage process, including first generate multi-view images by a multi-view diffusion model, and then utilize a feed-forward model to reconstruct images to 3D content. However, multi-view diffusion models often produce low-quality and inconsistent images, adversely affecting the quality of the final 3D reconstruction. To address this issue, we propose a unified 3D generation framework called Cycle3D, which cyclically utilizes a 2D diffusion-based generation module and a feed-forward 3D reconstruction module during the multi-step diffusion process. Concretely, 2D diffusion model is applied for generating high-quality texture, and the reconstruction model guarantees multi-view consistency. Moreover, 2D diffusion model can further control the generated content and inject reference-view information for unseen views, thereby enhancing the diversity and texture consistency of 3D generation during the denoising process. Extensive experiments demonstrate the superior ability of our method to create 3D content with high-quality and consistency compared with state-of-the-art baselines.

PDF Details DOI

EAAI Journal 2025 Journal Article

Deep learning-based time series forecasting for bearing remaining useful life: Recent advances, hybrid architectures, and targeted enhancements

Zhenyu Tang
Dalian Yang
Linrong Tan
Liying Zeng

This paper presents a systematic review of deep learning(DL)-based time series forecasting hybrid model for bearing RUL prediction by evaluating these studies from 2020 to 2025, addressing the gap in analyzing cross-model collaboration strategies. Firstly, the general workflow and modeling framework for remaining useful life (RUL) prediction were introduced. Secondly, a comparative analysis of time dependence capture mechanisms -including Recurrent Neural Networks(RNN) and their variants and Temporal Convolutional Network(TCN), and Transformer-was conducted. The findings reveal a shift from local sequential modeling to globally contextualized frameworks, enhanced by self-supervised learning and multimodal fusion. These developments underscore the importance of aligning model selection with specific task objectives and balancing multiple dimensions to achieve optimal performance in complex industrial scenarios. Thirdly, to address common challenges in RUL prediction, such as noisy data, sparse features, high model complexity, and limited interpretability, this paper explores performance optimization strategies from the perspectives of data quality, architectural design, and decision transparency. Finally, a comprehensive review was made of the current challenges: unclear applicability of attention, limited interpretability, deployment difficulties under resource constraints, difficulties in real-time prediction, and weak generalization in a data-scarce environment. To address these issues, future research should explore more trustworthy attention evaluation, interpretable architectures, lightweight design, continuous iteration for real-time prediction, and leveraging embedding other models to complement the lack of generalization. Overall, this work provides a structured perspective to support the development of the next generation of intelligent prediction systems.

Details DOI

NeurIPS Conference 2025 Conference Paper

GS2E: Gaussian Splatting is an Effective Data Generator for Event Stream Generation

Yuchen Li
Chaoran Feng
Zhenyu Tang
Kaiyuan Deng
Wangbo Yu
Yonghong Tian
Li Yuan

We introduce GS2E (Gaussian Splatting to Event Generation), a large-scale synthetic event dataset designed for high-fidelity event vision tasks, captured from real-world sparse multi-view RGB images. Existing event datasets are often synthesized from dense RGB videos, which typically suffer from limited viewpoint diversity and geometric inconsistency, or rely on expensive, hard-to-scale hardware setups. GS2E addresses these limitations by first reconstructing photorealistic static scenes using 3D Gaussian Splatting, followed by a novel, physically-informed event simulation pipeline. This pipeline integrates adaptive trajectory interpolation with physically-consistent event contrast threshold modeling. As a result, it generates temporally dense and geometrically consistent event streams under diverse motion and lighting conditions, while maintaining strong alignment with the underlying scene structure. Experimental results on event-based 3D reconstruction highlight GS2E’s superior generalization capabilities and its practical value as a benchmark for advancing event vision research.

PDF Details

JBHI Journal 2025 Journal Article

Pre-Operative Overall Survival Prediction of Diffuse Glioma Enhanced by Longitudinal Data

Zhenyu Tang
Jiannan Li
Jingliang Cheng
Zhi-Cheng Li
Zhenyu Zhang
Jing Yan

Many pre-operative overall survival (OS) prediction methods have been proposed to assist personalized treatment of diffuse glioma for better prognosis. Most of them utilize pre-operative data, while post-operative data, which contains essential prognosis-related information (e. g. , surgical outcomes and lesion evolution) is neglected, hindering prediction accuracy. However, incorporating post-operative data could make OS prediction inapplicable at pre-operative stage, affecting clinical utility. To address this contradiction, in this paper, we propose an effective framework that leverages longitudinal data (pre- and post-operative data) to enhance pre-operative OS prediction. Specifically, two OS prediction networks are built in a knowledge distillation framework. One is the teacher network trained with longitudinal data, and the other is the student network relying solely on pre-operative data. Distillation of deep features is conducted to align the performance of the student network with that of the teacher network. Moreover, mass effect and its distillation are adopted to incorporate lesion evolution information, further enhancing prediction performance. Based on our framework, the student network can leverage essential post-operative information without compromising its applicability at pre-operative stage. Experiments on both in-house and public datasets demonstrate that the student network outperforms all state-of-the-art methods under evaluation with statistical significance. Further ablation study reveals that distillation of mass effect and deep features play positive roles in OS prediction. Moreover, new prognosis-related factors are discovered by comparing the student network with and without distillation.

Details DOI

ICLR Conference 2025 Conference Paper

Vevo: Controllable Zero-Shot Voice Imitation with Self-Supervised Disentanglement

Xueyao Zhang
Xiaohui Zhang
Kainan Peng
Zhenyu Tang
Vimal Manohar
Yingru Liu
Jeff Hwang
Dangna Li

The imitation of voice, targeted on specific speech attributes such as timbre and speaking style, is crucial in speech generation. However, existing methods rely heavily on annotated data, and struggle with effectively disentangling timbre and style, leading to challenges in achieving controllable generation, especially in zero-shot scenarios. To address these issues, we propose Vevo, a versatile zero-shot voice imitation framework with controllable timbre and style. Vevo operates in two core stages: (1) Content-Style Modeling: Given either text or speech's content tokens as input, we utilize an autoregressive transformer to generate the content-style tokens, which is prompted by a style reference; (2) Acoustic Modeling: Given the content-style tokens as input, we employ a flow-matching transformer to produce acoustic representations, which is prompted by a timbre reference. To obtain the content and content-style tokens of speech, we design a fully self-supervised approach that progressively decouples the timbre, style, and linguistic content of speech. Specifically, we adopt VQ-VAE as the tokenizer for the continuous hidden features of HuBERT. We treat the vocabulary size of the VQ-VAE codebook as the information bottleneck, and adjust it carefully to obtain the disentangled speech representations. Solely self-supervised trained on 60K hours of audiobook speech data, without any fine-tuning on style-specific corpora, Vevo matches or surpasses existing methods in accent and emotion conversion tasks. Additionally, Vevo’s effectiveness in zero-shot voice conversion and text-to-speech tasks further demonstrates its strong generalization and versatility. Audio samples are available at https://versavoice.github.io/.

Details

JBHI Journal 2024 Journal Article

A New Multi-Atlas Based Deep Learning Segmentation Framework With Differentiable Atlas Feature Warping

Huabing Liu
Dong Nie
Jian Yang
Jinda Wang
Zhenyu Tang

Deep learning based multi-atlas segmentation (DL-MA) has achieved the state-of-the-art performance in many medical image segmentation tasks, e. g. , brain parcellation. In DL-MA methods, atlas-target correspondence is the key for accurate segmentation. In most existing DL-MA methods, such correspondence is usually established using traditional or deep learning based registration methods at image level with no further feature level adaption. This could cause possible atlas-target feature inconsistency. As a result, the information from atlases often has limited positive and even counteractive impact on the final segmentation results. To tackle this issue, in this paper, we propose a new DL-MA framework, where a novel differentiable atlas feature warping module with a new smooth regularization term is presented to establish feature level atlas-target correspondence. Comparing with the existing DL-MA methods, in our framework, atlas features containing anatomical prior knowledge are more relevant to the target image feature, leading the final segmentation results to a high accuracy level. We evaluate our framework in the context of brain parcellation using two public MR brain image datasets: LPBA40 and NIREP-NA0. The experimental results demonstrate that our framework outperforms both traditional multi-atlas segmentation (MAS) and state-of-the-art DL-MA methods with statistical significance. Further ablation studies confirm the effectiveness of the proposed differentiable atlas feature warping module.

Details DOI

NeurIPS Conference 2024 Conference Paper

ShareGPT4Video: Improving Video Understanding and Generation with Better Captions

Lin Chen
Xilin Wei
Jinsong Li
Xiaoyi Dong
Pan Zhang
Yuhang Zang
Zehui Chen
Haodong Duan

We present the ShareGPT4Video series, aiming to facilitate the video understanding of large video-language models (LVLMs) and the video generation of text-to-video models (T2VMs) via dense and precise captions. The series comprises: 1) ShareGPT4Video, 40K GPT4V annotated dense captions of videos with various lengths and sources, developed through carefully designed data filtering and annotating strategy. 2) ShareCaptioner-Video, an efficient and capable captioning model for arbitrary videos, with 4. 8M high-quality aesthetic videos annotated by it. 3) ShareGPT4Video-8B, a simple yet superb LVLM that reached SOTA performance on three advancing video benchmarks. To achieve this, taking aside the non-scalable costly human annotators, we find using GPT4V to caption video with a naive multi-frame or frame-concatenation input strategy leads to less detailed and sometimes temporal-confused results. We argue the challenge of designing a high-quality video captioning strategy lies in three aspects: 1) Inter-frame precise temporal change understanding. 2) Intra-frame detailed content description. 3) Frame-number scalability for arbitrary-length videos. To this end, we meticulously designed a differential video captioning strategy, which is stable, scalable, and efficient for generating captions for videos with arbitrary resolution, aspect ratios, and length. Based on it, we construct ShareGPT4Video, which contains 40K high-quality videos spanning a wide range of categories, and the resulting captions encompass rich world knowledge, object attributes, camera movements, and crucially, detailed and precise temporal descriptions of events. Based on ShareGPT4Video, we further develop ShareCaptioner-Video, a superior captioner capable of efficiently generating high-quality captions for arbitrary videos. We annotated 4. 8M aesthetically appealing videos by it and verified their effectiveness on a 10-second text2video generation task. For video understanding, we verified the effectiveness of ShareGPT4Video on several current LVLM architectures and presented our superb new LVLM ShareGPT4Video-8B. All the models, strategies, and annotations will be open-sourced and we hope this project can serve as a pivotal resource for advancing both the LVLMs and T2VMs community.

PDF Details DOI

IJCAI Conference 2021 Conference Paper

Point-based Acoustic Scattering for Interactive Sound Propagation via Surface Encoding

Hsien-Yu Meng
Zhenyu Tang
Dinesh Manocha

We present a novel geometric deep learning method to compute the acoustic scattering properties of geometric objects. Our learning algorithm uses a point cloud representation of objects to compute the scattering properties and integrates them with ray tracing for interactive sound propagation in dynamic scenes. We use discrete Laplacian-based surface encoders and approximate the neighborhood of each point using a shared multi-layer perceptron. We show that our formulation is permutation invariant and present a neural network that computes the scattering function using spherical harmonics. Our approach can handle objects with arbitrary topologies and deforming models, and takes less than 1ms per object on a commodity GPU. We have analyzed the accuracy and perform validation on thousands of unseen 3D objects and highlight the benefits over other point-based geometric deep learning methods. To the best of our knowledge, this is the first real-time learning algorithm that can approximate the acoustic scattering properties of arbitrary objects with high accuracy.

PDF Details DOI