Author name cluster

Weining Wang

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

3 papers

1 author row

AAAI Conference 2026 Conference Paper

UniAlignment: Semantic Alignment for Unified Image Generation, Understanding, Manipulation and Perception

Xinyang Song
Libin Wang
Weining Wang
Shaozhen Liu
DanDan Zheng
Jingdong Chen
Qi Li
Zhenan Sun

The remarkable success of diffusion models in text-to-image generation has sparked growing interest in expanding their capabilities to a variety of multi-modal tasks, including image understanding, manipulation, and perception. These tasks require advanced semantic comprehension across both visual and textual modalities, especially in scenarios involving complex semantic instructions. However, existing approaches often rely heavily on vision-language models (VLMs) or modular designs for semantic guidance, leading to fragmented architectures and computational inefficiency. To address these challenges, we propose UniAlignment, a unified multimodal generation framework within a single diffusion transformer. UniAlignment introduces a dual-stream diffusion training strategy that incorporates both intrinsic-modal semantic alignment and cross-modal semantic alignment, thereby enhancing the model's cross-modal consistency and instruction-following robustness. Additionally, we present SemGen-Bench, a new benchmark specifically designed to evaluate multimodal semantic consistency under complex textual instructions. Extensive experiments across multiple tasks and benchmarks demonstrate that UniAlignment outperforms existing baselines, underscoring the significant potential of diffusion models in unified multimodal generation.

PDF Details DOI

NeurIPS Conference 2023 Conference Paper

GLOBER: Coherent Non-autoregressive Video Generation via GLOBal Guided Video DecodER

Mingzhen Sun
Weining Wang
Zihan Qin
Jiahui Sun
Sihan Chen
Jing Liu

Video generation necessitates both global coherence and local realism. This work presents a novel non-autoregressive method GLOBER, which first generates global features to obtain comprehensive global guidance and then synthesizes video frames based on the global features to generate coherent videos. Specifically, we propose a video auto-encoder, where a video encoder encodes videos into global features, and a video decoder, built on a diffusion model, decodes the global features and synthesizes video frames in a non-autoregressive manner. To achieve maximum flexibility, our video decoder perceives temporal information through normalized frame indexes, which enables it to synthesize arbitrary sub video clips with predetermined starting and ending frame indexes. Moreover, a novel adversarial loss is introduced to improve the global coherence and local realism between the synthesized video frames. Finally, we employ a diffusion-based video generator to fit the global features outputted by the video encoder for video generation. Extensive experimental results demonstrate the effectiveness and efficiency of our proposed method, and new state-of-the-art results have been achieved on multiple benchmarks.

PDF Details

EAAI Journal 2023 Journal Article

Ink painting style transfer using asymmetric cycle-consistent GAN

Weining Wang
Yifan Li
Huan Ye
Fenghua Ye
Xiangmin Xu

Chinese ink painting, an artistic and cultural treasure, necessitates automatic generation for its preservation and evolution. We've innovatively observed and validated that the domain information between photographs and ink paintings, is asymmetrical, which has been overlooked by current style transfer algorithms. We propose an innovative generative adversarial network featuring an asymmetric cyclic consistency structure to address this in ink painting style transfer. This structure uses generators of differing capabilities to align with the asymmetry in transformation directions, improving image quality and model optimization speed. Additionally, we introduce two unique loss functions within the network. The salient edge loss intensifies the subject in the real photo and enhances the edge stroke of the drawn subject, a distinct attribute of ink painting. The feature-wise cycle consistency loss is designed to speed up model optimization. We've also built a Chinese bird ink painting dataset to validate effectiveness of the model. Extensive experiments on this and a public dataset demonstrate that our algorithm can comprehensively learn various stylistic features of ink painting, especially regarding brushstroke style, ink diffusion, and detail preservation. Furthermore, the quantitative results indicate our approach achieves superior results in generation quality and model efficiency compared to existing methods. For instance, compared to the most recent style transfer method, our method achieves an average decrease of 9. 44% and 25. 32% for FID and KID metrics across three datasets and reduces training and inference time costs by 46. 81% and 22. 71% respectively.

Details DOI