Author name cluster

Yan Hong

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

12 papers

2 author rows

AAAI Conference 2025 Conference Paper

WildFake: A Large-Scale and Hierarchical Dataset for AI-Generated Images Detection

Yan Hong
Jianming Feng
Haoxing Chen
Jun Lan
Huijia Zhu
Weiqiang Wang
Jianfu Zhang

The development of text-to-image generative models has enabled the creation of images so realistic that distinguishing between AI-generated images and real photos is becoming a challenge. This progress offers new possibilities but also raises concerns over privacy, authenticity, and security. Detecting AI-generated images is crucial to prevent misuse. To assess the generalizability and robustness of AI-generated image detection, we present a large-scale dataset, referred to as WildFake. This dataset features cutting-edge image generators, a wide variety of generator categories, and generators for various applications, organized in a hierarchical framework. WildFake collects fake images from the open-source community, enriching its diversity with a broad range of image classes and image styles. Its design significantly improves the effectiveness of detection algorithms, making it a valuable resource for enhancing AI-generated image detection in practical applications. Our evaluations offer insights into the performance of generative models at various levels, showcasing WildFake's unique hierarchical structure's benefits.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

DomainGallery: Few-shot Domain-driven Image Generation by Attribute-centric Finetuning

Yuxuan Duan
Yan Hong
Bo Zhang
Jun Lan
Huijia Zhu
Weiqiang Wang
Jianfu Zhang
Li Niu

The recent progress in text-to-image models pretrained on large-scale datasets has enabled us to generate various images as long as we provide a text prompt describing what we want. Nevertheless, the availability of these models is still limited when we expect to generate images that fall into a specific domain either hard to describe or just unseen to the models. In this work, we propose DomainGallery, a few-shot domain-driven image generation method which aims at finetuning pretrained Stable Diffusion on few-shot target datasets in an attribute-centric manner. Specifically, DomainGallery features prior attribute erasure, attribute disentanglement, regularization and enhancement. These techniques are tailored to few-shot domain-driven generation in order to solve key issues that previous works have failed to settle. Extensive experiments are given to validate the superior performance of DomainGallery on a variety of domain-driven generation scenarios.

PDF Details DOI

AAAI Conference 2024 Conference Paper

Painterly Image Harmonization by Learning from Painterly Objects

Li Niu
Junyan Cao
Yan Hong
Liqing Zhang

Given a composite image with photographic object and painterly background, painterly image harmonization targets at stylizing the composite object to be compatible with the background. Despite the competitive performance of existing painterly harmonization works, they did not fully leverage the painterly objects in artistic paintings. In this work, we explore learning from painterly objects for painterly image harmonization. In particular, we learn a mapping from background style and object information to object style based on painterly objects in artistic paintings. With the learnt mapping, we can hallucinate the target style of composite object, which is used to harmonize encoder feature maps to produce the harmonized image. Extensive experiments on the benchmark dataset demonstrate the effectiveness of our proposed method.

PDF Details DOI

AAAI Conference 2024 Conference Paper

Progressive Painterly Image Harmonization from Low-Level Styles to High-Level Styles

Li Niu
Yan Hong
Junyan Cao
Liqing Zhang

Painterly image harmonization aims to harmonize a photographic foreground object on the painterly background. Different from previous auto-encoder based harmonization networks, we develop a progressive multi-stage harmonization network, which harmonizes the composite foreground from low-level styles (e.g., color, simple texture) to high-level styles (e.g., complex texture). Our network has better interpretability and harmonization performance. Moreover, we design an early-exit strategy to automatically decide the proper stage to exit, which can skip the unnecessary and even harmful late stages. Extensive experiments on the benchmark dataset demonstrate the effectiveness of our progressive harmonization network.

PDF Details DOI

AAAI Conference 2024 Conference Paper

Shadow Generation with Decomposed Mask Prediction and Attentive Shadow Filling

Xinhao Tao
Junyan Cao
Yan Hong
Li Niu

Image composition refers to inserting a foreground object into a background image to obtain a composite image. In this work, we focus on generating plausible shadows for the inserted foreground object to make the composite image more realistic. To supplement the existing small-scale dataset, we create a large-scale dataset called RdSOBA with rendering techniques. Moreover, we design a two-stage network named DMASNet with decomposed mask prediction and attentive shadow filling. Specifically, in the first stage, we decompose shadow mask prediction into box prediction and shape prediction. In the second stage, we attend to reference background shadow pixels to fill the foreground shadow. Abundant experiments prove that our DMASNet achieves better visual effects and generalizes well to real composite images.

PDF Details DOI

AAAI Conference 2024 Conference Paper

WeditGAN: Few-Shot Image Generation via Latent Space Relocation

Yuxuan Duan
Li Niu
Yan Hong
Liqing Zhang

In few-shot image generation, directly training GAN models on just a handful of images faces the risk of overfitting. A popular solution is to transfer the models pretrained on large source domains to small target ones. In this work, we introduce WeditGAN, which realizes model transfer by editing the intermediate latent codes w in StyleGANs with learned constant offsets (delta w), discovering and constructing target latent spaces via simply relocating the distribution of source latent spaces. The established one-to-one mapping between latent spaces can naturally prevents mode collapse and overfitting. Besides, we also propose variants of WeditGAN to further enhance the relocation process by regularizing the direction or finetuning the intensity of delta w. Experiments on a collection of widely used source/target datasets manifest the capability of WeditGAN in generating realistic and diverse images, which is simple yet highly effective in the research area of few-shot image generation. Codes are available at https://github.com/Ldhlwh/WeditGAN.

PDF Details DOI

AAAI Conference 2023 Conference Paper

Few-Shot Defect Image Generation via Defect-Aware Feature Manipulation

Yuxuan Duan
Yan Hong
Li Niu
Liqing Zhang

The performances of defect inspection have been severely hindered by insufficient defect images in industries, which can be alleviated by generating more samples as data augmentation. We propose the first defect image generation method in the challenging few-shot cases. Given just a handful of defect images and relatively more defect-free ones, our goal is to augment the dataset with new defect images. Our method consists of two training stages. First, we train a data-efficient StyleGAN2 on defect-free images as the backbone. Second, we attach defect-aware residual blocks to the backbone, which learn to produce reasonable defect masks and accordingly manipulate the features within the masked regions by training the added modules on limited defect images. Extensive experiments on MVTec AD dataset not only validate the effectiveness of our method in generating realistic and diverse defect images, but also manifest the benefits it brings to downstream defect inspection tasks. Codes are available at https://github.com/Ldhlwh/DFMGAN.

PDF Details DOI

AAAI Conference 2023 Conference Paper

Memorization Weights for Instance Reweighting in Adversarial Training

Jianfu Zhang
Yan Hong
Qibin Zhao

Adversarial training is an effective way to defend deep neural networks (DNN) against adversarial examples. However, there are atypical samples that are rare and hard to learn, or even hurt DNNs' generalization performance on test data. In this paper, we propose a novel algorithm to reweight the training samples based on self-supervised techniques to mitigate the negative effects of the atypical samples. Specifically, a memory bank is built to record the popular samples as prototypes and calculate the memorization weight for each sample, evaluating the "typicalness" of a sample. All the training samples are reweigthed based on the proposed memorization weights to reduce the negative effects of atypical samples. Experimental results show the proposed method is flexible to boost state-of-the-art adversarial training methods, improving both robustness and standard accuracy of DNNs.

PDF Details DOI

AAAI Conference 2023 Conference Paper

Painterly Image Harmonization in Dual Domains

Junyan Cao
Yan Hong
Li Niu

Image harmonization aims to produce visually harmonious composite images by adjusting the foreground appearance to be compatible with the background. When the composite image has photographic foreground and painterly background, the task is called painterly image harmonization. There are only few works on this task, which are either time-consuming or weak in generating well-harmonized results. In this work, we propose a novel painterly harmonization network consisting of a dual-domain generator and a dual-domain discriminator, which harmonizes the composite image in both spatial domain and frequency domain. The dual-domain generator performs harmonization by using AdaIN modules in the spatial domain and our proposed ResFFT modules in the frequency domain. The dual-domain discriminator attempts to distinguish the inharmonious patches based on the spatial feature and frequency feature of each patch, which can enhance the ability of generator in an adversarial manner. Extensive experiments on the benchmark dataset show the effectiveness of our method. Our code and model are available at https://github.com/bcmi/PHDNet-Painterly-Image-Harmonization.

PDF Details DOI

IROS Conference 2022 Conference Paper

A Miniature Continuum Robot with Integrated Piezoelectric Beacon Transducers and its Ultrasonic Shape Detection in Robot-Assisted Minimally Invasive Surgeries

Zhanpeng Yin
Yan Hong
Xiaoyu Sun
Zhiyuan Shen
Yingxuan Zhang
Feng Ju
Bruce W. Drinkwater

Minimally invasive surgeries (MIS) or natural orifice transluminal endoscopic surgeries (NOTES) such as the transurethral resection of bladder tumor (TURBT) require the surgical robot to be miniaturized to perform surgical procedures in confined spaces. However, the surgical robot's tiny size poses problems in its fabrication and shape sensing. In this paper, a miniature continuum surgical robot is proposed with a unique laminated structure which can be fabricated through a 2D lamination process and converted into 3D through folding. This multi-material laminated structure also facilitates the integration of tiny piezoelectric transducers on the robot's surface as beacons to generate ultrasonic waves for shape detection. A novel beacon total focusing method (b-TFM) algorithm is developed to process the received ultrasonic data and create a high-quality ultrasonic image from which the shape of the continuum robot can be extracted. The proposed robot and the ultrasonic shape detection method are validated through simulations and experiments. The error in the open-loop trajectory control is less than 4 mm without compensation, and the error in the ultrasonic shape detection is less than 1 mm. This confirms the possibility of improving the trajectory control accuracy by using the detected shape as a feedback for closed-loop control.

Details

AAAI Conference 2022 Conference Paper

Shadow Generation for Composite Image in Real-World Scenes

Yan Hong
Li Niu
Jianfu Zhang

Image composition targets at inserting a foreground object into a background image. Most previous image composition methods focus on adjusting the foreground to make it compatible with background while ignoring the shadow effect of foreground on the background. In this work, we focus on generating plausible shadow for the foreground object in the composite image. First, we contribute a real-world shadow generation dataset DESOBA by generating synthetic composite images based on paired real images and deshadowed images. Then, we propose a novel shadow generation network SGRNet, which consists of a shadow mask prediction stage and a shadow filling stage. In the shadow mask prediction stage, foreground and background information are thoroughly interacted to generate foreground shadow mask. In the shadow filling stage, shadow parameters are predicted to fill the shadow area. Extensive experiments on our DESOBA dataset and real composite images demonstrate the effectiveness of our proposed method. Our dataset and code are available at https: //github. com/bcmi/Object-Shadow-Generation- Dataset-DESOBA.

PDF Details

AAAI Conference 2020 Conference Paper

Reinforcement Mechanism Design: With Applications to Dynamic Pricing in Sponsored Search Auctions

Weiran Shen
Binghui Peng
Hanpeng Liu
Michael Zhang
Ruohan Qian
Yan Hong
Zhi Guo
Zongyao Ding

In many social systems in which individuals and organizations interact with each other, there can be no easy laws to govern the rules of the environment, and agents’ payoffs are often inﬂuenced by other agents’ actions. We examine such a social system in the setting of sponsored search auctions and tackle the search engine’s dynamic pricing problem by combining the tools from both mechanism design and the AI domain. In this setting, the environment not only changes over time, but also behaves strategically. Over repeated interactions with bidders, the search engine can dynamically change the reserve prices and determine the optimal strategy that maximizes the proﬁt. We ﬁrst train a buyer behavior model, with a real bidding data set from a major search engine, that predicts bids given information disclosed by the search engine and the bidders’ performance data from previous rounds. We then formulate the dynamic pricing problem as an MDP and apply a reinforcement-based algorithm that optimizes reserve prices over time. Experiments demonstrate that our model outperforms static optimization strategies including the ones that are currently in use as well as several other dynamic ones.

PDF Details