Author name cluster

Shengwu Xiong

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

19 papers

1 author row

AAAI Conference 2026 Conference Paper

Driving with Advice: Large Model as Motion Advisor for Joint Planning

Junyin Wang
Jinlei Yu
Hao Lin
Huikai Liu
Wenqian Zhu
Shengwu Xiong

We address the challenge of integrating high-level semantic reasoning with low-level trajectory planning in end-to-end autonomous driving, where most existing frameworks decouple perception, decision-making, and control, leading to limited interpretability and poor instruction compliance. To bridge this gap, we propose Driving with Advice, a novel closed-loop framework that treats a vision-language model (VLM) as a motion advisor to provide interpretable, language-mediated guidance for trajectory generation. Our approach introduces three key innovations: (1) Semantic-Intentional Pretraining (SIP), which injects driving rationale into a compact VLM via machine-generated question-answering pairs; (2) a discrete action space grounded in directional and speed primitives, enabling structured and interpretable policy learning; and (3) an advice-following diffusion policy refined via Group Relative Policy Optimization under a multi-objective reward that ensures safety, comfort, and alignment with semantic intent. We evaluate our method on the NAVSIM benchmark in a closed-loop setting, achieving a state-of-the-art Predictive Driver Model Score (PDMS) of 91.5, outperforming strong baselines in safety (NC: 99.2). The results demonstrate that leveraging language as a cognitive interface between perception and control enhances both generalization and behavioral transparency, advancing the paradigm of language-conditioned driving.

PDF Details DOI

EAAI Journal 2026 Journal Article

Exploring a double task learning framework for makeup transfer

Zhaoyang Sun
Shengwu Xiong
Yaxiong Chen
Yi Rong

Although impressive progress has been made with current makeup transfer methods, they are still facing two main challenges that have not been effectively addressed: (1) Due to the lack of real transferred targets, existing methods attempt to synthesize Pseudo Ground Truths (PGTs) to supervise the model training. Therefore, their performance is heavily dependent on the synthesis quality of PGTs. (2) Most previous works fail to achieve semantic alignment between the high-resolution feature maps of the source and reference images. As a result, some high-frequency makeup details will be lost, limiting their ability to deal with diverse makeup styles. In this paper, we propose a Double Task Makeup Transfer (DTMT) framework to handle these two challenges. Specifically, for the first one, DTMT jointly optimizes an unsupervised main makeup transfer task along with a self-supervised auxiliary reconstruction task to avoid the negative effects of sub-optimal PGTs. For the second challenge, we develop a novel Divide and Conquer Attention (DC-Attention) operation, which semantically aligns the high-resolution feature maps of the input images through a coarse-to-fine procedure. Compared to the traditional cross-attention operation, our DC-Attention has much less computational overhead and thus can be more efficient to process high-resolution feature maps. Extensive experiments on three publicly available datasets indicate that DTMT significantly outperforms eight benchmark methods in both quantitative and qualitative evaluations, promoting the development of artificial intelligence virtual makeup try-on. Moreover, DTMT demonstrates strong generalization and control capabilities for makeup styles, advancing the engineering applications of makeup transfer. Our code is available at https: //github. com/Snowfallingplum/DTMT.

Details DOI

AAAI Conference 2026 Conference Paper

ProPL: Universal Semi-Supervised Ultrasound Image Segmentation via Prompt-Guided Pseudo-Labeling

Yaxiong Chen
Qicong Wang
Chunlei Li
Jingliang Hu
Yilei Shi
Shengwu Xiong
Xiao Xiang Zhu
Lichao Mou

Existing approaches for the problem of ultrasound image segmentation, whether supervised or semi-supervised, are typically specialized for specific anatomical structures or tasks, limiting their practical utility in clinical settings. In this paper, we pioneer the task of universal semi-supervised ultrasound image segmentation and propose ProPL, a framework that can handle multiple organs and segmentation tasks while leveraging both labeled and unlabeled data. At its core, ProPL employs a shared vision encoder coupled with prompt-guided dual decoders, enabling flexible task adaptation through a prompting-upon-decoding mechanism and reliable self-training via an uncertainty-driven pseudo-label calibration (UPLC) module. To facilitate research in this direction, we introduce a comprehensive ultrasound dataset spanning 5 organs and 8 segmentation tasks. Extensive experiments demonstrate that ProPL outperforms state-of-the-art methods across various metrics, establishing a new benchmark for universal ultrasound image segmentation.

PDF Details DOI

EAAI Journal 2025 Journal Article

Federated clustering with mutual knowledge distillation for traffic flow prediction

Yao Lin
Shengwu Xiong

Traffic flow prediction plays a critical role in intelligent transportation systems. Conventional traffic flow prediction methods primarily rely on centralized training, which poses a risk of privacy leakage. Federated learning, a privacy-preserving framework to machine learning, enables distributed participants to jointly train a shared model without sharing local private data. However, traffic flow is typically collected from different devices and contains different temporal patterns, leading to non-independent and identically distributed. To address these challenges, we propose a traffic flow prediction method based on federated clustering with mutual knowledge distillation. We first perform temporal decomposition on the traffic flow data and use mutual learning with adaptive distillation loss to facilitate mutual knowledge transfer among local models during training. Then, we apply spectral clustering to cluster clients based on the cosine similarity of model parameters at the server and design a global model aggregation method to improve the performance of federated learning. Finally, the proposed method is evaluated on two real-world traffic datasets, and the experiment results show significant improvements over traditional federated learning approaches and also outperform federated mutual learning. The results demonstrate that the proposed method effectively captures temporal information and mitigates the effect of non-independent and identically distributed issues.

Details DOI

AAAI Conference 2025 Conference Paper

GarFast: Realistic and Fast Garment Transfer with a Simplified Parser-Free Approach

Chenghu Du
Junyin Wang
Yi Rong
Feng Yu
Shengwu Xiong

A good garment try-on model should learn the transfer between different types of garments while satisfying: 1) high fidelity and 2) low inference speed. Existing methods address either of these two issues, limited processing speed or low generation quality. We directly use a lightweight encoder-decoder, ensuring faster speeds. To tackle the problem of lower image quality typically generated by lighter models, we present GarFast, a simplified, parser-free framework that optimizes the same lightweight network through a two-stage transformation of real data roles (from input to supervision), thereby greatly promoting model convergence. Specifically, first, we propose a correction strategy to prevent the difficulty of convergence caused by the lack of ground truth in the first stage. Second, we propose a fine-grained domain consistency to ensure that the results generated in the unsupervised first stage are highly realistic clothed human images. Finally, we propose a skin-variant refinement loss and a skinMix regularization to amplify texture differences and enhance the realism of skin-variant regions, thereby improving the quality of the generated skin. Extensive experiments thoroughly demonstrate that our method achieves high resolution, near real-time performance, and superior reconstruction quality compared to state-of-the-art approaches, with processing times of less than 0.03 seconds on an Nvidia A100.

PDF Details DOI

AAAI Conference 2025 Conference Paper

Latent Diffusion-Enhanced Virtual Try-On via Optimized Pseudo-Label Generation

Chenghu Du
Junyin Wang
Feng Yu
Shengwu Xiong

Efficiently applying fully supervised learning to virtual try-on tasks is challenging due to the lack of paired ground truth in available training samples. Recent works have achieved virtual try-ons by employing self-supervised learning-based inpainting paradigms. However, this approach is heavily dependent on the constraints of inpainting masks. An incorrect mask can mislead the generated results, while overly large mask areas can lose essential original information, thereby hindering the synthesis of high-quality results. To address these problems, we propose a latent diffusion model-based virtual try-on network that achieves fully supervised learning using the concept of cycle consistency and knowledge distillation. Specifically, we divide our approach into pretext and downstream tasks. In the pretext task, we generate a pseudo-label (pseudo-person image) to form paired training samples, which enables the downstream task to achieve fully supervised learning. To prevent the unreliable pseudo-person image from introducing irresponsible prior knowledge, we propose a noise-covering strategy, which aims at fully optimizing the pseudo-label to eliminate the impact of the incorrect inpainting mask as much as possible. Additionally, we propose a skin refinement loss to further enhance the generation of details in the skin region. Extended experiments demonstrate that our proposed method is superior to state-of-the-art methods.

PDF Details DOI

IJCAI Conference 2025 Conference Paper

Mask Does Not Matter: A Unified Latent Diffusion-Enhanced Framework for Mask-Free Virtual Try-On

Chenghu Du
Junyin Wang
Kai Liu
Shengwu Xiong
Yi Rong

A good virtual try-on model should introduce minimal redundant conditional information to avoid instability and increase inference efficiency. Existing methods rely on inpainting masks to guide the generation of the object, but the masks, generated by unstable human parsers, often produce unreliable results with fabric residues due to wrong segmentation. Moreover, large mask regions can lose spatial structure and identity information, requiring extra conditional inputs to compensate, which increases model instability and reduces efficiency. To tackle the problem, we present a novel Mask-Free virtual Try-ON (MFTON) framework. Specifically, we propose a mask-free strategy to eliminate all denoising conditions except for clothing and person images, thereby directly extracting spatial structure and identity information from the person image to improve efficiency and reduce instability. Additionally, to optimize the generated clothing regions, we propose a clothing texture-aware attention mechanism to enable the model to focus on texture generation with significant visual differences. We then introduce a geometric detail capture loss to further enable the model to capture more high-frequency information. Finally, we propose an appearance consistency inference method to reduce the initial randomness of the sampling process significantly. Extensive experiments on popular datasets demonstrate that our method outperforms state-of-the-art virtual try-on methods.

PDF Details DOI

NeurIPS Conference 2025 Conference Paper

Mitigating Occlusions in Virtual Try-On via A Simple-Yet-Effective Mask-Free Framework

Chenghu Du
Shengwu Xiong
Junyin Wang
Yi Rong
Shili Xiong

This paper investigates the occlusion problems in virtual try-on (VTON) tasks. According to how they affect the try-on results, the occlusion issues of existing VTON methods can be grouped into two categories: (1) Inherent Occlusions, which are the ghosts of the clothing from reference input images that exist in the try-on results. (2) Acquired Occlusions, where the spatial structures of the generated human body parts are disrupted and appear unreasonable. To this end, we analyze the causes of these two types of occlusions, and propose a novel mask-free VTON framework based on our analysis to deal with these occlusions effectively. In this framework, we develop two simple-yet-powerful operations: (1) The background pre-replacement operation prevents the model from confusing the target clothing information with the human body or image background, thereby mitigating inherent occlusions. (2) The covering-and-eliminating operation enhances the model's ability of understanding and modeling human semantic structures, leading to more realistic human body generation and thus reducing acquired occlusions. Moreover, our method is highly generalizable, which can be applied in in-the-wild scenarios, and our proposed operations can also be easily integrated into different generative network architectures (e. g. , GANs and diffusion models) in a plug-and-play manner. Extensive experiments on three VTON datasets validate the effectiveness and generalization ability of our method. Both qualitative and quantitative results demonstrate that our method outperforms recently proposed VTON benchmarks.

PDF Details

AAAI Conference 2025 Conference Paper

RealisID: Scale-Robust and Fine-Controllable Identity Customization via Local and Global Complementation

Zhaoyang Sun
Fei Du
Weihua Chen
Fan Wang
Yaxiong Chen
Yi Rong
Shengwu Xiong

Recently, the success of text-to-image synthesis has greatly advanced the development of identity customization techniques, whose main goal is to produce realistic identity-specific photographs based on text prompts and reference face images. However, it is difficult for existing identity customization methods to simultaneously meet the various requirements of different real-world applications, including the identity fidelity of small face, the control of face location, pose and expression, as well as the customization of multiple persons. To this end, we propose a scale-robust and fine-controllable method, namely RealisID, which learns different control capabilities through the cooperation between a pair of local and global branches. Specifically, by using cropping and up-sampling operations to filter out face-irrelevant information, the local branch concentrates the fine control of facial details and the scale-robust identity fidelity within the face region. Meanwhile, the global branch manages the overall harmony of the entire image. It also controls the face location by taking the location guidance as input. As a result, RealisID can benefit from the complementarity of these two branches. Finally, by implementing our branches with two different variants of ControlNet, our method can be easily extended to handle multi-person customization, even only trained on single-person datasets. Extensive experiments and ablation studies indicate the effectiveness of RealisID and verify its ability in fulfilling all the requirements mentioned above.

PDF Details DOI

EAAI Journal 2024 Journal Article

A Fine Rendering High-Resolution Makeup Transfer network via inversion-editing strategy

Zhaoyang Sun
Shengwu Xiong
Yaxiong Chen
Yi Rong

While current makeup transfer methods have made progress in realism and color fidelity, they struggle with capturing texture details and producing high-resolution images, limiting their practical utility. To address these challenges, we propose a Fine Rendering High-resolution Makeup Transfer (FRHMT) network, which leverages a powerful style-based generator and introduces a novel inversion-editing strategy tailored for makeup transfer. Concretely, in the inversion phase, considering the semantic decoupling properties in the latent space, we design a Hierarchical Residual Inversion (HRI), which projects the image onto high-dimensional feature maps in coarse layers and low-dimensional style codes in fine layers. This design effectively restores the content information of the image while maintaining flexibility in editing the makeup styles. In the editing phase, the Makeup Modulation Module (MMM) learns two mapping networks to adjust the latent variables of the source image based on those of the reference image. This modification occurs in fine layers to transfer the makeup information and preserve the content information. With new network structures and customized loss functions, our training eliminates cumbersome pseudo-paired data synthesis and unstable adversarial learning. Extensive experiments have demonstrated that our method outperforms existing methods in both image quality and makeup similarity through quantitative and qualitative analysis. Additionally, we address the lack of high-resolution data by collecting a dataset of 9716 face images with a resolution of 1024 × 1024. In conclusion, our framework offers a novel artificial intelligence (AI) implementation of makeup transfer in engineering, with the collected dataset holding substantial value for further advancements in AI research.

Details DOI

AAAI Conference 2024 Conference Paper

CycleVTON: A Cycle Mapping Framework for Parser-Free Virtual Try-On

Chenghu Du
Junyin Wang
Yi Rong
Shuqing Liu
Kai Liu
Shengwu Xiong

Image-based virtual try-on aims to transfer a target clothing onto a specific person. A significant challenge is arbitrarily matched clothing and person lack corresponding ground truth to supervised learning. A recent pioneering work leveraged an improved cycleGAN to enable one network to generate the desired image for another network during training. However, there is no difference in the result distribution before and after the clothing changes. Therefore, using two different networks is unnecessary and may even increase the difficulty of convergence. Furthermore, the introduced human parsing used to provide body structure information in the input also have a negative impact on the try-on result. How to employ a single network for supervised learning while eliminating human parsing? To tackle these issues, we present a Cycle mapping Virtual Try-On Network (CycleVTON), which can produce photo-realistic try-on results by using a cycle mapping framework without the parser. In particular, we introduce a flow constraint loss to achieve supervised learning of arbitrarily matched clothing and person as inputs to the deformer, thus naturally mimicking the interaction between clothing and the human body. Additionally, we design a skin generation strategy that can adapt to the shape of the target clothing by dynamically adjusting the skin region, i.e., by first removing and then filling skin areas. Extensive experiments conducted on challenging benchmarks demonstrate that our proposed method exhibits superior performance compared to state-of-the-art methods.

PDF Details DOI

EAAI Journal 2024 Journal Article

Fisher ratio-based multi-domain frame-level feature aggregation for short utterance speaker verification

Yunfei Zi
Shengwu Xiong

As the durations of the short utterances are small, it is difficult to learn sufficient information to distinguish the person, thus, short utterance speaker recognition is highly challenging. In this paper, we propose a multi-domain frame-level feature joint learning method to aggregate the discriminative information from multiple dimensions and domain, which is different domains of the speech, time-domain, frequency-domain, and spectral-domain, represent distinct physical characteristics and provide different dimension information, the time domain captures information about the temporal aspect of the physical signal, the frequency domain represents the signal strength in different frequency ranges, and the spectral domain reflects the overall information of the speech, then, based on the extracted multi-domain frame-level features, using the Multi-Fisher criterion aggregates feature parameters categorically and match the corresponding Multi-Fisher ratio weights to the feature parameters as a way to achieve effective feature aggregation and to preserve more effective information, termed Firm-Domain. Extensive experiments are carried out on short-duration text-independent speaker verification datasets derived from the VoxCeleb, SITW, and NIST SRE corpora, which contain speech samples of varying lengths and scenarios. The results demonstrate that the proposed method outperforms the state-of-the-art deep learning architectures by at least 13%, respectively, in the test set. The results of the ablation experiments demonstrate that our proposed methods can significantly outperform previous approaches.

Details DOI

NeurIPS Conference 2024 Conference Paper

SHMT: Self-supervised Hierarchical Makeup Transfer via Latent Diffusion Models

Zhaoyang Sun
Shengwu Xiong
Yaxiong Chen
Fei Du
Weihua Chen
Fan Wang
Yi Rong

This paper studies the challenging task of makeup transfer, which aims to apply diverse makeup styles precisely and naturally to a given facial image. Due to the absence of paired data, current methods typically synthesize sub-optimal pseudo ground truths to guide the model training, resulting in low makeup fidelity. Additionally, different makeup styles generally have varying effects on the person face, but existing methods struggle to deal with this diversity. To address these issues, we propose a novel Self-supervised Hierarchical Makeup Transfer (SHMT) method via latent diffusion models. Following a "decoupling-and-reconstruction" paradigm, SHMT works in a self-supervised manner, freeing itself from the misguidance of imprecise pseudo-paired data. Furthermore, to accommodate a variety of makeup styles, hierarchical texture details are decomposed via a Laplacian pyramid and selectively introduced to the content representation. Finally, we design a novel Iterative Dual Alignment (IDA) module that dynamically adjusts the injection condition of the diffusion model, allowing the alignment errors caused by the domain gap between content and makeup representations to be corrected. Extensive quantitative and qualitative analyses demonstrate the effectiveness of our method. Our code is available at https: //github. com/Snowfallingplum/SHMT.

PDF Details DOI

AAAI Conference 2023 Conference Paper

ESPT: A Self-Supervised Episodic Spatial Pretext Task for Improving Few-Shot Learning

Yi Rong
Xiongbo Lu
Zhaoyang Sun
Yaxiong Chen
Shengwu Xiong

Self-supervised learning (SSL) techniques have recently been integrated into the few-shot learning (FSL) framework and have shown promising results in improving the few-shot image classification performance. However, existing SSL approaches used in FSL typically seek the supervision signals from the global embedding of every single image. Therefore, during the episodic training of FSL, these methods cannot capture and fully utilize the local visual information in image samples and the data structure information of the whole episode, which are beneficial to FSL. To this end, we propose to augment the few-shot learning objective with a novel self-supervised Episodic Spatial Pretext Task (ESPT). Specifically, for each few-shot episode, we generate its corresponding transformed episode by applying a random geometric transformation to all the images in it. Based on these, our ESPT objective is defined as maximizing the local spatial relationship consistency between the original episode and the transformed one. With this definition, the ESPT-augmented FSL objective promotes learning more transferable feature representations that capture the local spatial features of different images and their inter-relational structural information in each input episode, thus enabling the model to generalize better to new categories with only a few samples. Extensive experiments indicate that our ESPT method achieves new state-of-the-art performance for few-shot image classification on three mainstay benchmark datasets. The source code will be available at: https://github.com/Whut-YiRong/ESPT.

PDF Details DOI

EAAI Journal 2023 Journal Article

Federated clustering for recognizing driving styles from private trajectories

Lin Lu
Yao Lin
YUAN WEN
Jinxiong Zhu
Shengwu Xiong

Driving style recognition of real-world drivers is beneficial for various reasons, such as safe and economic driving, auto-insurance and designing autonomous systems. A common way to achieve this goal is to group drivers using clustering methods according to their trajectory data. However, the conventional model training process is centralized, where all drivers’ private trajectories are collected and shared, which has resulted in privacy concerns. Considering that the driving data are sourced from various vehicles and have a decentralized distribution, we introduce a federated clustering approach for privacy-preserving driving style recognition. The method preserves the trajectory data in edge devices, such as connected vehicles or roadside units, and allows only the exchange of model parameters and not raw, sensitive data under the coordination of a central server to meet the privacy protection requirements. To address the clustering heterogeneous challenge posed by the imbalanced distribution of trajectory data in this setting, we combine local Bayesian Gaussian mixture and global weighted K-means to output high-quality global initialization centers. Then, novel local training and global aggregation strategies are proposed to ensure the convergence of training and improve the performance of the final global model. Through comparison experiments on benchmark and real-world datasets, we conclude that this method outperforms existing methods.

Details DOI

NeurIPS Conference 2023 Conference Paper

Greatness in Simplicity: Unified Self-Cycle Consistency for Parser-Free Virtual Try-On

Chenghu Du
Junyin Wang
Shuqing Liu
Shengwu Xiong

Image-based virtual try-on tasks remain challenging, primarily due to inherent complexities associated with non-rigid garment deformation modeling and strong feature entanglement of clothing within human body. Recent groundbreaking formulations, such as in-painting, cycle consistency, and knowledge distillation, have facilitated self-supervised generation of try-on images. However, these paradigms necessitate the disentanglement of garment features within human body features through auxiliary tasks, such as leveraging 'teacher knowledge' and dual generators. The potential presence of irresponsible prior knowledge in the auxiliary task can serve as a significant bottleneck for the main generator (e. g. , 'student model') in the downstream task. Moreover, existing garment deformation methods lack the ability to perceive the correlation between the garment and the human body in the real world, leading to unrealistic alignment effects. To tackle these limitations, we present a new parser-free virtual try-on network based on unified self-cycle consistency (USC-PFN), which enables robust translation between different garments using just a single generator, faithfully replicating non-rigid geometric deformation of garments in real-life scenarios. Specifically, we first propose a self-cycle consistency architecture with a circular mode. It utilizes real unpaired garment-person images exclusively as input for training, effectively eliminating the impact of irresponsible prior knowledge at the model input end. Additionally, we formulate a Markov Random Field to simulate a more natural and realistic garment deformation. Furthermore, USC-PFN can leverage a general generator for self-supervised cycle training. Experiments demonstrate that our method achieves state-of-the-art performance on a popular virtual try-on benchmark.

PDF Details

AAAI Conference 2022 Conference Paper

SSAT: A Symmetric Semantic-Aware Transformer Network for Makeup Transfer and Removal

Zhaoyang Sun
Yaxiong Chen
Shengwu Xiong

Makeup transfer is not only to extract the makeup style of the reference image, but also to render the makeup style to the semantic corresponding position of the target image. However, most existing methods focus on the former and ignore the latter, resulting in a failure to achieve desired results. To solve the above problems, we propose a unified Symmetric Semantic-Aware Transformer (SSAT) network, which incorporates semantic correspondence learning to realize makeup transfer and removal simultaneously. In SSAT, a novel Symmetric Semantic Corresponding Feature Transfer (SSCFT) module and a weakly supervised semantic loss are proposed to model and facilitate the establishment of accurate semantic correspondence. In the generation process, the extracted makeup features are spatially distorted by SSCFT to achieve semantic alignment with the target image, then the distorted makeup features are combined with unmodified makeup irrelevant features to produce the final result. Experiments show that our method obtains more visually accurate makeup transfer results, and user study in comparison with other state-ofthe-art makeup transfer methods reflects the superiority of our method. Besides, we verify the robustness of the proposed method in the difference of expression and pose, object occlusion scenes, and extend it to video makeup transfer.

PDF Details

EAAI Journal 2021 Journal Article

A bi-level distribution mixture framework for unsupervised driving performance evaluation from naturalistic truck driving data

Lin Lu
Shengwu Xiong
Yaxiong Chen

Driving performance evaluations can contribute to fleet management and lead to safer and more economical driving conditions for manned or driverless fleet vehicles. One approach to driving performance evaluation involves quantitative mapping or categorical labeling of skill levels and categorizing of driving patterns from extraordinarily mild to the most aggressive. This paper presents a big data system for driving performance evaluations of drivers and trips using a probabilistic framework. The proposed framework combines a feature mixture model for scoring driving performance through defined objective comparison criteria and a latent style mixture model for classifying drivers by the main driving styles they exhibit. To demonstrate the effectiveness of the proposed models, we perform both quantitative and qualitative experiments. The results show that the former produces an interpretable and normal scorecard model, while the latter helps build an improved clustering model that represents enhanced driver behavior.

Details DOI

AAAI Conference 2020 Conference Paper

Patchy Image Structure Classification Using Multi-Orientation Region Transform

Xiaohan Yu
Yang Zhao
Yongsheng Gao
Shengwu Xiong
Xiaohui Yuan

Exterior contour and interior structure are both vital features for classifying objects. However, most of the existing methods consider exterior contour feature and internal structure feature separately, and thus fail to function when classifying patchy image structures that have similar contours and ﬂexible structures. To address above limitations, this paper proposes a novel Multi-Orientation Region Transform (MORT), which can effectively characterize both contour and structure features simultaneously, for patchy image structure classiﬁcation. MORT is performed over multiple orientation regions at multiple scales to effectively integrate patchy features, and thus enables a better description of the shape in a coarse-to-ﬁne manner. Moreover, the proposed MORT can be extended to combine with the deep convolutional neural network techniques, for further enhancement of classiﬁcation accuracy. Very encouraging experimental results on the challenging ultra-ﬁne-grained cultivar recognition task, insect wing recognition task, and large variation butterﬂy recognition task are obtained, which demonstrate the effectiveness and superiority of the proposed MORT over the state-of-theart methods in classifying patchy image structures. Our code and three patchy image structure datasets are available at: https: //github. com/XiaohanYu-GU/MReT2019.

PDF Details