Author name cluster

Chaoyue Wang

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

20 papers

2 author rows

EAAI Journal 2026 Journal Article

Behavioral three-way decision based multi-attribute decision-making for credible hesitant fuzzy information systems

Tingquan Deng
Chaoyue Wang
Wenjie Wang
Jianming Zhan

Multi-attribute decision-making, as a core of decision analysis, plays important roles in knowledge discovery for artificial intelligence, and finds extensive applications in medical assessments and assisted diagnosis. Assessment information systems are fundamental models of describing properties of objects or patients, which are composed of evaluation values of objects regarding multiple attributes, captured from such as biochemical tests, physical and instrument examinations, and statistic inferences. Due to existence of uncertainty and imprecision in collected these data at different time or from different hospitals, hesitant fuzzy sets are an effective tool of quantizing the evaluation values. In view of experiences, techniques or instrument sensitivity, the quantitative hesitant membership values may be not completely credible. The derived information systems are referred to as credible hesitant fuzzy information systems. To tackle the challenge of multi-attribute decision-making in the scenarios of credible hesitant fuzzy information systems, a behavioral three-way decision model is proposed. Firstly, the notions of dominance degree between credible hesitant fuzzy elements and dominance relation between objects are introduced in credible hesitant fuzzy information systems. With these notions, the weights of attributes are quantitatively assigned to discriminate impact of attributes on decision-making. Secondly, the regret and rejoice functions in regret theory are improved to reflect human psychological emotions for decision-making. A set of payoff functions are then constructed and decision rules are subsequently extracted by introducing the behavior three-way decision model. Finally, three medical assessment cases are comparatively analyzed using representative decision-making methods. The experimental results demonstrate that the proposed method reduces the error rate by at least 50% and promotes the F 1 score by 1. 4% in comparison to the best results among representative decision-making methods on the benchmark baseline cases.

Details DOI

AAAI Conference 2025 Conference Paper

MagicNaming: Consistent Identity Generation by Finding a “Name Space” in T2I Diffusion Models

Jing Zhao
Heliang Zheng
Chaoyue Wang
Long Lan
Wanrong Huang
Yuhua Tang

Large-scale text-to-image diffusion models, (e.g., DALL-E, SDXL) are capable of generating famous persons by simply referring to their names. Is it possible to make such models generate generic identities as simple as the famous ones, e.g., just use a name? In this paper, we explore the existence of a ``Name Space'', where any point in the space corresponds to a specific identity. Fortunately, we find some clues in the feature space spanned by text embedding of celebrities' names. Specifically, we first extract the embeddings of celebrities' names in the Laion5B dataset with the text encoder of diffusion models. Such embeddings are used as supervision to learn an encoder that can predict the name (actually an embedding) of a given face image. We experimentally find that such name embeddings work well in promising the generated image with good identity consistency. Note that like the names of celebrities, our predicted name embeddings are disentangled from the semantics of text inputs, making the original generation capability of text-to-image models well-preserved. Moreover, by simply plugging such name embeddings, all variants (e.g., from Civitai) derived from the same base model (i.e., SDXL) readily become identity-aware text-to-image models.

PDF Details DOI

ICLR Conference 2025 Conference Paper

Semantix: An Energy-guided Sampler for Semantic Style Transfer

Huiang He
Minghui Hu 0001
Chuanxia Zheng
Chaoyue Wang
Tat-Jen Cham

Recent advances in style and appearance transfer are impressive, but most methods isolate global style and local appearance transfer, neglecting semantic correspondence. Additionally, image and video tasks are typically handled in isolation, with little focus on integrating them for video transfer. To address these limitations, we introduce a novel task, *Semantic Style Transfer*, which involves transferring style and appearance features from a reference image to a target visual content based on semantic correspondence. We subsequently propose a training-free method, *Semantix*, an energy-guided sampler designed for Semantic Style Transfer that simultaneously guides both style and appearance transfer based on semantic understanding capacity of pre-trained diffusion models. Additionally, as a sampler, *Semantix* can be seamlessly applied to both image and video models, enabling semantic style transfer to be generic across various visual media. Specifically, once inverting both reference and context images or videos to noise space by SDEs, *Semantix* utilizes a meticulously crafted energy function to guide the sampling process, including three key components: *Style Feature Guidance*, *Spatial Feature Guidance* and *Semantic Distance* as a regularisation term. Experimental results demonstrate that *Semantix* not only effectively accomplishes the task of semantic style transfer across images and videos, but also surpasses existing state-of-the-art solutions in both fields.

Details

AAAI Conference 2024 Conference Paper

Autoregressive Omni-Aware Outpainting for Open-Vocabulary 360-Degree Image Generation

Zhuqiang Lu
Kun Hu
Chaoyue Wang
Lei Bai
Zhiyong Wang

A 360-degree (omni-directional) image provides an all-encompassing spherical view of a scene. Recently, there has been an increasing interest in synthesising 360-degree images from conventional narrow field of view (NFoV) images captured by digital cameras and smartphones, for providing immersive experiences in various scenarios such as virtual reality. Yet, existing methods typically fall short in synthesizing intricate visual details or ensure the generated images align consistently with user-provided prompts. In this study, autoregressive omni-aware generative network (AOG-Net) is proposed for 360-degree image generation by outpainting an incomplete 360-degree image progressively with NFoV and text guidances joinly or individually. This autoregressive scheme not only allows for deriving finer-grained and text-consistent patterns by dynamically generating and adjusting the process but also offers users greater flexibility to edit their conditions throughout the generation process. A global-local conditioning mechanism is devised to comprehensively formulate the outpainting guidance in each autoregressive step. Text guidances, omni-visual cues, NFoV inputs and omni-geometry are encoded and further formulated with cross-attention based transformers into a global stream and a local stream into a conditioned generative backbone model. As AOG-Net is compatible to leverage large-scale models for the conditional encoder and the generative prior, it enables the generation to use extensive open-vocabulary text guidances. Comprehensive experiments on two commonly used 360-degree image datasets for both indoor and outdoor settings demonstrate the state-of-the-art performance of our proposed method. Our code is available at https://github.com/zhuqiangLu/AOG-NET-360.

PDF Details DOI

AAAI Conference 2024 Conference Paper

Decomposing Semantic Shifts for Composed Image Retrieval

Xingyu Yang
Daqing Liu
Heng Zhang
Yong Luo
Chaoyue Wang
Jing Zhang

Composed image retrieval is a type of image retrieval task where the user provides a reference image as a starting point and specifies a text on how to shift from the starting point to the desired target image. However, most existing methods focus on the composition learning of text and reference images and oversimplify the text as a description, neglecting the inherent structure and the user's shifting intention of the texts. As a result, these methods typically take shortcuts that disregard the visual cue of the reference images. To address this issue, we reconsider the text as instructions and propose a Semantic Shift Network (SSN) that explicitly decomposes the semantic shifts into two steps: from the reference image to the visual prototype and from the visual prototype to the target image. Specifically, SSN explicitly decomposes the instructions into two components: degradation and upgradation, where the degradation is used to picture the visual prototype from the reference image, while the upgradation is used to enrich the visual prototype into the final representations to retrieve the desired target image. The experimental results show that the proposed SSN demonstrates a significant improvement of 5.42% and 1.37% on the CIRR and FashionIQ datasets, respectively, and establishes a new state-of-the-art performance. The code is available at https://github.com/starxing-yuu/SSN.

PDF Details DOI

AAAI Conference 2024 Short Paper

Dual Mapping of 2D StyleGAN for 3D-Aware Image Generation and Manipulation (Student Abstract)

Zhuo Chen
Haimei Zhao
Chaoyue Wang
Bo Yuan
Xiu Li

3D-aware GANs successfully solve the problem of 3D-consistency generation and furthermore provide a 3D shape of the generated object. However, the application of the volume renderer disturbs the disentanglement of the latent space, which makes it difficult to manipulate 3D-aware GANs and lowers the image quality of style-based generators. In this work, we devise a dual-mapping framework to make the generated images of pretrained 2D StyleGAN consistent in 3D space. We utilize a tri-plane representation to estimate the 3D shape of the generated object and two mapping networks to bridge the latent space of StyleGAN and the 3D tri-plane space. Our method does not alter the parameters of the pretrained generator, which means the interpretability of latent space is preserved for various image manipulations. Experiments show that our method lifts the 3D awareness of pretrained 2D StyleGAN to 3D-aware GANs and outperforms the 3D-aware GANs in controllability and image quality.

PDF Details DOI

IJCAI Conference 2024 Conference Paper

Eliminating the Cross-Domain Misalignment in Text-guided Image Inpainting

Muqi Huang
Chaoyue Wang
Yong Luo
Lefei Zhang

Text-guided image inpainting has rapidly garnered prominence as a task in user-directed image synthesis, aiming to complete the occluded image regions following the textual prompt provided. However, current methods usually grapple with issues arising from the disparity between low-level pixel data and high-level semantic descriptions, which results in inpainted sections not harmonizing with the original image (either structurally or texturally). In this study, we introduce a Structure-Aware Inpainting Learning scheme and an Asymmetric Cross Domain Attention to address these cross-domain misalignment challenges. The proposed structure-aware learning scheme employs features of an intermediate modality as structure guidance to bridge the gap between text information and low-level pixels. Meanwhile, asymmetric cross-domain attention enhances the texture consistency between inpainted and unmasked regions. Our experiments show exceptional performance on leading datasets such as MS-COCO and Open Images, surpassing state-of-the-art text-guided image inpainting methods. Code is released at: https: //github. com/MucciH/ECDM-inpainting

PDF Details DOI

AAAI Conference 2024 Conference Paper

Multi-Step Denoising Scheduled Sampling: Towards Alleviating Exposure Bias for Diffusion Models

Zhiyao Ren
Yibing Zhan
Liang Ding
Gaoang Wang
Chaoyue Wang
Zhongyi Fan
Dacheng Tao

Denoising Diffusion Probabilistic Models (DDPMs) have achieved significant success in generation tasks. Nevertheless, the exposure bias issue, i.e., the natural discrepancy between the training (the output of each step is calculated individually by a given input) and inference (the output of each step is calculated based on the input iteratively obtained based on the model), harms the performance of DDPMs. To our knowledge, few works have tried to tackle this issue by modifying the training process for DDPMs, but they still perform unsatisfactorily due to 1) partially modeling the discrepancy and 2) ignoring the prediction error accumulation. To address the above issues, in this paper, we propose a multi-step denoising scheduled sampling (MDSS) strategy to alleviate the exposure bias for DDPMs. Analyzing the formulations of the training and inference of DDPMs, MDSS 1) comprehensively considers the discrepancy influence of prediction errors on the output of the model (the Gaussian noise) and the output of the step (the calculated input signal of the next step), and 2) efficiently models the prediction error accumulation by using multiple iterations of a mathematical formulation initialized from one-step prediction error obtained from the model. The experimental results, compared with previous works, demonstrate that our approach is more effective in mitigating exposure bias in DDPM, DDIM, and DPM-solver. In particular, MDSS achieves an FID score of 3.86 in 100 sample steps of DDIM on the CIFAR-10 dataset, whereas the second best obtains 4.78. The code will be available on GitHub.

PDF Details DOI

NeurIPS Conference 2023 Conference Paper

All Points Matter: Entropy-Regularized Distribution Alignment for Weakly-supervised 3D Segmentation

Liyao Tang
Zhe Chen
Shanshan Zhao
Chaoyue Wang
Dacheng Tao

Pseudo-labels are widely employed in weakly supervised 3D segmentation tasks where only sparse ground-truth labels are available for learning. Existing methods often rely on empirical label selection strategies, such as confidence thresholding, to generate beneficial pseudo-labels for model training. This approach may, however, hinder the comprehensive exploitation of unlabeled data points. We hypothesize that this selective usage arises from the noise in pseudo-labels generated on unlabeled data. The noise in pseudo-labels may result in significant discrepancies between pseudo-labels and model predictions, thus confusing and affecting the model training greatly. To address this issue, we propose a novel learning strategy to regularize the generated pseudo-labels and effectively narrow the gaps between pseudo-labels and model predictions. More specifically, our method introduces an Entropy Regularization loss and a Distribution Alignment loss for weakly supervised learning in 3D segmentation tasks, resulting in an ERDA learning strategy. Interestingly, by using KL distance to formulate the distribution alignment loss, it reduces to a deceptively simple cross-entropy-based loss which optimizes both the pseudo-label generation network and the 3D segmentation network simultaneously. Despite the simplicity, our method promisingly improves the performance. We validate the effectiveness through extensive experiments on various baselines and large-scale datasets. Results show that ERDA effectively enables the effective usage of all unlabeled data points for learning and achieves state-of-the-art performance under different settings. Remarkably, our method can outperform fully-supervised baselines using only 1\% of true annotations. Code and model will be made publicly available at https: //github. com/LiyaoTang/ERDA.

PDF Details

NeurIPS Conference 2023 Conference Paper

Cocktail: Mixing Multi-Modality Control for Text-Conditional Image Generation

Minghui Hu
Jianbin Zheng
Daqing Liu
Chuanxia Zheng
Chaoyue Wang
Dacheng Tao
Tat-Jen Cham

Text-conditional diffusion models are able to generate high-fidelity images with diverse contents. However, linguistic representations frequently exhibit ambiguous descriptions of the envisioned objective imagery, requiring the incorporation of additional control signals to bolster the efficacy of text-guided diffusion models. In this work, we propose Cocktail, a pipeline to mix various modalities into one embedding, amalgamated with a generalized ControlNet (gControlNet), a controllable normalisation (ControlNorm), and a spatial guidance sampling method, to actualize multi-modal and spatially-refined control for text-conditional diffusion models. Specifically, we introduce a hyper-network gControlNet, dedicated to the alignment and infusion of the control signals from disparate modalities into the pre-trained diffusion model. gControlNet is capable of accepting flexible modality signals, encompassing the simultaneous reception of any combination of modality signals, or the supplementary fusion of multiple modality signals. The control signals are then fused and injected into the backbone model according to our proposed ControlNorm. Furthermore, our advanced spatial guidance sampling methodology proficiently incorporates the control signal into the designated region, thereby circumventing the manifestation of undesired objects within the generated image. We demonstrate the results of our method in controlling various modalities, proving high-quality synthesis and fidelity to multiple external signals.

PDF Details

NeurIPS Conference 2023 Conference Paper

Domain Re-Modulation for Few-Shot Generative Domain Adaptation

Yi Wu
Ziqiang Li
Chaoyue Wang
Heliang Zheng
Shanshan Zhao
Bin Li
Dacheng Tao

In this study, we delve into the task of few-shot Generative Domain Adaptation (GDA), which involves transferring a pre-trained generator from one domain to a new domain using only a few reference images. Inspired by the way human brains acquire knowledge in new domains, we present an innovative generator structure called $\textbf{Domain Re-Modulation (DoRM)}$. DoRM not only meets the criteria of $\textit{high quality}$, $\textit{large synthesis diversity}$, and $\textit{cross-domain consistency}$, which were achieved by previous research in GDA, but also incorporates $\textit{memory}$ and $\textit{domain association}$, akin to how human brains operate. Specifically, DoRM freezes the source generator and introduces new mapping and affine modules (M\&A modules) to capture the attributes of the target domain during GDA. This process resembles the formation of new synapses in human brains. Consequently, a linearly combinable domain shift occurs in the style space. By incorporating multiple new M\&A modules, the generator gains the capability to perform high-fidelity multi-domain and hybrid-domain generation. Moreover, to maintain cross-domain consistency more effectively, we introduce a similarity-based structure loss. This loss aligns the auto-correlation map of the target image with its corresponding auto-correlation map of the source image during training. Through extensive experiments, we demonstrate the superior performance of our DoRM and similarity-based structure loss in few-shot GDA, both quantitatively and qualitatively. Code will be available at https: //github. com/wuyi2020/DoRM.

PDF Details

ICLR Conference 2023 Conference Paper

Unified Discrete Diffusion for Simultaneous Vision-Language Generation

Minghui Hu 0001
Chuanxia Zheng
Zuopeng Yang
Tat-Jen Cham
Heliang Zheng
Chaoyue Wang
Dacheng Tao
Ponnuthurai N. Suganthan

The recently developed discrete diffusion model performs extraordinarily well in generation tasks, especially in the text-to-image task, showing great potential for modeling multimodal signals. In this paper, we leverage these properties and present a unified multimodal generation model, which can perform text-based, image-based, and even vision-language simultaneous generation using a single model. Specifically, we unify the discrete diffusion process for multimodal signals by proposing a unified Markov transition matrix and a unified objective. Moreover, we design a multimodal mutual attention module to highlight the inter-modal linkages, which is vital for multimodal generation. Extensive experiments indicate that our proposed method can perform comparably to the state-of-the-art solutions in various generation tasks.

Details

YNIMG Journal 2022 Journal Article

An automated pipeline for extracting histological stain area fraction for voxelwise quantitative MRI-histology comparisons

Daniel Z.L. Kor
Saad Jbabdi
Istvan N. Huszar
Jeroen Mollink
Benjamin C. Tendler
Sean Foxley
Chaoyue Wang
Connor Scott

The acquisition of MRI and histology in the same post-mortem tissue sample enables direct correlation between MRI and histologically-derived parameters. However, there still lacks a standardised automated pipeline to process histology data, with most studies relying on manual intervention. Here, we introduce an automated pipeline to extract a quantitative histological measure for staining density (stain area fraction, SAF) from multiple immunohistochemical (IHC) stains. The pipeline is designed to directly address key IHC artefacts related to tissue staining and slide digitisation. Here, the pipeline was applied to post-mortem human brain data from multiple subjects, relating MRI parameters (FA, MD, RD, AD, R2*, R1) to IHC slides stained for myelin, neurofilaments, microglia and activated microglia. Utilising high-quality MRI-histology co-registrations, we then performed whole-slide voxelwise comparisons (simple correlations, partial correlations and multiple regression analyses) between multimodal MRI- and IHC-derived parameters. The pipeline was found to be reproducible, robust to artefacts and generalisable across multiple IHC stains. Our partial correlation results suggest that some simple MRI-SAF correlations should be interpreted with caution, due to the co-localisation of other tissue features (e.g., myelin and neurofilaments). Further, we find activated microglia-a generic biomarker of inflammation-to consistently be the strongest predictor of high DTI FA and low RD, which may suggest sensitivity of diffusion MRI to aspects of neuroinflammation related to microglial activation, even after accounting for other microstructural changes (demyelination, axonal loss and general microglia infiltration). Together, these results show the utility of this approach in carefully curating IHC data and performing multimodal analyses to better understand microstructural relationships with MRI.

Details DOI

JBHI Journal 2022 Journal Article

S2Q-Net: Mining the High-Pass Filtered Phase Data in Susceptibility Weighted Imaging for Quantitative Susceptibility Mapping

Zhiyang Lu
Jun Li
Chaoyue Wang
Rongjun Ge
Lili Chen
Hongjian He
Jun Shi

Susceptibility weighted imaging (SWI) is a routine magnetic resonance imaging (MRI) sequence that combines the magnitude and high-pass filtered phase images to qualitatively enhance the image contrasts related to tissue susceptibility. Tremendous amounts of the high-pass filtered phase data with low signal to noise ratio and incomplete background field removal have thus been collected under default clinical settings. Since SWI cannot quantitatively estimate the susceptibility, it is thus non-trivial to derive quantitative susceptibility mapping (QSM) directly from these redundant phase data, which effectively promotes the mining of the SWI data collected previously. To this end, a novel deep learning based SWI-to-QSM-Net (S2Q-Net) is proposed for QSM reconstruction from SWI high-pass filtered phase data. S2Q-Net firstly estimates the edge maps of QSM to integrate edge prior into features, which benefits the network to reconstruct QSM with realistic and clear tissue boundaries. Furthermore, a novel Second-order Cross Dense Block is proposed in S2Q-Net, which can capture rich inter-region interactions to provide more non-local phase information related to local tissue susceptibility. Experimental results on both simulated and in-vivo data indicate its superiority over all the compared deep learning based QSM reconstruction methods.

Details DOI

NeurIPS Conference 2022 Conference Paper

SemMAE: Semantic-Guided Masking for Learning Masked Autoencoders

Gang Li
Heliang Zheng
Daqing Liu
Chaoyue Wang
Bing Su
Changwen Zheng

Recently, significant progress has been made in masked image modeling to catch up to masked language modeling. However, unlike words in NLP, the lack of semantic decomposition of images still makes masked autoencoding (MAE) different between vision and language. In this paper, we explore a potential visual analogue of words, i. e. , semantic parts, and we integrate semantic information into the training process of MAE by proposing a Semantic-Guided Masking strategy. Compared to widely adopted random masking, our masking strategy can gradually guide the network to learn various information, i. e. , from intra-part patterns to inter-part relations. In particular, we achieve this in two steps. 1) Semantic part learning: we design a self-supervised part learning method to obtain semantic parts by leveraging and refining the multi-head attention of a ViT-based encoder. 2) Semantic-guided MAE (SemMAE) training: we design a masking strategy that varies from masking a portion of patches in each part to masking a portion of (whole) parts in an image. Extensive experiments on various vision tasks show that SemMAE can learn better image representation by integrating semantic information. In particular, SemMAE achieves 84. 5% fine-tuning accuracy on ImageNet-1k, which outperforms the vanilla MAE by 1. 4%. In the semantic segmentation and fine-grained recognition tasks, SemMAE also brings significant improvements and yields the state-of-the-art performance.

PDF Details

AAAI Conference 2022 Conference Paper

Visual Semantics Allow for Textual Reasoning Better in Scene Text Recognition

Yue He
Chen Chen
Jing Zhang
Juhua Liu
Fengxiang He
Chaoyue Wang
Bo Du

Existing Scene Text Recognition (STR) methods typically use a language model to optimize the joint probability of the 1D character sequence predicted by a visual recognition (VR) model, which ignore the 2D spatial context of visual semantics within and between character instances, making them not generalize well to arbitrary shape scene text. To address this issue, we make the first attempt to perform textual reasoning based on visual semantics in this paper. Technically, given the character segmentation maps predicted by a VR model, we construct a subgraph for each instance, where nodes represent the pixels in it and edges are added between nodes based on their spatial similarity. Then, these subgraphs are sequentially connected by their root nodes and merged into a complete graph. Based on this graph, we devise a graph convolutional network for textual reasoning (GTR) by supervising it with a cross-entropy loss. GTR can be easily plugged in representative STR models to improve their performance owing to better textual reasoning. Specifically, we construct our model, namely S-GTR, by paralleling GTR to the language model in a segmentation-based STR baseline, which can effectively exploit the visual-linguistic complementarity via mutual learning. S-GTR sets new state-of-the-art on six challenging STR benchmarks and generalizes well to multi-linguistic datasets. Code is available at https: //github. com/adeline-cs/GTR.

PDF Details

YNIMG Journal 2021 Journal Article

MRI-based Alzheimer’s disease prediction via distilling the knowledge in multi-modal data

Hao Guan
Chaoyue Wang
Dacheng Tao

Mild cognitive impairment (MCI) conversion prediction, i.e., identifying MCI patients of high risks converting to Alzheimer's disease (AD), is essential for preventing or slowing the progression of AD. Although previous studies have shown that the fusion of multi-modal data can effectively improve the prediction accuracy, their applications are largely restricted by the limited availability or high cost of multi-modal data. Building an effective prediction model using only magnetic resonance imaging (MRI) remains a challenging research topic. In this work, we propose a multi-modal multi-instance distillation scheme, which aims to distill the knowledge learned from multi-modal data to an MRI-based network for MCI conversion prediction. In contrast to existing distillation algorithms, the proposed multi-instance probabilities demonstrate a superior capability of representing the complicated atrophy distributions, and can guide the MRI-based network to better explore the input MRI. To our best knowledge, this is the first study that attempts to improve an MRI-based prediction model by leveraging extra supervision distilled from multi-modal information. Experiments demonstrate the advantage of our framework, suggesting its potentials in the data-limited clinical settings.

Details DOI

YNIMG Journal 2020 Journal Article

Methods for quantitative susceptibility and R2* mapping in whole post-mortem brains at 7T applied to amyotrophic lateral sclerosis

Chaoyue Wang
Sean Foxley
Olaf Ansorge
Sarah Bangerter-Christensen
Mark Chiew
Anna Leonte
Ricarda AL Menke
Jeroen Mollink

Susceptibility weighted magnetic resonance imaging (MRI) is sensitive to the local concentration of iron and myelin. Here, we describe a robust image processing pipeline for quantitative susceptibility mapping (QSM) and R2* mapping of fixed post-mortem, whole-brain data. Using this pipeline, we compare the resulting quantitative maps in brains from patients with amyotrophic lateral sclerosis (ALS) and controls, with validation against iron and myelin histology. Twelve post-mortem brains were scanned with a multi-echo gradient echo sequence at 7T, from which susceptibility and R2* maps were generated. Semi-quantitative histological analysis for ferritin (the principal iron storage protein) and myelin proteolipid protein was performed in the primary motor, anterior cingulate and visual cortices. Magnetic susceptibility and R2* values in primary motor cortex were higher in ALS compared to control brains. Magnetic susceptibility and R2* showed positive correlations with both myelin and ferritin estimates from histology. Four out of nine ALS brains exhibited clearly visible hyperintense susceptibility and R2* values in the primary motor cortex. Our results demonstrate the potential for MRI-histology studies in whole, fixed post-mortem brains to investigate the biophysical source of susceptibility weighted MRI signals in neurodegenerative diseases like ALS.

Details DOI

NeurIPS Conference 2020 Conference Paper

Video Frame Interpolation without Temporal Priors

Youjian Zhang
Chaoyue Wang
Dacheng Tao

Video frame interpolation, which aims to synthesize non-exist intermediate frames in a video sequence, is an important research topic in computer vision. Existing video frame interpolation methods have achieved remarkable results under specific assumptions, such as instant or known exposure time. However, in complicated real-world situations, the temporal priors of videos, i. e. frames per second (FPS) and frame exposure time, may vary from different camera sensors. When test videos are taken under different exposure settings from training ones, the interpolated frames will suffer significant misalignment problems. In this work, we solve the video frame interpolation problem in a general situation, where input frames can be acquired under uncertain exposure (and interval) time. Unlike previous methods that can only be applied to a specific temporal prior, we derive a general curvilinear motion trajectory formula from four consecutive sharp frames or two consecutive blurry frames without temporal priors. Moreover, utilizing constraints within adjacent motion trajectories, we devise a novel optical flow refinement strategy for better interpolation results. Finally, experiments demonstrate that one well-trained model is enough for synthesizing high-quality slow-motion videos under complicated real-world situations. Codes are available on https: //github. com/yjzhang96/UTI-VFI.

PDF Details

IJCAI Conference 2017 Conference Paper

Tag Disentangled Generative Adversarial Network for Object Image Re-rendering

Chaoyue Wang
Chaohui Wang
Chang Xu
Dacheng Tao

In this paper, we propose a principled Tag Disentangled Generative Adversarial Networks (TD-GAN) for re-rendering new images for the object of interest from a single image of it by specifying multiple scene properties (such as viewpoint, illumination, expression, etc. ). The whole framework consists of a disentangling network, a generative network, a tag mapping net, and a discriminative network, which are trained jointly based on a given set of images that are completely/partially tagged (i. e. , supervised/semi-supervised setting). Given an input image, the disentangling network extracts disentangled and interpretable representations, which are then used to generate images by the generative network. In order to boost the quality of disentangled representations, the tag mapping net is integrated to explore the consistency between the image and its tags. Furthermore, the discriminative network is introduced to implement the adversarial training strategy for generating more realistic images. Experiments on two challenging datasets demonstrate the state-of-the-art performance of the proposed framework in the problem of interest.

PDF Details