Author name cluster

Hui Huang

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

25 papers

2 author rows

AAAI Conference 2026 Conference Paper

A Reasoning Paradigm for Named Entity Recognition

Hui Huang
Yanping Chen
Ruizhang Huang
Chuan Lin
Yongbin Qin

Generative LLMs typically improve Named Entity Recognition (NER) performance through instruction tuning. They excel at generating entities by semantic pattern matching but lack an explicit, verifiable reasoning mechanism. This "cognitive shortcutting" leads to suboptimal performance and weak generalization, especially in zero-shot and low-resource scenarios where reasoning from limited contextual cues is crucial. To address this issue, a reasoning framework is proposed for NER, which shifts the extraction paradigm from implicit pattern matching to explicit reasoning. This framework consists of three stages: Chain of Thought (CoT) generation, CoT tuning, and reasoning enhancement. First, a dataset annotated with NER-oriented CoTs is generated, which contain task-relevant reasoning chains. Then, they are used to tune the NER model to generate coherent rationales before deriving the final answer. Finally, a reasoning enhancement stage is implemented to optimize the reasoning process using a comprehensive reward signal. This stage ensures explicit and verifiable extractions. Experiments show that ReasoningNER demonstrates impressive cognitive ability in the NER task, achieving competitive performance. In zero-shot settings, it achieves SoTA performance, outperforming GPT-4 by 12.3 percentage points on the F1 score. Analytical results demonstrate its great potential to advance research in reasoning-oriented information extraction.

PDF Details DOI

AAAI Conference 2026 Conference Paper

Long-form RewardBench: Evaluating Reward Models for Long-form Generation

Hui Huang
Yancheng He
Wei Liu
Muyun Yang
Jiaheng Liu
Kehai Chen
Bing Xu
Conghui Zhu

The widespread adoption of reinforcement learning-based alignment highlights the growing importance of reward models. Various benchmarks have been built to evaluate reward models in various domains and scenarios. However, a significant gap remains in assessing reward models for long-form generation, despite its critical role in real-world applications. To bridge this, we introduce Long-form RewardBench, the first reward modeling testbed specifically designed for long-form generation. Our benchmark encompasses five key subtasks: QA, RAG, Chat, Writing, and Reasoning. We collected instruction and preference data through a meticulously designed multi-stage data collection process, and conducted extensive experiments on 20+ mainstream reward models, including both classifiers and generative models. Our findings reveal that current models still lack long-form reward modeling capabilities. Furthermore, we designed a novel Long-form Needle-in-a-Haystack Test, which revealed a correlation between reward modeling performance and the error's position within a response, as well as the overall response length, with distinct characteristics observed between classification and generative models. Finally, we demonstrate that classifier exhibit better generalizability compared to generative models trained on the same data. As the first benchmark for long-form reward modeling, this work aims to offer a robust platform for visualizing progress in this crucial area.

PDF Details DOI

AAAI Conference 2026 Conference Paper

Lost in Benchmarks? Rethinking Large Language Model Benchmarking with Item Response Theory

Hongli Zhou
Hui Huang
Ziqing Zhao
Lvyuan Han
Huicheng Wang
Kehai Chen
Muyun Yang
Wei Bao

The evaluation of large language models (LLMs) via benchmarks is widespread, yet inconsistencies between different leaderboards and poor separability among top models raise concerns about their ability to accurately reflect authentic model capabilities. This paper provides a critical analysis of benchmark effectiveness, examining mainstream prominent LLM benchmarks using results from diverse models. We first propose Pseudo-Siamese Network for Item Response Theory (PSN-IRT), an enhanced Item Response Theory framework that incorporates a rich set of item parameters within an IRT-grounded architecture. PSN-IRT can be utilized for accurate and reliable estimations of item characteristics and model abilities. Based on PSN-IRT, we conduct extensive analysis on 11 LLM benchmarks comprising 41,871 items, revealing significant and varied shortcomings in their measurement quality. Furthermore, we demonstrate that leveraging PSN-IRT is able to construct smaller benchmarks while maintaining stronger alignment with human preference.

PDF Details DOI

AAAI Conference 2026 Conference Paper

StrokeFusion: Vector Sketch Generation via Joint Stroke-UDF Encoding and Latent Sequence Diffusion

Jin Zhou
Yi Zhou
Hongliang Yang
Pengfei Xu
Hui Huang

In the field of sketch generation, raster-format trained models often produce non-stroke artifacts, while vector-format trained models typically lack a holistic understanding of sketches, leading to compromised recognizability. Moreover, existing methods struggle to extract common features from similar elements (e.g., eyes of animals) appearing at varying positions across sketches. To address these challenges, we propose StrokeFusion, a two-stage framework for vector sketch generation. It contains a dual-modal sketch feature learning network that maps strokes into a high-quality latent space. This network decomposes sketches into normalized strokes and jointly encodes stroke sequences with Unsigned Distance Function (UDF) maps, representing sketches as sets of stroke feature vectors. Building upon this representation, our framework exploits a stroke-level latent diffusion model that simultaneously adjusts stroke position, scale, and trajectory during generation. This enables high-fidelity stroke generation while supporting stroke interpolation editing. Extensive experiments across multiple sketch datasets, demonstrate that our framework outperforms state-of-the-art techniques, validating its effectiveness in preserving structural integrity and semantic features.

PDF Details DOI

AAAI Conference 2026 Conference Paper

TG-Field: Geometry-Aware Radiative Gaussian Fields for Tomographic Reconstruction

Yuxiang Zhong
Jun Wei
Chaoqi Chen
Senyou An
Hui Huang

3D Gaussian Splatting (3DGS) has revolutionized 3D scene representation with superior efficiency and quality. While recent adaptations for computed tomography (CT) show promise, they struggle with severe artifacts under highly sparse-view projections and dynamic motions. To address these challenges, we propose Tomographic Geometry Field (TG-Field), a geometry-aware Gaussian deformation framework tailored for both static and dynamic CT reconstruction. A multi-resolution hash encoder is employed to capture local spatial priors, regularizing primitive parameters under ultra-sparse settings. We further extend the framework to dynamic reconstruction by introducing time-conditioned representations and a spatiotemporal attention block to adaptively aggregate features, thereby resolving spatiotemporal ambiguities and enforcing temporal coherence. In addition, a motion-flow network models fine-grained respiratory motion to track local anatomical deformations. Extensive experiments on synthetic and real-world datasets demonstrate that TG-Field consistently outperforms existing methods, achieving state-of-the-art reconstruction accuracy under highly sparse-view conditions.

PDF Details DOI

AAAI Conference 2026 Conference Paper

Think-J: Learning to Think for Generative LLM-as-a-Judge

Hui Huang
Yancheng He
Hongli Zhou
Rui Zhang
Wei Liu
Weixun Wang
Jiaheng Liu
Wenbo Su

LLM-as-a-Judge refers to the automatic modeling of preferences for responses generated by Large Language Models (LLMs), which is of significant importance for both LLM evaluation and reward modeling. Although generative LLMs have made substantial progress in various tasks, their performance as LLM-Judge still falls short of expectations. In this work, we propose Think-J, which improves generative LLM-as-a-Judge by learning how to think. We first utilized a small amount of curated data to develop the model with initial judgment thinking capabilities. Subsequently, we optimize the judgment thinking traces based on reinforcement learning (RL). We propose two methods for judgment thinking optimization, based on offline and online RL, respectively. The offline method requires training a critic model to construct positive and negative examples for learning. The online method defines rule-based reward as feedback for optimization. Experimental results showed that our approach can significantly enhance the evaluation capability of generative LLM-Judge, surpassing both generative and classifier-based LLM-Judge without requiring extra human annotations.

PDF Details DOI

AAAI Conference 2025 Conference Paper

Out-of-Distribution Detection with Prototypical Outlier Proxy

Mingrong Gong
Chaoqi Chen
Qingqiang Sun
Yue Wang
Hui Huang

Out-of-distribution (OOD) detection is a crucial task for deploying deep learning models in the wild. One of the major challenges is that well-trained deep models tend to perform over-confidence on unseen test data. Recent research attempts to leverage real or synthetic outliers to mitigate the issue, which may significantly increase computational costs and be biased toward specific outlier characteristics. In this paper, we propose a simple yet effective framework, Prototypical Outlier Proxy (POP), which introduces virtual OOD prototypes to reshape the decision boundaries between ID and OOD data. Specifically, we transform the learnable classifier into a fixed one and augment it with a set of prototypical weight vectors. Then, we introduce a hierarchical similarity boundary loss to impose adaptive penalties depending on the degree of misclassification. Extensive experiments across various benchmarks demonstrate the effectiveness of POP. Notably, POP achieves average FPR95 reductions of 7.70%, 6.30%, and 5.42% over the second-best methods on CIFAR-10, CIFAR-100, and ImageNet-200, respectively. Moreover, compared to the recent method NPOS, which relies on outlier synthesis, POP trains 7.2 times faster and performs inference 19.5 times faster.

PDF Details DOI

EAAI Journal 2025 Journal Article

Unsupervised wear detection for abrasive tools using audio features and dual-masked graph autoencoder

Shuangjin Shi
Lili Tang
Hui Tian
Hui Huang
Ching-Chun Chang
Chin-Chen Chang

Abrasive tool wear detection is a critical technology for advancing Industry 4. 0, ensuring high-quality and efficient machining in industries such as aerospace and optoelectronics. However, developing a high-accuracy, non-contact detection method that performs reliably under complex operating conditions remains a significant challenge. To address this problem, this paper presents a novel unsupervised approach for abrasive tool wear detection, leveraging audio-cepstral features and a dual-masked graph autoencoder, marking the first application of graph-based structures in audio-driven tool wear detection. The proposed method models audio signals with a cross-graph structure, fully integrating both numerical and structural characteristics of mel-frequency cepstral coefficients. A dual-masked graph autoencoder is introduced to learn wear-specific features, enhancing robustness against by masking the nodes again in the decoding. Additionally, a novel loss function for wear detection is designed by combining mean squared error and cosine similarity with focal and weighting factors, enabling a comprehensive reconstruction of both value differences and directional relationships in non-Euclidean graph structures. Finally, a lightweight wear detection network uses the learned features for accurate classification of wear states. Experimental results demonstrate robust and accurate detection performance under eight complex operating conditions (including various operating conditions and noise types). The proposed method achieves an area under the curve of 0. 9461 in mixed conditions, outperforming state-of-the-art methods by 1. 48–21. 59%. Ablation studies on backbone network, masking schemes, graph transformations, and loss functions further validate the effectiveness of the proposed method.

Details DOI

AAAI Conference 2025 Conference Paper

View Transformation Robustness for Multi-View 3D Object Reconstruction with Reconstruction Error-Guided View Selection

Qi Zhang
Zhouhang Luo
Tao Yu
Hui Huang

View transformation robustness (VTR) is critical for deep-learning-based multi-view 3D object reconstruction models, which indicates the methods' stability under inputs with various view transformations. However, existing research seldom focused on view transformation robustness in multi-view 3D object reconstruction. One direct way to improve the models' VTR is to produce data with more view transformations and add them to model training. Recent progress on large vision models, particularly Stable Diffusion models, has provided great potential for generating 3D models or synthesizing novel view images with only a single image input. Directly deploying these models at inference consumes heavy computation resources and their robustness to view transformations is not guaranteed either. To fully utilize the power of Stable Diffusion models without extra inference computation burdens, we propose to generate novel views with Stable Diffusion models for better view transformation robustness. Instead of synthesizing random views, we propose a reconstruction error-guided view selection method, which considers the reconstruction errors' spatial distribution of the 3D predictions and chooses the views that could cover the reconstruction errors as much as possible. The methods are trained and tested on sets with large view transformations to validate the 3D reconstruction models' robustness to view transformations. Extensive experiments demonstrate that the proposed method can outperform state-of-the-art 3D reconstruction methods and other view transformation robustness comparison methods.

PDF Details DOI

AIJ Journal 2024 Journal Article

A multi-graph representation for event extraction

Hui Huang
Yanping Chen
Chuan Lin
Ruizhang Huang
Qinghua Zheng
Yongbin Qin

Event extraction has a trend in identifying event triggers and arguments in a unified framework, which has the advantage of avoiding the cascading failure in pipeline methods. The main problem is that joint models usually assume a one-to-one relationship between event triggers and arguments. It leads to the argument multiplexing problem, in which an argument mention can serve different roles in an event or shared by different events. To address this problem, we propose a multigraph-based event extraction framework. It allows parallel edges between any nodes, which is effective to represent semantic structures of an event. The framework enables the neural network to map a sentence(s) into a structurized semantic representation, which encodes multi-overlapped events. After evaluated on four public datasets, our method achieves the state-of-the-art performance, outperforming all compared models. Analytical experiments show that the multigraph representation is effective to address the argument multiplexing problem and helpful to advance the discriminability of the neural network for event extraction.

Details DOI

NeurIPS Conference 2024 Conference Paper

CRAYM: Neural Field Optimization via Camera RAY Matching

Liqiang Lin
Wenpeng Wu
Chi-Wing Fu
Hao Zhang
Hui Huang

We introduce camera ray matching (CRAYM) into the joint optimization of camera poses and neural fields from multi-view images. The optimized field, referred to as a feature volume, can be “probed” by the camera rays for novel view synthesis (NVS) and 3D geometry reconstruction. One key reason for matching camera rays, instead of pixels as in prior works, is that the camera rays can be parameterized by the feature volume to carry both geometric and photometric information. Multi-view consistencies involving the camera rays and scene rendering can be naturally integrated into the joint optimization and network training, to impose physically meaningful constraints to improve the final quality of both the geometric reconstruction and photorealistic rendering. We formulate our per-ray optimization and matched ray coherence by focusing on camera rays passing through keypoints in the input images to elevate both the efficiency and accuracy of scene correspondences. Accumulated ray features along the feature volume provide a means to discount the coherence constraint amid erroneous ray matching. We demonstrate the effectiveness of CRAYM for both NVS and geometry reconstruction, over dense- or sparse-view settings, with qualitative and quantitative comparisons to state-of-the-art alternatives.

PDF Details DOI

AAAI Conference 2024 Conference Paper

Feature Fusion from Head to Tail for Long-Tailed Visual Recognition

Mengke Li
Zhikai Hu
Yang Lu
Weichao Lan
Yiu-ming Cheung
Hui Huang

The imbalanced distribution of long-tailed data presents a considerable challenge for deep learning models, as it causes them to prioritize the accurate classification of head classes but largely disregard tail classes. The biased decision boundary caused by inadequate semantic information in tail classes is one of the key factors contributing to their low recognition accuracy. To rectify this issue, we propose to augment tail classes by grafting the diverse semantic information from head classes, referred to as head-to-tail fusion (H2T). We replace a portion of feature maps from tail classes with those belonging to head classes. These fused features substantially enhance the diversity of tail classes. Both theoretical analysis and practical experimentation demonstrate that H2T can contribute to a more optimized solution for the decision boundary. We seamlessly integrate H2T in the classifier adjustment stage, making it a plug-and-play module. Its simplicity and ease of implementation allow for smooth integration with existing long-tailed recognition methods, facilitating a further performance boost. Extensive experiments on various long-tailed benchmarks demonstrate the effectiveness of the proposed H2T. The source code is available at https://github.com/Keke921/H2T.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

Improving Visual Prompt Tuning by Gaussian Neighborhood Minimization for Long-Tailed Visual Recognition

Mengke Li
Ye Liu
Yang Lu
Yiqun Zhang
Yiu-ming Cheung
Hui Huang

Long-tailed visual recognition has received increasing attention recently. Despite fine-tuning techniques represented by visual prompt tuning (VPT) achieving substantial performance improvement by leveraging pre-trained knowledge, models still exhibit unsatisfactory generalization performance on tail classes. To address this issue, we propose a novel optimization strategy called Gaussian neighborhood minimization prompt tuning (GNM-PT), for VPT to address the long-tail learning problem. We introduce a novel Gaussian neighborhood loss, which provides a tight upper bound on the loss function of data distribution, facilitating a flattened loss landscape correlated to improved model generalization. Specifically, GNM-PT seeks the gradient descent direction within a random parameter neighborhood, independent of input samples, during each gradient update. Ultimately, GNM-PT enhances generalization across all classes while simultaneously reducing computational overhead. The proposed GNM-PT achieves state-of-the-art classification accuracies of 90. 3%, 76. 5%, and 50. 1% on benchmark datasets CIFAR100-LT (IR 100), iNaturalist 2018, and Places-LT, respectively. The source code is available at https: //github. com/Keke921/GNM-PT.

PDF Details DOI

EAAI Journal 2024 Journal Article

Mgformer: Multi-group transformer for multivariate time series classification

Jianfeng Wen
Nan Zhang
Xuzhe Lu
Zhongyi Hu
Hui Huang

Multivariate time series classification (MTSC) is a crucial task in data science, providing a foundation for analyzing and predicting complex, multi-dimensional data patterns. However, traditional MTSC methods are challenging to handle high-dimensional data effectively and necessitate complex feature engineering. Although deep learning methods have shown excellent performance in handling high-dimensional data, they have difficulty learning diverse temporal patterns in multivariate time series (MTS) and fail to capture deep channel-wise correlations. To this end, we propose a novel MTSC model based on Transformers named Mgformer, which has two basic structures, i. e. , the multi-group Transformer module and the channel attention mask module. The multi-group Transformer module combines temporal patching and multiple groups of Transformers to learn complex with diverse temporal patterns at different scales. An attention masking strategy is employed by the channel attention mask module to improve the model’s capacity to learn channel-wise correlations and decrease information loss during training. Experimental results on 27 benchmark MTS datasets show that Mgformer is better than state-of-the-art MTSC methods.

Details DOI

AAAI Conference 2024 Conference Paper

Multi-View People Detection in Large Scenes via Supervised View-Wise Contribution Weighting

Qi Zhang
Yunfei Gong
Daijie Chen
Antoni B. Chan
Hui Huang

Recent deep learning-based multi-view people detection (MVD) methods have shown promising results on existing datasets. However, current methods are mainly trained and evaluated on small, single scenes with a limited number of multi-view frames and fixed camera views. As a result, these methods may not be practical for detecting people in larger, more complex scenes with severe occlusions and camera calibration errors. This paper focuses on improving multi-view people detection by developing a supervised view-wise contribution weighting approach that better fuses multi-camera information under large scenes. Besides, a large synthetic dataset is adopted to enhance the model's generalization ability and enable more practical evaluation and comparison. The model's performance on new testing scenes is further improved with a simple domain adaptation technique. Experimental results demonstrate the effectiveness of our approach in achieving promising cross-scene multi-view people detection performance.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

Reconstruct and Match: Out-of-Distribution Robustness via Topological Homogeneity

Chaoqi Chen
Luyao Tang
Hui Huang

Since deep learning models are usually deployed in non-stationary environments, it is imperative to improve their robustness to out-of-distribution (OOD) data. A common approach to mitigate distribution shift is to regularize internal representations or predictors learned from in-distribution (ID) data to be domain invariant. Past studies have primarily learned pairwise invariances, ignoring the intrinsic structure and high-order dependencies of the data. Unlike machines, human recognizes objects by first dividing them into major components and then identifying the topological relation of these components. Motivated by this, we propose Reconstruct and Match (REMA), a general learning framework for object recognition tasks to endow deep models with the capability of capturing the topological homogeneity of objects without human prior knowledge or fine-grained annotations. To identify major components from objects, REMA introduces a selective slot-based reconstruction module to dynamically map dense pixels into a sparse and discrete set of slot vectors in an unsupervised manner. Then, to model high-order dependencies among these components, we propose a hypergraph-based relational reasoning module that models the intricate relations of nodes (slots) with structural constraints. Experiments on standard benchmarks show that REMA outperforms state-of-the-art methods in OOD generalization and test-time adaptation settings.

PDF Details DOI

ICML Conference 2024 Conference Paper

TERD: A Unified Framework for Safeguarding Diffusion Models Against Backdoors

Yichuan Mo
Hui Huang
Mingjie Li 0007
Ang Li
Yisen Wang 0001

Diffusion models have achieved notable success in image generation, but they remain highly vulnerable to backdoor attacks, which compromise their integrity by producing specific undesirable outputs when presented with a pre-defined trigger. In this paper, we investigate how to protect diffusion models from this dangerous threat. Specifically, we propose TERD, a backdoor defense framework that builds unified modeling for current attacks, which enables us to derive an accessible reversed loss. A trigger reversion strategy is further employed: an initial approximation of the trigger through noise sampled from a prior distribution, followed by refinement through differential multi-step samplers. Additionally, with the reversed trigger, we propose backdoor detection from the noise space, introducing the first backdoor input detection approach for diffusion models and a novel model detection algorithm that calculates the KL divergence between reversed and benign distributions. Extensive evaluations demonstrate that TERD secures a 100% True Positive Rate (TPR) and True Negative Rate (TNR) across datasets of varying resolutions. TERD also demonstrates nice adaptability to other Stochastic Differential Equation (SDE)-based models. Our code is available at https: //github. com/PKU-ML/TERD.

Details

JMLR Journal 2021 Journal Article

Consensus-Based Optimization on the Sphere: Convergence to Global Minimizers and Machine Learning

Massimo Fornasier
Lorenzo Pareschi
Hui Huang
Philippe Sünnen

We investigate the implementation of a new stochastic Kuramoto-Vicsek-type model for global optimization of nonconvex functions on the sphere. This model belongs to the class of Consensus-Based Optimization. In fact, particles move on the sphere driven by a drift towards an instantaneous consensus point, which is computed as a convex combination of particle locations, weighted by the cost function according to Laplace's principle, and it represents an approximation to a global minimizer. The dynamics is further perturbed by a random vector field to favor exploration, whose variance is a function of the distance of the particles to the consensus point. In particular, as soon as the consensus is reached the stochastic component vanishes. The main results of this paper are about the proof of convergence of the numerical scheme to global minimizers provided conditions of well-preparation of the initial datum. The proof combines previous results of mean-field limit with a novel asymptotic analysis, and classical convergence results of numerical methods for SDE. We present several numerical experiments, which show that the algorithm proposed in the present paper scales well with the dimension and is extremely versatile. To quantify the performances of the new approach, we show that the algorithm is able to perform essentially as good as ad hoc state of the art methods in challenging problems in signal processing and machine learning, namely the phase retrieval problem and the robust subspace detection. [abs] [ pdf ][ bib ] [ code ] &copy JMLR 2021. ( edit, beta )

PDF Details

IJCAI Conference 2021 Conference Paper

Improving Stylized Neural Machine Translation with Iterative Dual Knowledge Transfer

Xuanxuan Wu
Jian Liu
Xinjie Li
Jinan Xu
Yufeng Chen
Yujie Zhang
Hui Huang

Stylized neural machine translation (NMT) aims to translate sentences of one style into sentences of another style, which is essential for the application of machine translation in a real-world scenario. However, a major challenge in this task is the scarcity of high-quality parallel data which is stylized paired. To address this problem, we propose an iterative dual knowledge transfer framework that utilizes informal training data of machine translation and formality style transfer data to create large-scale stylized paired data, for the training of stylized machine translation model. Specifically, we perform bidirectional knowledge transfer between translation model and text style transfer model iteratively through knowledge distillation. Then, we further propose a data-refinement module to process the noisy synthetic parallel data generated during knowledge transfer. Experiment results demonstrate the effectiveness of our method, achieving an improvement over the existing best model by 5 BLEU points on MTFC dataset. Meanwhile, extensive analyses illustrate our method can also improve the accuracy of formality style transfer.

PDF Details DOI

ICRA Conference 2021 Conference Paper

Origami-Inspired Snap-through Bistability in Parallel and Curved Mechanisms Through the Inflection of Degree Four Vertexes

Bok Seng Yeow
Catherine Jiayi Cai
Manivannan Sivaperuman Kalairaj
Feng Wen Hoo
Zu Xuan Lee
Janice Chui Shien Tan
Jian Rong Ho
Vienna Minhui Ma

Origami, the art of folding paper, can impart useful design inspirations to the creation of mechanical structures and mechanisms. Bistability is a useful property for origami designs, which can help compartmentalize different actuations and stiffness tuning regimes. Given the benefits of bistability, we investigated origami designs used to build robots and deployable structures. We show snap-through bistable designs applied to parallel and curved mechanisms, which are of value to robotic mechanical design. The designs proposed were investigated through geometry analysis and stress-strain experiments. The origami designs were modified to show that the mechanical properties of our bistable designs can be modulated. Initial actuation utilized magnetic and tendon-driven mechanisms for the parallel and curved structures, respectively. We anticipate that these bistable snap-through designs can contribute to deployable mechanisms and give such devices additional capabilities in mechanical response tuning.

Details

JBHI Journal 2021 Journal Article

SCNET: A Novel UGI Cancer Screening Framework Based on Semantic-Level Multimodal Data Fusion

Shuai Ding
Hui Huang
Zhenmin Li
Xiao Liu
Shanlin Yang

Upper gastrointestinal (UGI) cancer has been identified as one of the ten most common causes of cancer deaths globally. UGI cancer screening is critical to improving the survival rate of UGI cancer patients. While many approaches to UGI cancer screening rely on single-modality data such as gastroscope imaging, limited studies have been dedicated to UGI cancer screening exploiting multisource and multimodal medical data, which could potentially lead to improved screening results. In this paper, we propose semantic-level cancer-screening network (SCNET), a framework for UGI cancer screening based on semantic-level multimodal upper gastrointestinal data fusion. Specifically, the proposed SCNET consists of a gastrointestinal image recognition flow and a textual medical record processing flow. High-level features of upper gastrointestinal data are extracted by identifying effective feature channels according to the correlation between the textual features and the spatial structure of the image features. The final screening results are obtained after the data fusion step. The experimental results show that the improvement of our approach over the state-of-the-art ones reached 4. 01% in average. The source code of SCNET is available at https://github.com/netflymachine/SCNET.

Details DOI

AAAI Conference 2020 Conference Paper

EFANet: Exchangeable Feature Alignment Network for Arbitrary Style Transfer

Zhijie Wu
Chunjin Song
Yang Zhou
Minglun Gong
Hui Huang

Style transfer has been an important topic both in computer vision and graphics. Since the seminal work of Gatys et al. ﬁrst demonstrates the power of stylization through optimization in the deep feature space, quite a few approaches have achieved real-time arbitrary style transfer with straightforward statistic matching techniques. In this work, our key observation is that only considering features in the input style image for the global deep feature statistic matching or local patch swap may not always ensure a satisfactory style transfer; see e. g. , Figure 1. Instead, we propose a novel transfer framework, EFANet, that aims to jointly analyze and better align exchangeable features extracted from the content and style image pair. In this way, the style feature from the style image seeks for the best compatibility with the content information in the content image, leading to more structured stylization results. In addition, a new whitening loss is developed for purifying the computed content features and better fusion with styles in feature space. Qualitative and quantitative experiments demonstrate the advantages of our approach.

PDF Details

JMLR Journal 2020 Journal Article

Semi-parametric Learning of Structured Temporal Point Processes

Ganggang Xu
Ming Wang
Jiangze Bian
Hui Huang
Timothy R. Burch
Sandro C. Andrade
Jingfei Zhang
Yongtao Guan

We propose a general framework of using a multi-level log-Gaussian Cox process to model repeatedly observed point processes with complex structures; such type of data has become increasingly available in various areas including medical research, social sciences, economics, and finance due to technological advances. A novel nonparametric approach is developed to efficiently and consistently estimate the covariance functions of the latent Gaussian processes at all levels. To predict the functional principal component scores, we propose a consistent estimation procedure by maximizing the conditional likelihood of super-positions of point processes. We further extend our procedure to the bivariate point process case in which potential correlations between the processes can be assessed. Asymptotic properties of the proposed estimators are investigated, and the effectiveness of our procedures is illustrated through a simulation study and an application to a stock trading dataset. [abs] [ pdf ][ bib ] &copy JMLR 2020. ( edit, beta )

PDF Details

NeurIPS Conference 2019 Conference Paper

ETNet: Error Transition Network for Arbitrary Style Transfer

Chunjin Song
Zhijie Wu
Yang Zhou
Minglun Gong
Hui Huang

Numerous valuable efforts have been devoted to achieving arbitrary style transfer since the seminal work of Gatys et al. However, existing state-of-the-art approaches often generate insufficiently stylized results under challenging cases. We believe a fundamental reason is that these approaches try to generate the stylized result in a single shot and hence fail to fully satisfy the constraints on semantic structures in the content images and style patterns in the style images. Inspired by the works on error-correction, instead, we propose a self-correcting model to predict what is wrong with the current stylization and refine it accordingly in an iterative manner. For each refinement, we transit the error features across both the spatial and scale domain and invert the processed features into a residual image, with a network we call Error Transition Network (ETNet). The proposed model improves over the state-of-the-art methods with better semantic structures and more adaptive style pattern details. Various qualitative and quantitative experiments show that the key concept of both progressive strategy and error-correction leads to better results. Code and models are available at https: //github. com/zhijieW94/ETNet.

PDF Details

JBHI Journal 2013 Journal Article

A Watermarking-Based Medical Image Integrity Control System and an Image Moment Signature for Tampering Characterization

Gouenou Coatrieux
Hui Huang
Huazhong Shu
Limin Luo
Christian Roux

In this paper, we present a medical image integrity verification system to detect and approximate local malevolent image alterations (e. g. , removal or addition of lesions) as well as identifying the nature of a global processing an image may have undergone (e. g. , lossy compression, filtering, etc.). The proposed integrity analysis process is based on nonsignificant region watermarking with signatures extracted from different pixel blocks of interest, which are compared with the recomputed ones at the verification stage. A set of three signatures is proposed. The first two devoted to detection and modification location are cryptographic hashes and checksums, while the last one is issued from the image moment theory. In this paper, we first show how geometric moments can be used to approximate any local modification by its nearest generalized 2-D Gaussian. We then demonstrate how ratios between original and recomputed geometric moments can be used as image features in a classifier-based strategy in order to determine the nature of a global image processing. Experimental results considering both local and global modifications in MRI and retina images illustrate the overall performances of our approach. With a pixel block signature of about 200 bit long, it is possible to detect, to roughly localize, and to get an idea about the image tamper.

Details DOI