Author name cluster

Tong Sun

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

12 papers

2 author rows

EAAI Journal 2026 Journal Article

A deformable convolution and shuffle attention enhanced network for surface defect detection

Yinggan Tang
Xueguang Lv
Tong Sun

Steel surface defect detection is vital for industrial quality control but remains challenging due to irregular shapes and complex backgrounds. In this paper, we propose a novel network, named deformable convolution and shuffle attention enhanced network (DSA-Net), to address these challenges. First, we design a Cross-Stage Partial Block with Kernel Size 2 and Deformable Convolution and Shuffle Attention (C3k2-DSA) module that integrates deformable convolution with shuffle attention to adaptively model irregular defect geometries while suppressing background interference, thereby enhancing the focus on critical defect regions. Second, a bidirectional multi-scale feature fusion neck is constructed by combining top-down and bottom-up pathways, enabling effective interaction between fine-grained details and high-level semantic features for accurate multi-scale defect detection. Third, we introduce a Convolutional Block with Large Separable Kernel Attention (C2LSKA) module that combines cross-stage partial connections with large separable kernel attention to efficiently capture long-range dependencies and strengthen multi-scale feature representation. Finally, the Unified Intersection over Union (UIoU) loss function is adopted to dynamically redistribute regression weights, jointly optimizing bounding box overlap, quality awareness, and confidence weighting, thereby improving localization accuracy. Experimental results demonstrate that DSA-Net achieves 81. 0% mean Average Precision (mAP) on Northeastern University surface defect database for detection (NEU-DET), 72. 4% on Metallic Surface Defect Datasets: The new Benchmark (GC10-DET), and 73. 3% on Severstal, outperforming You Only Look Once version 11 small (YOLOv11s) by 5. 3%, 4. 8%, and 8. 3%, respectively. These results confirm the superior accuracy and robustness of the proposed method for steel surface defect detection.

Details DOI

AAAI Conference 2026 Conference Paper

OIDA-QA: A Multimodal Benchmark for Analyzing the Opioid Industry Documents Archive

Xuan Shen
Brian Wingenroth
Zichao Wang
Jason Kuen
Wanrong Zhu
Ruiyi Zhang
Yiwei Wang
Lichun Ma

The opioid crisis represents a significant moment in public health that reveals systemic shortcomings across regulatory systems, healthcare practices, corporate governance, and public policy. Analyzing how these interconnected systems simultaneously failed to protect public health requires innovative analytic approaches for exploring the vast amounts of data and documents disclosed in the UCSF-JHU Opioid Industry Documents Archive (OIDA). The complexity, multimodal nature, and specialized characteristics of these healthcare-related legal and corporate documents necessitate more advanced methods and models tailored to specific data types and detailed annotations, ensuring the precision and professionalism in the analysis. In this paper, we tackle this challenge by organizing the original dataset according to document attributes and constructing a benchmark with 400k training documents and 10k for testing. From each document, we extract rich multimodal information—including textual content, visual elements, and layout structures—to capture a comprehensive range of features. Using multiple AI models, we then generate a large-scale dataset comprising 360k training QA pairs and 10k testing QA pairs. Building on this foundation, we develop domain-specific multimodal Large Language Models (LLMs) and explore the impact of multimodal inputs on task performance. To further enhance response accuracy, we incorporate historical QA pairs as contextual grounding for answering current queries. Additionally, we incorporate page references within the answers and introduce an importance-based page classifier, further improving the precision and relevance of the information provided. Preliminary results indicate the improvements with our AI assistant in document information extraction and question-answering tasks.

PDF Details DOI

AAAI Conference 2025 Conference Paper

Numerical Pruning for Efficient Autoregressive Models

Xuan Shen
Zhao Song
Yufa Zhou
Bo Chen
Jing Liu
Ruiyi Zhang
Ryan A. Rossi
Hao Tan

Transformers have emerged as the leading architecture in deep learning, proving to be versatile and highly effective across diverse domains beyond language and image processing. However, their impressive performance often incurs high computational costs due to their substantial model size. This paper focuses on compressing decoder-only transformer-based autoregressive models through structural weight pruning to improve the model efficiency while preserving performance for both language and image generation tasks. Specifically, we propose a training-free pruning method that calculates a numerical score with Newton's method for the Attention and MLP modules, respectively. Besides, we further propose another compensation algorithm to recover the pruned model for better performance. To verify the effectiveness of our method, we provide both theoretical support and extensive experiments. Our experiments show that our method achieves state-of-the-art performance with reduced memory usage and faster generation speeds on GPUs.

PDF Details DOI

ICRA Conference 2025 Conference Paper

ROD: RGB-Only Fast and Efficient Off-Road Freespace Detection

Tong Sun
Hongliang Ye
Jilin Mei
Liang Chen
Fangzhou Zhao
Leiqiang Zong
Yu Hu 0001

Off-road freespace detection is more challenging than on-road scenarios because of the blurred boundaries of traversable areas. Previous state-of-the-art (SOTA) methods employ multi-modal fusion of RGB images and LiDAR data. However, due to the significant increase in inference time when calculating surface normal maps from LiDAR data, multimodal methods are not suitable for real-time applications, particularly in real-world scenarios where higher FPS is required compared to slow navigation. This paper presents a novel RGB-only approach for off-road freespace detection, named ROD, eliminating the reliance on LiDAR data and its computational demands. Specifically, we utilize a pre-trained Vision Transformer (ViT) to extract rich features from RGB images. Additionally, we design a lightweight yet efficient decoder, which together improve both precision and inference speed. ROD establishes a new SOTA on ORFD and RELLIS-3D datasets, as well as an inference speed of 50 FPS, significantly outperforming prior models. Our code will be available at https://github.com/STLIFE97/offroad_roadseg.

Details

JBHI Journal 2024 Journal Article

Collaborative Transfer Network for Multi-Classification of Breast Cancer Histopathological Images

Liangliang Liu
Ying Wang
Pei Zhang
Hongbo Qiao
Tong Sun
Hui Zhang
Xue Xu
Hongcai Shang

The incidence of breast cancer is increasing rapidly around the world. Accurate classification of the breast cancer subtype from hematoxylin and eosin images is the key to improve the precision of treatment. However, the high consistency of disease subtypes and uneven distribution of cancer cells seriously affect the performance of multi-classification methods. Furthermore, it is difficult to apply existing classification methods to multiple datasets. In this article, we propose a collaborative transfer network (CTransNet) for multi-classification of breast cancer histopathological images. CTransNet consists of a transfer learning backbone branch, a residual collaborative branch, and a feature fusion module. The transfer learning branch adopts the pre-trained DenseNet structure to extract image features from ImageNet. The residual branch extracts target features from pathological images in a collaborative manner. The feature fusion strategy of optimizing these two branches is used to train and fine-tune CTransNet. Experiments show that CTransNet achieves 98. 29% classification accuracy on the public BreaKHis breast cancer dataset, exceeding the performance of state-of-the-art methods. Visual analysis is carried out under the guidance of oncologists. Based on the training parameters of the BreaKHis dataset, CTransNet achieves superior performance on other two public breast cancer datasets (breast-cancer-grade-ICT and ICIAR2018_BACH_Challenge), indicating that CTransNet has good generalization performance.

Details DOI

JBHI Journal 2024 Journal Article

H-Net: Heterogeneous Neural Network for Multi-Classification of Neuropsychiatric Disorders

Liangliang Liu
Jinpu Xie
Jing Chang
Zhihong Liu
Tong Sun
Hongbo Qiao
Gongbo Liang
Wei Guo

Clinical studies have proved that both structural magnetic resonance imaging (sMRI) and functional magnetic resonance imaging (fMRI) are implicitly associated with neuropsychiatric disorders (NDs), and integrating multi-modal to the binary classification of NDs has been thoroughly explored. However, accurately classifying multiple classes of NDs remains a challenge due to the complexity of disease subclass. In our study, we develop a heterogeneous neural network (H-Net) that integrates sMRI and fMRI modes for classifying multi-class NDs. To account for the differences between the two modes, H-Net adopts a heterogeneous neural network strategy to extract information from each mode. Specifically, H-Net includes an multi-layer perceptron based (MLP-based) encoder, a graph attention network based (GAT-based) encoder, and a cross-modality transformer block. The MLP-based and GAT-based encoders extract semantic features from sMRI and features from fMRI, respectively, while the cross-modality transformer block models the attention of two types of features. In H-Net, the proposed MLP-mixer block and cross-modality alignment are powerful tools for improving the multi-classification performance of NDs. H-Net is validate on the public dataset (CNP), where H-Net achieves 90% classification accuracy in diagnosing multi-class NDs. Furthermore, we demonstrate the complementarity of the two MRI modalities in improving the identification of multi-class NDs. Both visual and statistical analyses show the differences between ND subclasses.

Details DOI

NeurIPS Conference 2023 Conference Paper

Label-Retrieval-Augmented Diffusion Models for Learning from Noisy Labels

Jian Chen
Ruiyi Zhang
Tong Yu
Rohan Sharma
Zhiqiang Xu
Tong Sun
Changyou Chen

Learning from noisy labels is an important and long-standing problem in machine learning for real applications. One of the main research lines focuses on learning a label corrector to purify potential noisy labels. However, these methods typically rely on strict assumptions and are limited to certain types of label noise. In this paper, we reformulate the label-noise problem from a generative-model perspective, i. e. , labels are generated by gradually refining an initial random guess. This new perspective immediately enables existing powerful diffusion models to seamlessly learn the stochastic generative process. Once the generative uncertainty is modeled, we can perform classification inference using maximum likelihood estimation of labels. To mitigate the impact of noisy labels, we propose the L abel- R etrieval- A ugmented (LRA) diffusion model, which leverages neighbor consistency to effectively construct pseudo-clean labels for diffusion training. Our model is flexible and general, allowing easy incorporation of different types of conditional information, e. g. , use of pre-trained models, to further boost model performance. Extensive experiments are conducted for evaluation. Our model achieves new state-of-the-art (SOTA) results on all the standard real-world benchmark datasets. Remarkably, by incorporating conditional information from the powerful CLIP model, our method can boost the current SOTA accuracy by 10-20 absolute points in many cases. Code is available: https: //anonymous. 4open. science/r/LRA-diffusion-5F2F

PDF Details

AAAI Conference 2022 Conference Paper

TiGAN: Text-Based Interactive Image Generation and Manipulation

Yufan Zhou
Ruiyi Zhang
Jiuxiang Gu
Chris Tensmeyer
Tong Yu
Changyou Chen
Jinhui Xu
Tong Sun

Using natural-language feedback to guide image generation and manipulation can greatly lower the required efforts and skills. This topic has received increased attention in recent years through refinement of Generative Adversarial Networks (GANs); however, most existing works are limited to singleround interaction, which is not reflective of real world interactive image editing workflows. Furthermore, previous works dealing with multi-round scenarios are limited to predefined feedback sequences, which is also impractical. In this paper, we propose a novel framework for Text-based interactive image generation and manipulation (TiGAN) that responds to users’ natural-language feedback. TiGAN utilizes the powerful pre-trained CLIP model to understand users’ naturallanguage feedback and exploits contrastive learning for a better text-to-image mapping. To maintain the image consistency during interactions, TiGAN generates intermediate feature vectors aligned with the feedback and selectively feeds these vectors to our proposed generative model. Empirical results on several datasets show that TiGAN improves both interaction efficiency and image quality while better avoids undesirable image manipulation during interactions.

PDF Details

NeurIPS Conference 2021 Conference Paper

UniDoc: Unified Pretraining Framework for Document Understanding

Jiuxiang Gu
Jason Kuen
Vlad I Morariu
Handong Zhao
Rajiv Jain
Nikolaos Barmpalios
Ani Nenkova
Tong Sun

Document intelligence automates the extraction of information from documents and supports many business applications. Recent self-supervised learning methods on large-scale unlabeled document datasets have opened up promising directions towards reducing annotation efforts by training models with self-supervised objectives. However, most of the existing document pretraining methods are still language-dominated. We present UDoc, a new unified pretraining framework for document understanding. UDoc is designed to support most document understanding tasks, extending the Transformer to take multimodal embeddings as input. Each input element is composed of words and visual features from a semantic region of the input document image. An important feature of UDoc is that it learns a generic representation by making use of three self-supervised losses, encouraging the representation to model sentences, learn similarities, and align modalities. Extensive empirical analysis demonstrates that the pretraining procedure learns better joint representations and leads to improvements in downstream tasks.

PDF Details

NeurIPS Conference 2020 Conference Paper

Self-Supervised Relationship Probing

Jiuxiang Gu
Jason Kuen
Shafiq Joty
Jianfei Cai
Vlad Morariu
Handong Zhao
Tong Sun

Structured representations of images that model visual relationships are beneficial for many vision and vision-language applications. However, current human-annotated visual relationship datasets suffer from the long-tailed predicate distribution problem which limits the potential of visual relationship models. In this work, we introduce a self-supervised method that implicitly learns the visual relationships without relying on any ground-truth visual relationship annotations. Our method relies on 1) intra- and inter-modality encodings to respectively model relationships within each modality separately and jointly, and 2) relationship probing, which seeks to discover the graph structure within each modality. By leveraging masked language modeling, contrastive learning, and dependency tree distances for self-supervision, our method learns better object features as well as implicit visual relationships. We verify the effectiveness of our proposed method on various vision-language tasks that benefit from improved visual relationship understanding.

PDF Details

IJCAI Conference 2019 Conference Paper

CLVSA: A Convolutional LSTM Based Variational Sequence-to-Sequence Model with Attention for Predicting Trends of Financial Markets

Jia Wang
Tong Sun
Benyuan Liu
Yu Cao
Hongwei Zhu

Financial markets are a complex dynamical system. The complexity comes from the interaction between a market and its participants, in other words, the integrated outcome of activities of the entire participants determines the markets trend, while the markets trend affects activities of participants. These interwoven interactions make financial markets keep evolving. Inspired by stochastic recurrent models that successfully capture variability observed in natural sequential data such as speech and video, we propose CLVSA, a hybrid model that consists of stochastic recurrent networks, the sequence-to-sequence architecture, the self- and inter-attention mechanism, and convolutional LSTM units to capture variationally underlying features in raw financial trading data. Our model outperforms basic models, such as convolutional neural network, vanilla LSTM network, and sequence-to-sequence model with attention, based on backtesting results of six futures from January 2010 to December 2017. Our experimental results show that, by introducing an approximate posterior, CLVSA takes advantage of an extra regularizer based on the Kullback-Leibler divergence to prevent itself from overfitting traps.

PDF Details

TIST Journal 2012 Journal Article

Mining the “Voice of the Customer” for Business Prioritization

Wei Peng
Tong Sun
Shriram Revankar
Tao Li

To gain competitiveness and sustained growth in the 21st century, most businesses are on a mission to become more customer-centric. In order to succeed in this endeavor, it is crucial not only to synthesize and analyze the VOC (the VO ice of the C ustomer) data (i.e., the feedbacks or requirements raised by customers), but also to quickly turn these data into actionable knowledge. Although there are many technologies being developed in this complex problem space, most existing approaches in analyzing customer requests are ad hoc, time-consuming, error-prone, people-based processes which hardly scale well as the quantity of customer information explodes. This often results in the slow response to customer requests. In this article, in order to mine VOC to extract useful knowledge for the best product or service quality, we develop a hybrid framework that integrates domain knowledge with data-driven approaches to analyze the semi-structured customer requests. The framework consists of capturing functional features, discovering the overlap or correlation among the features, and identifying the evolving feature trend by using the knowledge transformation model. In addition, since understanding the relative importance of the individual customer request is very critical and has a direct impact on the effective prioritization in the development process, we develop a novel semantic enhanced link-based ranking (SELRank) algorithm for relatively rating/ranking both customer requests and products. The framework has been successfully applied on Xerox Office Group Feature Enhancement Requirements (XOG FER) datasets to analyze customer requests.

Details DOI