Author name cluster

Huadong Ma

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

23 papers

2 author rows

AAAI Conference 2026 Conference Paper

HDRMovieformer: A Transformer Framework and Benchmark for Cinematic SDR-to-HDR Conversion

Xianwei Li
Huiyuan Fu
Chuanming Wang
Huadong Ma

With the growing prevalence of HDR-capable cinema venues such as Cinity LED theaters, there is an increasing demand to convert existing Standard Dynamic Range (SDR) films into High Dynamic Range (HDR) formats for theatrical presentation. However, existing SDR-to-HDR conversion methods are primarily tailored for consumer-grade content such as television and therefore fall short of the stringent requirements of professional cinematic material. To bridge this gap, we present HDRMovie7K, the first large-scale, lossless dataset of cinematic SDR-HDR frame pairs sourced from professional Digital Cinema Distribution Master (DCDM) workflows. Based on this foundation, we introduce HDRMovieformer, a transformer-based framework featuring a Luminance Estimator module for luminance guidance, a Luminance-Guided Multi-Head Self-Attention to focus on critical fine-detail recovery, and a Chroma Refiner for color accuracy, optimized with a novel Wide Color Gamut Loss. To further evaluate our model in online streaming media scenarios, we introduce HDRMovie1K, a dataset curated from publicly available HDR film clips. Extensive experiments on both HDRMovie7K and HDRMovie1K demonstrate that our method achieves state-of-the-art performance.

PDF Details DOI

AAAI Conference 2026 Conference Paper

Improving Batch Normalization with Test-Time Adaptation for Robust Object Detection in Self-Driving

Dacheng Liao
Mengshi Qi
Liang Liu
Huadong Ma

In open real-world autonomous driving scenarios, challenges such as sensor failure and extreme weather hinder the generalization of current autonomous driving perception models to these unseen domain, due to the domain shifts between the test and training data. As the parameter scale of autonomous driving perception models grows, traditional test-time adaptation (TTA) methods become unstable and often degrade model performance in most scenarios. To address these challenges, this paper proposes two new robust methods to improve the Batch Normalization with TTA for object detection in autonomous driving: (1) We introduce a new LearnableBN layer based on Geometric Confidence Maximization and Entropy Minimization. Specifically, we modify the traditional BN layer by incorporating auxiliary learnable parameters, which enables the BN layer to dynamically update the statistics according to the different input data. (2) We propose a novel semantic-consistency based dual-stage adaptation strategy, which encourages the model to iteratively search for the optimal solution and eliminates unstable samples during the adaptation process. Extensive experiments on the NuScenes-C dataset shows that our method achieves a maximum improvement of about 10\% using BEVFormer as the baseline across six corruption types and three levels of severity.

PDF Details DOI

AAAI Conference 2026 Conference Paper

Regression over Classification: Assessing Image Aesthetics via Multimodal Large Language Models

Xingyuan Ma
Shuai He
Anlong Ming
Haobin Zhong
Huadong Ma

Image Aesthetics Assessment (IAA) evaluates visual quality through user-centered perceptual analysis and can guide various applications. Recent advances in Multimodal Large Language Models (MLLMs) have sparked interest in adapting them for IAA. However, two critical limitations persist in applying MLLMs to IAA: 1) the tokenization strategy leads to insensitivity to scores, and 2) the classification-based decoding mechanisms introduce score quantization errors. Current MLLM-based IAA methods treat the task as coarse rating classification followed by probability-to-score mapping, which loses fine-grained information. To address these challenges, we propose ROC4MLLM, offering complementary solutions from two perspectives:1) Representation: We separate scores from the word token space to avoid tokenizing scores as text. An independent position token bridges these spaces, improving the sensitivity of the model to score positions in text. 2) Computation: We apply distinct loss functions for text and score predictions to enhance the sensitivity of the model to score gradients. Decoupling scores from text ensures effective supervision while preventing interference between scores and text in the loss computation. Extensive experiments across five datasets demonstrate that ROC4MLLM achieves state-of-the-art performance without requiring additional training data. Additionally, its plug-and-play design ensures seamless integration with existing MLLMs, boosting their IAA performance.

PDF Details DOI

AAAI Conference 2026 Conference Paper

Selective Diffusion Distillation for Real-World High-Scale Image Super-Resolution

Wenli Zheng
Huiyuan Fu
Zekai Xu
Xin Wang
Huadong Ma

High-scale image super-resolution (SR) has become increasingly important with the rapid growth of mobile devices and high-resolution displays. However, current SR methods primarily focus on lower scales and generalize poorly to high-scale scenarios due to severe information loss and complex real-world degradations. In this paper, we propose a novel Selective Diffusion Distillation (SDD) framework for real-world high-scale SR, which distills reliable knowledge from a low-scale diffusion teacher to a high-scale student. Specifically, considering severe information loss in high-scale inputs, directly distilling from low-scale models may result in feature misalignment. To address this, we introduce a Degradation-aware Metric Learning (DML) approach to align feature distributions across different degradation levels. In addition, since the diffusion-based teacher may hallucinate artifacts in ambiguous regions, blindly imitating these unreliable outputs can degrade the student’s fidelity. To tackle this, we propose a Region-aware Selective Distillation (RSD) strategy to filter out uncertain predictions and adaptively supervise only on reliable areas. To evaluate the effectiveness of our method, we introduce Real-UltraSR, a new real-world benchmark that contains diverse high-scale LR-HR pairs, including x8, x10, x12, and x14. Extensive experiments demonstrate that our SDD framework achieves state-of-the-art performance across multiple benchmarks.

PDF Details DOI

ECAI Conference 2025 Conference Paper

A Dynamic Service Offloading Algorithm Based on Lyapunov Optimization in Edge Computing

Peiyan Yuan
Ming Li 0004
Chenyang Wang 0001
Ledong An
Xiaoyan Zhao 0001
Junna Zhang
Xiang-Yang Li 0001
Huadong Ma

This study investigates the trade-off between system stability and offloading cost in collaborative edge computing. While collaborative offloading among multiple edge servers enhances resource utilization, existing methods often overlook the role of queue stability in overall system performance. To address this, a multi-hop data transmission model is developed, along with a cost model that captures both energy consumption and delay. A time-varying queue model is then introduced to maintain system stability. Based on Lyapunov optimization, a dynamic offloading algorithm (LDSO) is proposed to minimize offloading cost while ensuring long-term stability. Theoretical analysis and experimental results verify that the proposed LDSO achieves significant improvements in both cost efficiency and system stability compared to the state-of-the-art.

Details

TIST Journal 2025 Journal Article

Self-optimizing Teacher and Auto-matching Student Framework for Change-point Representation Learning in Time Series Forecasting

Jinxiao Fan
Pengfei Wang
Liang Liu
Huadong Ma

Real-world time series data is inherently complex, noisy, and exhibits abrupt changes, posing various challenges in data modeling. Given the ubiquity and importance of time-series data, accurately forecasting change points, instead of the overall predictive performance, has become increasingly attractive as it assists in risk mitigation and loss prevention. In this task, we argue that the past and future interactions involving the target points determine the comprehensive structure contributing to abrupt changes. However, traditional left-to-right auto-regressive approaches only consider the historical sequence, resulting in a flawed learning process and limited performance. In this paper, we extend the teacher-student learning and propose a novel S elf-optimizing T eacher and A uto-matching S tudent framework (named ST-AS) to predict change points in time series data. Our framework models change point representations specific to the target points by integrating future knowledge while avoiding data leakage. Specifically, we design a Gumbel-enhanced filter for our self-optimizing teacher, which constructs selected and filtered sub-groups to derive discriminative representations using a positive-unlabeled learning strategy. Given this well-trained teacher, we propose an adaptive pattern matcher for our auto-matching student model, which learns missing information by automatically aligning relevant features. After that, a novel two-stage dual-guided learning process is then designed to mimic teacher’s decision-making behavior and enhance student’s excavate capability. Finally, we conduct extensive experiments on four real-world datasets to demonstrate that our proposed ST-AS exhibits significantly better prediction performance compared to existing state-of-the-art alternatives.

Details DOI

NeurIPS Conference 2025 Conference Paper

Synergistic Tensor and Pipeline Parallelism

Mengshi Qi
Jiaxuan Peng
Jie Zhang
Juan Zhu
Yong Li
Huadong Ma

In the machine learning system, the hybrid model parallelism combining tensor parallelism (TP) and pipeline parallelism (PP) has become the dominant solution for distributed training of Large Language Models~(LLMs) and Multimodal LLMs (MLLMs). However, TP introduces significant collective communication overheads, while PP suffers from synchronization inefficiencies such as pipeline bubbles. Existing works primarily address these challenges from isolated perspectives, focusing either on overlapping TP communication or on flexible PP scheduling to mitigate pipeline bubbles. In this paper, we propose a new synergistic tensor and pipeline parallelism schedule that simultaneously reduces both types of bubbles. Our proposed schedule decouples the forward and backward passes in PP into fine-grained computation units, which are then braided to form a composite computation sequence. This compositional structure enables near-complete elimination of TP-related bubbles. Building upon this structure, we further design the PP schedule to minimize PP bubbles. Experimental results demonstrate that our approach improves training throughput by up to 12\% for LLMs and 16\% for MLLMs compared to existing scheduling methods. Our source code is avaiable at https: //github. com/MICLAB-BUPT/STP.

PDF Details

AAAI Conference 2025 Conference Paper

Towards Efficient Object Re-Identification with a Novel Cloud-Edge Collaborative Framework

Chuanming Wang
Yuxin Yang
Mengshi Qi
Huanhuan Zhang
Huadong Ma

Object re-identification (ReID) is committed to searching for objects of the same identity across cameras, and its real-world deployment is gradually increasing. Current ReID methods assume that the deployed system follows the centralized processing paradigm, i.e., all computations are conducted in the cloud server and edge devices are only used to capture images. As the number of videos experiences a rapid escalation, this paradigm has become impractical due to the finite computational resources in the cloud server. Therefore, the ReID system should be converted to fit in the cloud-edge collaborative processing paradigm, which is crucial to boost its scalability and practicality. However, current works lack relevant research on this important specific issue, making it difficult to adapt them into a cloud-edge framework effectively. In this paper, we propose a cloud-edge collaborative inference framework for ReID systems, aiming to expedite the return of the desired image captured by the camera to the cloud server by learning the spatial-temporal correlations among objects. In the system, a Distribution-aware Correlation Modeling network (DaCM) is particularly proposed to embed the spatial-temporal correlations of the camera network implicitly into a graph structure, and it can be applied 1) in the cloud to regulate the size of the upload window and 2) on the edge device to adjust the sequence of images, respectively. Notably, the proposed DaCM can be seamlessly combined with traditional ReID methods, enabling their application within our proposed edge-cloud collaborative framework. Extensive experiments demonstrate that our method obviously reduces transmission overhead and significantly improves performance.

PDF Details DOI

TIST Journal 2025 Journal Article

Towards Predicting Urban Land Use Changes: A Dynamic Graph Alignment Perspective

Yu Fan
Xinjiang Lu
Hao Liu
Pengfei Wang
Liang Liu
Huadong Ma
Jingbo Zhou

Urban land use, intrinsically linked to people’s daily activities, undergoes continuous evolution, presenting a complex interplay that remains partially understood. To bridge this gap, our study leverages fine-grained human mobility data to predict these changes, adopting a novel approach that conceptualizes “community-level” land use shifts as a regression problem and represents citywide changes through dynamic graphs. We harness recent advancements in graph neural networks (GNNs), which, despite their success in various applications, face challenges in directly predicting land use changes due to the temporal mismatch between the slow evolution of urban land and the immediacy of human mobility data. Our research stands out by introducing a temporal skeleton for dynamic GNNs to synchronize human activity graphs with urban land use changes, a dynamic heterogeneous GNN approach for integrating diverse human activity data to capture essential temporal dependencies, and a novel algorithm powered by causal inference to elucidate the primary factors influencing land use predictions at the community level, all of which contribute to a training process informed by the generated causal graph. Empirically validated on three real-world datasets, our model demonstrates a performance leap over state-of-the-art baselines, marking a pivotal step toward understanding and predicting the dynamics of urban land use.

Details DOI

AAAI Conference 2025 Conference Paper

VIoTGPT: Learning to Schedule Vision Tools Towards Intelligent Video Internet of Things

Yaoyao Zhong
Mengshi Qi
Rui Wang
Yuhan Qiu
Yang Zhang
Huadong Ma

Video Internet of Things (VIoT) has shown full potential in collecting an unprecedented volume of video data. How to schedule the domain-specific perceiving models and analyze the collected videos uniformly, efficiently, and especially intelligently to accomplish complicated tasks is challenging. To address the challenge, we build VIoTGPT, the framework based on LLMs to correctly interact with humans, query knowledge videos, and invoke vision models to analyze multimedia data collaboratively. To support VIoTGPT and related future works, we meticulously crafted the VIoT-Tool dataset, including the training dataset and the benchmark involving 11 representative vision models across three categories based on semi-automatic annotations. To guide LLM to act as the intelligent agent towards intelligent VIoT, we resort to ReAct instruction tuning method based on VIoT-Tool to learn the tool capability. Quantitative and qualitative experiments and analyses demonstrate the effectiveness of VIoTGPT. We believe VIoTGPT contributes to improving human-centered experiences in VIoT applications.

PDF Details DOI

ECAI Conference 2024 Conference Paper

CSAdv: Class-Specific Adversarial Patches for DETR-Style Object Detection

Yue Xu
Chuanming Wang
Xiaolong Zheng 0002
Yi Huang
Peilun Du
Zeyuan Zhou
Liang Liu 0001
Huadong Ma

Remarkable advancements have been made in the field of object detection, and given its widespread application, it is of paramount importance to investigate the robustness of detection models. However, previous methods have primarily focused on models based on Convolutional Neural Networks (CNNs), seriously neglecting the Transformer-based models that develop rapidly but exhibit obvious differences in terms of information processing. Therefore, this paper aims to address this gap by exploring potential attacks arising from the self-attention mechanism inhered in Transformer. Specifically, we propose a novel adversarial attack scenario targeting Transformer-based object detection models, where only objects of specific class fail to be detected, while irrelevant objects remain undisturbed. Therefore, human perception is hard to find errors even with the detector fail. To achieve this goal, we introduce an adversarial patch generation method, termed Class-Specific Adversarial (CSAdv) patches, which simultaneously leverages class probability to attack specific objects and utilizes the output from Transformer decoder structures, Query Output, to protect irrelevant objects. Due to the long-range interactions of Transformer, the adversarial patch does not need to directly cover or closely surround the specific objects. Instead, it achieves remote targeted attacks simply by being placed in the corner of image, which greatly enhances the concealment of patches. Extensive experiments are conducted on various benchmark datasets and Transformer-based baselines, and the experimental results show that CSAdv can effectively mask certain class while keeping other classes as unaffected as far as possible.

Details

ICML Conference 2024 Conference Paper

ELTA: An Enhancer against Long-Tail for Aesthetics-oriented Models

Limin Liu
Shuai He
Anlong Ming
Rui Xie
Huadong Ma

Real-world datasets often exhibit long-tailed distributions, compromising the generalization and fairness of learning-based models. This issue is particularly pronounced in Image Aesthetics Assessment (IAA) tasks, where such imbalance is difficult to mitigate due to a severe distribution mismatch between features and labels, as well as the great sensitivity of aesthetics to image variations. To address these issues, we propose an Enhancer against Long-Tail for Aesthetics-oriented models (ELTA). ELTA first utilizes a dedicated mixup technique to enhance minority feature representation in high-level space while preserving their intrinsic aesthetic qualities. Next, it aligns features and labels through a similarity consistency approach, effectively alleviating the distribution mismatch. Finally, ELTA adopts a specific strategy to refine the output distribution, thereby enhancing the quality of pseudo-labels. Experiments on four representative datasets (AVA, AADB, TAD66K, and PARA) show that our proposed ELTA achieves state-of-the-art performance by effectively mitigating the long-tailed issue in IAA datasets. Moreover, ELTA is designed with plug-and-play capabilities for seamless integration with existing methods. To our knowledge, this is the first contribution in the IAA community addressing long-tail. All resources are available in here.

Details

AAAI Conference 2024 Conference Paper

Region-Aware Exposure Consistency Network for Mixed Exposure Correction

Jin Liu
Huiyuan Fu
Chuanming Wang
Huadong Ma

Exposure correction aims to enhance images suffering from improper exposure to achieve satisfactory visual effects. Despite recent progress, existing methods generally mitigate either overexposure or underexposure in input images, and they still struggle to handle images with mixed exposure, i.e., one image incorporates both overexposed and underexposed regions. The mixed exposure distribution is non-uniform and leads to varying representation, which makes it challenging to address in a unified process. In this paper, we introduce an effective Region-aware Exposure Correction Network (RECNet) that can handle mixed exposure by adaptively learning and bridging different regional exposure representations. Specifically, to address the challenge posed by mixed exposure disparities, we develop a region-aware de-exposure module that effectively translates regional features of mixed exposure scenarios into an exposure-invariant feature space. Simultaneously, as de-exposure operation inevitably reduces discriminative information, we introduce a mixed-scale restoration unit that integrates exposure-invariant features and unprocessed features to recover local information. To further achieve a uniform exposure distribution in the global image, we propose an exposure contrastive regularization strategy under the constraints of intra-regional exposure consistency and inter-regional exposure continuity. Extensive experiments are conducted on various datasets, and the experimental results demonstrate the superiority and generalization of our proposed method. The code is released at: https://github.com/kravrolens/RECNet.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

Rethinking No-reference Image Exposure Assessment from Holism to Pixel: Models, Datasets and Benchmarks

Shuai He
Shuntian Zheng
Anlong Ming
Banyu Wu
Huadong Ma

The past decade has witnessed an increasing demand for enhancing image quality through exposure, and as a crucial prerequisite in this endeavor, Image Exposure Assessment (IEA) is now being accorded serious attention. However, IEA encounters two persistent challenges that remain unresolved over the long term: the accuracy and generalizability of No-reference IEA are inadequate for practical applications; the scope of IEA is confined to qualitative and quantitative analysis of the entire image or subimage, such as providing only a score to evaluate the exposure level, thereby lacking intuitive and precise fine-grained evaluation for complex exposure conditions. The objective of this paper is to address the persistent bottleneck challenges from three perspectives: model, dataset, and benchmark. 1) Model-level: we propose a Pixel-level IEA Network (P-IEANet) that utilizes Haar discrete wavelet transform (DWT) to analyze, decompose, and assess exposure from both lightness and structural perspectives, capable of generating pixel-level assessment results under no-reference scenarios. 2) Dataset-level: we elaborately build an exposure-oriented dataset, IEA40K, containing 40K images, covering 17 typical lighting scenarios, 27 devices, and 50+ scenes, with each image densely annotated by more than 10 experts with pixel-level labels. 3) Benchmark-level: we develop a comprehensive benchmark of 19 methods based on IEA40K. Our P-IEANet not only achieves state-of-the-art (SOTA) performance on all metrics but also seamlessly integrates with existing exposure correction and lighting enhancement methods. To our knowledge, this is the first work that explicitly emphasizes assessing complex image exposure problems at a pixel level, providing a significant boost to the IEA and exposure-related community. The code and dataset are available in \href{https: //github. com/mRobotit/Pixel-level-No-reference-Image-Exposure-Assessment}{\textcolor{red} {here}}.

PDF Details DOI

AAAI Conference 2024 Conference Paper

SGFormer: Semantic Graph Transformer for Point Cloud-Based 3D Scene Graph Generation

Changsheng Lv
Mengshi Qi
Xia Li
Zhengyuan Yang
Huadong Ma

In this paper, we propose a novel model called SGFormer, Semantic Graph TransFormer for point cloud-based 3D scene graph generation. The task aims to parse a point cloud-based scene into a semantic structural graph, with the core challenge of modeling the complex global structure. Existing methods based on graph convolutional networks (GCNs) suffer from the over-smoothing dilemma and can only propagate information from limited neighboring nodes. In contrast, SGFormer uses Transformer layers as the base building block to allow global information passing, with two types of newly-designed layers tailored for the 3D scene graph generation task. Specifically, we introduce the graph embedding layer to best utilize the global information in graph edges while maintaining comparable computation costs. Furthermore, we propose the semantic injection layer to leverage linguistic knowledge from large-scale language model (i.e., ChatGPT), to enhance objects' visual features. We benchmark our SGFormer on the established 3DSSG dataset and achieve a 40.94% absolute improvement in relationship prediction's R@50 and an 88.36% boost on the subset with complex scenes over the state-of-the-art. Our analyses further show SGFormer's superiority in the long-tail and zero-shot scenarios. Our source code is available at https://github.com/Andy20178/SGFormer.

PDF Details DOI

AAAI Conference 2024 Conference Paper

Weakly-Supervised Temporal Action Localization by Inferring Salient Snippet-Feature

Wulian Yun
Mengshi Qi
Chuanming Wang
Huadong Ma

Weakly-supervised temporal action localization aims to locate action regions and identify action categories in untrimmed videos simultaneously by taking only video-level labels as the supervision. Pseudo label generation is a promising strategy to solve the challenging problem, but the current methods ignore the natural temporal structure of the video that can provide rich information to assist such a generation process. In this paper, we propose a novel weakly-supervised temporal action localization method by inferring salient snippet-feature. First, we design a saliency inference module that exploits the variation relationship between temporal neighbor snippets to discover salient snippet-features, which can reflect the significant dynamic change in the video. Secondly, we introduce a boundary refinement module that enhances salient snippet-features through the information interaction unit. Then, a discrimination enhancement module is introduced to enhance the discriminative nature of snippet-features. Finally, we adopt the refined snippet-features to produce high-fidelity pseudo labels, which could be used to supervise the training of the action localization network. Extensive experiments on two publicly available datasets, i.e., THUMOS14 and ActivityNet v1.3, demonstrate our proposed method achieves significant improvements compared to the state-of-the-art methods. Our source code is available at https://github.com/wuli55555/ISSF.

PDF Details DOI

TIST Journal 2023 Journal Article

Configure Your Federation: Hierarchical Attention-enhanced Meta-Learning Network for Personalized Federated Learning

Yujia Gao
Pengfei Wang
Liang Liu
Chi Zhang
Huadong Ma

Federated learning, as a distributed machine learning framework, enables clients to conduct model training without transmitting their data to the server, which is used to solve the dilemma of data silos and data privacy. It can work well on clients having similar data characteristics and distribution. However, it has some limitations where the dataset of clients may be different in distribution, quantity, and concept in many application scenarios. Personalized federated learning is a new federated learning paradigm that aims to guarantee client personalized models’ effectiveness when collaborating with the cloud server. Intuitively, providing further facilitated collaborations for the clients with similar data characteristics and distribution can benefit personalized model building. However, due to the invisibility of client data, it is challenging to extract client characteristics and define collaborative relationships among them from a fine-grained view. Moreover, a reasonable collaborative training approach needs to be designed for a distributed server–client framework. In this article, we design a Hierarchical Attention-enhanced Meta-learning Network (HAM) to address this issue. The main advantage of HAM is that it utilizes the meta-learning approach of taking model parameters as features and learns to learn an extra model for each client to analyze similarities according to their local dataset automatically. According to its two-layers framework, HAM can reasonably achieve a tradeoff between clients’ personality and commonality and provides a hybrid model with useful information from all clients. Considering there are two networks (HAM and base network) that need to learn for each client during the federated training process, we then provide an alternative learning approach to train them in an end-to-end fashion. To further clarify the approach, we describe the personalized federated learning settings framework as FedHAM where the HAM network is distributed deployed in each client. Extensive experiments based on two datasets prove that our method outperforms state-of-the-art baselines under different evaluation metrics.

Details DOI

NeurIPS Conference 2023 Conference Paper

Disentangled Counterfactual Learning for Physical Audiovisual Commonsense Reasoning

Changsheng Lv
Shuai Zhang
Yapeng Tian
Mengshi Qi
Huadong Ma

In this paper, we propose a Disentangled Counterfactual Learning (DCL) approach for physical audiovisual commonsense reasoning. The task aims to infer objects’ physics commonsense based on both video and audio input, with the main challenge is how to imitate the reasoning ability of humans. Most of the current methods fail to take full advantage of different characteristics in multi-modal data, and lacking causal reasoning ability in models impedes the progress of implicit physical knowledge inferring. To address these issues, our proposed DCL method decouples videos into static (time-invariant) and dynamic (time-varying) factors in the latent space by the disentangled sequential encoder, which adopts a variational autoencoder (VAE) to maximize the mutual information with a contrastive loss function. Furthermore, we introduce a counterfactual learning module to augment the model’s reasoning ability by modeling physical knowledge relationships among different objects under counterfactual intervention. Our proposed method is a plug-and-play module that can be incorporated into any baseline. In experiments, we show that our proposed method improves baseline methods and achieves state-of-the-art performance. Our source code is available at https: //github. com/Andy20178/DCL.

PDF Details

AAAI Conference 2020 Conference Paper

Gait Recognition for Co-Existing Multiple People Using Millimeter Wave Sensing

Zhen Meng
Song Fu
Jie Yan
Hongyuan Liang
Anfu Zhou
Shilin Zhu
Huadong Ma
Jianhua Liu

Gait recognition, i. e. , recognizing persons from their walking postures, has found versatile applications in security check, health monitoring, and novel human-computer interaction. The millimeter-wave (mmWave) based gait recognition represents the most recent advance. Compared with traditional camera-based solutions, mmWave based gait recognition bears unique advantages of being still effective under non-line-of-sight scenarios, such as in black, weak light, or blockage conditions. Moreover, they are able to accomplish person identiﬁcation while preserving privacy. Currently, there are only few works in mmWave gait recognition, since no public data set is available. In this paper, we build a ﬁrst-of-its-kind mmWave gait data set, in which we collect gait of 95 volunteers ’seen’ from two mmWave radars in two different scenarios, which together lasts about 30 hours. Using the data set, we propose a novel deep-learning driven mmWave gait recognition method called mmGaitNet, and compare it with ﬁve state-of-the-art algorithms. We ﬁnd that mmGaitNet is able to achieve 90% accuracy for single-person scenarios, 88% accuracy for ﬁve co-existing persons, while the existing methods achieve less than 66% accuracy for both scenarios.

PDF Details

AAAI Conference 2020 Conference Paper

Region-Based Global Reasoning Networks

Chuanming Wang
Huiyuan Fu
Charles X. Ling
Peilun Du
Huadong Ma

Global reasoning plays a signiﬁcant role in many computer vision tasks which need to capture long-distance relationships. However, most current studies on global reasoning focus on exploring the relationship between pixels and ignore the critical role of the regions. In this paper, we propose an novel approach that explores the relationship between regions which have richer semantics than pixels. Speciﬁcally, we design a region aggregation method that can gather regional features automatically into a uniform shape, and adjust theirs positions adaptively for better alignment. To achieve the best performance of global reasoning, we propose various relationship exploration methods and apply them on the regional features. Our region-based global reasoning module, named ReGr, is end-to-end and can be inserted into existing visual understanding models without extra supervision. To evaluate our approach, we apply ReGr to ﬁne-grained classiﬁcation and action recognition benchmark tasks, and the experimental results demonstrate the effectiveness of our approach.

PDF Details

ECAI Conference 2020 Conference Paper

SI-AGAN: Spatial Interpolation with Attentional Generative Adversarial Networks for Environment Monitoring

Yujia Gao
Liang Liu 0001
Chi Zhang 0019
Xiao Wang 0017
Huadong Ma

Monitoring the status of urban environmental phenomena is of great significance for urban research and management. While The monitoring sites are often insufficient and unevenly, interpolation values vary in urban spaces non-linearly. It is difficult to find a method that fulfills the requirements of accuracy, robustness, and flexibility for various types of phenomena. In this paper, we present a new kind of deep learning driven spatial interpolation method which works on the grid data that can be applied on the unevenly distributed sites. To generate better accurate spatial continuous data, we design the S2 attention structure and incorporate it with the GAN to turn it into SI-AGAN which can model spatial dependencies across different regions via sparsely and unevenly distributed sampling. It can directly learn an end-to-end mapping between low- and high-quality environmental signals without in-depth knowledge of the phenomenon. Experiments on two real-world air-pollution datasets demonstrate that our training strategy effectively makes the GAN work for the interpolation of uneven data and our proposed SI-AGAN significantly outperforms previous state-of-the-art spatial interpolation methods.

Details

AAAI Conference 2018 Conference Paper

T-C3D: Temporal Convolutional 3D Network for Real-Time Action Recognition

Kun Liu
Wu Liu
Chuang Gan
Mingkui Tan
Huadong Ma

Video-based action recognition with deep neural networks has shown remarkable progress. However, most of the existing approaches are too computationally expensive due to the complex network architecture. To address these problems, we propose a new real-time action recognition architecture, called Temporal Convolutional 3D Network (T-C3D), which learns video action representations in a hierarchical multi-granularity manner. Speciﬁcally, we combine a residual 3D convolutional neural network which captures complementary information on the appearance of a single frame and the motion between consecutive frames with a new temporal encoding method to explore the temporal dynamics of the whole video. Thus heavy calculations are avoided when doing the inference, which enables the method to be capable of real-time processing. On two challenging benchmark datasets, UCF101 and HMDB51, our method is signiﬁcantly better than state-of-the-art real-time methods by over 5. 4% in terms of accuracy and 2 times faster in terms of inference speed (969 frames per second), demonstrating comparable recognition performance to the state-of-the-art methods. The source code for the complete system as well as the pre-trained models are publicly available at https: //github. com/tc3d.

PDF Details

TIST Journal 2012 Journal Article

A Reliable People Counting System via Multiple Cameras

Huadong Ma
Chengbin Zeng
Charles X. Ling

Reliable and real-time people counting is crucial in many applications. Most previous works can only count moving people from a single camera, which cannot count still people or can fail badly when there is a crowd (i.e., heavy occlusion occurs). In this article, we build a system for robust and fast people counting under occlusion through multiple cameras. To improve the reliability of human detection from a single camera, we use a dimensionality reduction method on the multilevel edge and texture features to handle the large variations in human appearance and poses. To accelerate the detection speed, we propose a novel two-stage cascade-of-rejectors method. To handle the heavy occlusion in crowded scenes, we present a fusion method with error tolerance to combine human detection from multiple cameras. To improve the speed and accuracy of moving people counting, we combine our multiview fusion detection method with particle tracking to count the number of people moving in/out the camera view (“border control”). Extensive experiments and analyses show that our method outperforms state-of-the-art techniques in single- and multicamera datasets for both speed and reliability. We also design a deployed system for fast and reliable people (still or moving) counting by using multiple cameras.

Details DOI