Arrow Research search

Author name cluster

Jian Cheng

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

46 papers
2 author rows

Possible papers

46

EAAI Journal 2026 Journal Article

A fuzzy cross domain matrix machine for fault diagnosis under multi-objective domain

  • Haiyang Pan
  • Chunan Chen
  • Tong Chen
  • Jian Cheng
  • Jinde Zheng
  • Shuchao Deng

Mechanical fault diagnosis faces significant challenges due to the susceptibility of traditional matrix classifiers to abnormal samples and their inability to leverage cross domain information. To address these limitations, this paper introduces a novel fuzzy cross domain matrix machine (FCDMM), which integrates fuzzy membership modeling with multi-task learning to enhance robustness and enable knowledge transfer across related tasks. FCDMM constructs fuzzy-based decision hyperplanes and designs adaptive boundary factors to mitigate the influence of outliers. Extensive experiments on multiple domain fault datasets demonstrate that FCDMM achieves remarkable accuracy rates of 98. 86 % for bearing faults and 99. 57 % for gear faults, significantly outperforming state-of-the-art methods.

Details DOI

AAAI Conference 2026 Conference Paper

Deep (Predictive) Discounted Counterfactual Regret Minimization

  • Hang Xu
  • Kai Li
  • Haobo Fu
  • Qiang Fu
  • Junliang Xing
  • Jian Cheng

Counterfactual regret minimization (CFR) is a family of algorithms for effectively solving imperfect-information games. To enhance CFR's applicability in large games, researchers use neural networks to approximate its behavior. However, existing methods are mainly based on vanilla CFR and struggle to effectively integrate more advanced CFR variants. In this work, we propose an efficient model-free neural CFR algorithm, overcoming the limitations of existing methods in approximating advanced CFR variants. At each iteration, it collects variance-reduced sampled advantages based on a value network, fits cumulative advantages by bootstrapping, and applies discounting and clipping operations to simulate the update mechanisms of advanced CFR variants. Experimental results show that, compared with model-free neural algorithms, it exhibits faster convergence in typical imperfect-information games and demonstrates stronger adversarial performance in a large poker game.

PDF Details DOI

EAAI Journal 2026 Journal Article

Multiple restriction cross-border matrix machine for multiple objective fault diagnosis

  • Haiyang Pan
  • Chunan Chen
  • Zhiming Chen
  • Jian Cheng
  • Jinde Zheng

As a single-task classification method, support matrix machine (SMM) is widely used in mechanical equipment fault diagnosis. However, when diagnosing multi-objective tasks, it is difficult to fully utilize the common information between multiple tasks, resulting in limited information contained in the model. Meanwhile, SMM is extremely sensitive to abnormal samples, which is not conducive to model construction. To address the aforementioned issues, a novel multiple restriction cross-border matrix machine (MRCBMM) is proposed. In MRCBMM, a weighted constraint group (WCG) is firstly designed to adjust the influence of different abnormal samples on the hyperplane, thereby determining the optimal position of the hyperplane. Meanwhile, MRCBMM defines a matrix kernel expansion (MKE) that maps matrix samples to high-dimensional space to fully utilize the structural information of the original signal. In addition, to achieve cross-border diagnosis of MRCBMM, a regularized multi-task learning framework is constructed to complete the features and parameters. Two sets of multi-objective rotating mechanical fault datasets are used for validation, and the results show that MRCBMM improves diagnostic accuracy by approximately 2% in scenarios containing abnormal samples compared with existing methods, and consistently achieves over 98% accuracy on clean datasets.

Details DOI

EAAI Journal 2026 Journal Article

Multiple task nonparallel embedded matrix machine and its application in multi-objective fault diagnosis

  • Haiyang Pan
  • Chunan Chen
  • Wenfeng Hu
  • Jinde Zheng
  • Jian Cheng
  • Jingcheng Wang

With the increasing complexity of mechanical equipment structures, monitoring and diagnosing multiple objects simultaneously is of great significance. Traditional diagnostic methods such as support matrix machine (SMM) often adopt single task modeling, which makes the diagnostic model under multiple tasks unable to consider the correlation between multiple tasks and misses the common information between multiple components. Therefore, this paper proposes a new classifier called multiple task nonparallel embedded matrix machine (MTNPEMM). In MTNPEMM, a kernel enhancement embedded term is defined, which can achieve full mining of the correlation between matrix rows and columns and utilization of multi-task common information through mutual mapping between matrix sample slices. Meanwhile, the nonlinear offset is introduced to capture nonlinear relationships in data, which can improve the stability and accuracy. By comparing existing methods with MTNPEMM through experiments, the results verifies that MTNPEMM achieves multi-task average accuracy of 99. 50 % and 99. 79 % in Case 1 and Case 2, respectively, significantly higher than traditional classification methods.

Details DOI

AAMAS Conference 2026 Conference Paper

Towards Foresighted AI Cooperators with LLM-driven Decision-Time Planning

  • Yuheng Jing
  • Kai Li
  • Bingyun Liu
  • Ziwen Zhang
  • Zhe Wu
  • Yifan Zhang
  • Junliang Xing
  • Jian Cheng

In multi-agent systems, building agents capable of seamlessly collaborating with unknown partners is a long-standing research goal. Existing approaches primarily generate a diverse population of partners and then train an agent against this population to master various cooperation conventions. However, these approaches are often hindered in two aspects: (1) They heavily rely on taskspecific training; (2) Their trained agents lack adaptability at test time. In this paper, we investigate how to leverage Large Language Models (LLMs) to build agents capable of foresighted coordination, addressing the challenges faced by existing work. To facilitate structured reasoning mechanisms, we introduce DTPAgent, a novel LLM-driven Decision-Time Planning (DTP) framework. Within this framework, LLMs, without relying on task-specific training and solely through in-context learning, estimate the partner policy and the transition-reward function to model the full dynamics of the environment. Based on these LLM-driven modelings, DTPAgent This work is licensed under a Creative Commons Attribution International 4. 0 License. Proc. of the 25th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2026), C. Amato, L. Dennis, V. Mascardi, J. Thangarajah (eds.), May 25 – 29, 2026, Paphos, Cyprus. © 2026 International Foundation for Autonomous Agents and Multiagent Systems (www. ifaamas. org). https: //doi. org/10. 65109/XALP4331 simulates a range of possible trajectories to dynamically search for the most advantageous policy at each timestep. We demonstrate on the popular benchmark, Overcooked, that DTPAgent, built with small-scale LLMs, effectively outperforms various types of baselines when faced with unseen partners. Our DTPAgent also exhibits a scalable property that existing agents lack: the ability to translate test-time computation into improved performance.

PDF

AAAI Conference 2025 Conference Paper

An Open-Ended Learning Framework for Opponent Modeling

  • Yuheng Jing
  • Kai Li
  • Bingyun Liu
  • Haobo Fu
  • Qiang Fu
  • Junliang Xing
  • Jian Cheng

Opponent Modeling (OM) aims to enhance decision-making by modeling other agents in multi-agent environments. Existing works typically learn opponent models against a pre-designated fixed set of opponents during training. However, this will cause poor generalization when facing unknown opponents during testing, as previously unseen opponents can exhibit out-of-distribution (OOD) behaviors that the learned opponent models cannot handle. To tackle this problem, we introduce a novel Open-Ended Opponent Modeling (OEOM) framework, which continuously generates opponents with diverse strengths and styles to reduce the possibility of OOD situations occurring during testing. Founded on population-based training and information-theoretic trajectory space diversity regularization, OEOM generates a dynamic set of opponents. This set is then fed to any OM approaches to train a potentially generalizable opponent model. Upon this, we further propose a simple yet effective OM approach that naturally fits within the OEOM framework. This approach is based on in-context reinforcement learning and learns a Transformer that dynamically recognizes and responds to opponents based on their trajectories. Extensive experiments in cooperative, competitive, and mixed environments demonstrate that OEOM is an approach-agnostic framework that improves generalizability compared to training against a fixed set of opponents, regardless of OM approaches or testing opponent settings. The results also indicate that our proposed approach generally outperforms existing OM baselines.

PDF Details DOI

NeurIPS Conference 2025 Conference Paper

Bi-Level Knowledge Transfer for Multi-Task Multi-Agent Reinforcement Learning

  • Junkai Zhang
  • Jinmin He
  • Yifan Zhang
  • Yifan Zang
  • Ning Xu
  • Jian Cheng

Multi-Agent Reinforcement Learning (MARL) has achieved remarkable success in various real-world scenarios, but its high cost of online training makes it impractical to learn each task from scratch. To enable effective policy reuse, we consider the problem of zero-shot generalization from offline data across multiple tasks. While prior work focuses on transferring individual skills of agents, we argue that the effective policy transfer across tasks should also capture the team-level coordination knowledge. In this paper, we propose Bi-Level Knowledge Transfer (BiKT) for Multi-Task MARL, which performs knowledge transfer at both the individual and team levels. At the individual level, we extract transferable individual skill embeddings from offline MARL trajectories. At the team level, we define tactics as coordinated patterns of skill combinations and capture them by leveraging the learned skill embeddings. We map skill combinations into compact tactic embeddings and then construct a tactic codebook. To incorporate both skills and tactics into decision-making, we design a bi-level decision transformer that infers them in sequence. Our BiKT leverages both the generalizability of individual skills and the diversity of tactics, enabling the learned policy to perform effectively across multiple tasks. Extensive experiments on SMAC and MPE benchmarks demonstrate that BiKT achieves strong generalization to previously unseen tasks.

PDF Details

NeurIPS Conference 2025 Conference Paper

DartQuant: Efficient Rotational Distribution Calibration for LLM Quantization

  • YUANTIAN SHAO
  • Yuanteng Chen
  • Peisong Wang
  • Jianlin Yu
  • Jing Lin
  • yiwu yao
  • Zhihui Wei
  • Jian Cheng

Quantization plays a crucial role in accelerating the inference of large-scale models, and rotational matrices have been shown to effectively improve quantization performance by smoothing outliers. However, end-to-end fine-tuning of rotational optimization algorithms incurs high computational costs and is prone to overfitting. To address this challenge, we propose an efficient distribution-aware rotational calibration method, DartQuant, which reduces the complexity of rotational optimization by constraining the distribution of the activations after rotation. This approach also effectively reduces reliance on task-specific losses, thereby mitigating the risk of overfitting. Additionally, we introduce the QR-Orth optimization scheme, which replaces expensive alternating optimization with a more efficient solution. In a variety of model quantization experiments, DartQuant demonstrates superior performance. Compared to existing methods, it achieves 47$\times$ acceleration and 10$\times$ memory savings for rotational optimization on a 70B model. Furthermore, it is the first to successfully complete rotational calibration for a 70B model on a single 3090 GPU, making quantization of large language models feasible in resource-constrained environments.

PDF Details

AAAI Conference 2025 Conference Paper

HOGSA: Bimanual Hand-Object Interaction Understanding with 3D Gaussian Splatting Based Data Augmentation

  • Wentian Qu
  • Jiahe Li
  • Jian Cheng
  • Jian Shi
  • Chenyu Meng
  • Cuixia Ma
  • Hongan Wang
  • Xiaoming Deng

Understanding of bimanual hand-object interaction plays an important role in robotics and virtual reality. However, due to significant occlusions between hands and object as well as the high degree-of-freedom motions, it is challenging to collect and annotate a high-quality, large-scale dataset, which prevents further improvement of bimanual hand-object interaction-related baselines. In this work, we propose a new 3D Gaussian Splatting based data augmentation framework for bimanual hand-object interaction, which is capable of augmenting existing dataset to large-scale photorealistic data with various hand-object pose and viewpoints. First, we use mesh-based 3DGS to model objects and hands, and to deal with the rendering blur problem due to multi-resolution input images used, we design a super-resolution module. Second, we extend the single hand grasping pose optimization module for the bimanual hand object to generate various poses of bimanual hand-object interaction, which can significantly expand the pose distribution of the dataset. Third, we conduct an analysis for the impact of different aspects of the proposed data augmentation on the understanding of the bimanual hand-object interaction. We perform our data augmentation on two benchmarks, H2O and Arctic, and verify that our method can improve the performance of the baselines.

PDF Details DOI

EAAI Journal 2025 Journal Article

Two-dimensional refined composite multi-scale revised ensemble dispersion entropy and its application to fault diagnosis of rolling bearing

  • Wenqing Ding
  • Jinde Zheng
  • Haiyang Pan
  • Jian Cheng
  • Jinyu Tong

One-dimensional ensemble dispersion entropy (EDE1D) can effectively characterize the nonlinear dynamic characteristics of one-dimensional time series, but the complexity of two-dimensional space is not reflected, and only single-scale features can be captured. Firstly, to comprehensively capture the feature information of two-dimensional space, the symmetrized dot pattern (SDP) is introduced to overcome the shortcomings of the ordinary images lack of physical meaning and the time-frequency distribution methods exhibit incomplete information representation, etc. Simultaneously, the amplitude and frequency information are intuitively expressed by a two-dimensional mirror snowflake symmetrized image (MSSI2D). Secondly, to overcome the shortcomings of single-scale and traditional coarse-graining, a two-dimensional refined composite multi-scale coarse-graining method is proposed, which improves the accuracy of feature extraction and reduces the calculation deviation. After that, a new feature extraction method namely two-dimensional refined composite multi-scale revised ensemble dispersion entropy (RCMREDE2D) is proposed, whose parameter stability and performance are explored through simulation analysis. The results demonstrate that the RCMREDE2D exhibits excellent stability and anti-noise interference ability. Based on the advantages of RCMREDE2D, a novel fault diagnosis method for rolling bearings is developed by integrating RCMREDE2D and a firefly algorithm optimized support vector machine (FA-SVM) multi-fault classifier for pattern recognition. The proposed method is further validated through two measured bearing data sets and five comparative methods, and the results indicate that the RCMREDE2D and FA-SVM achieve the highest recognition accuracy while demonstrating superior stability.

Details DOI

AAAI Conference 2025 Conference Paper

Universal Features Guided Zero-Shot Category-Level Object Pose Estimation

  • Wentian Qu
  • Chenyu Meng
  • Heng Li
  • Jian Cheng
  • Cuixia Ma
  • Hongan Wang
  • Xiao Zhou
  • Xiaoming Deng

Object pose estimation, crucial in computer vision and robotics applications, faces challenges with the diversity of unseen categories. We propose a zero-shot method to achieve category-level 6-DOF object pose estimation, which exploits both 2D and 3D universal features of input RGB-D image to establish semantic similarity-based correspondences and can be extended to unseen categories without additional model fine-tuning. Our method begins with combining efficient 2D universal features to find sparse correspondences between intra-category objects and gets initial coarse pose. To handle the correspondence degradation of 2D universal features if the pose deviates much from the target pose, we use an iterative strategy to optimize the pose. Subsequently, to resolve pose ambiguities due to shape differences between intra-category objects, the coarse pose is refined by optimizing with dense alignment constraint of 3D universal features. Our method outperforms previous methods on the REAL275 and Wild6D benchmarks for unseen categories.

PDF Details DOI

AAAI Conference 2024 Conference Paper

Analyzing Generalization in Policy Networks: A Case Study with the Double-Integrator System

  • Ruining Zhang
  • Haoran Han
  • Maolong Lv
  • Qisong Yang
  • Jian Cheng

Extensive utilization of deep reinforcement learning (DRL) policy networks in diverse continuous control tasks has raised questions regarding performance degradation in expansive state spaces where the input state norm is larger than that in the training environment. This paper aims to uncover the underlying factors contributing to such performance deterioration when dealing with expanded state spaces, using a novel analysis technique known as state division. In contrast to prior approaches that employ state division merely as a post-hoc explanatory tool, our methodology delves into the intrinsic characteristics of DRL policy networks. Specifically, we demonstrate that the expansion of state space induces the activation function $\tanh$ to exhibit saturability, resulting in the transformation of the state division boundary from nonlinear to linear. Our analysis centers on the paradigm of the double-integrator system, revealing that this gradual shift towards linearity imparts a control behavior reminiscent of bang-bang control. However, the inherent linearity of the division boundary prevents the attainment of an ideal bang-bang control, thereby introducing unavoidable overshooting. Our experimental investigations, employing diverse RL algorithms, establish that this performance phenomenon stems from inherent attributes of the DRL policy network, remaining consistent across various optimization algorithms.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

Efficient Multi-task Reinforcement Learning with Cross-Task Policy Guidance

  • Jinmin He
  • Kai Li
  • Yifan Zang
  • Haobo Fu
  • Qiang Fu
  • Junliang Xing
  • Jian Cheng

Multi-task reinforcement learning endeavors to efficiently leverage shared information across various tasks, facilitating the simultaneous learning of multiple tasks. Existing approaches primarily focus on parameter sharing with carefully designed network structures or tailored optimization procedures. However, they overlook a direct and complementary way to exploit cross-task similarities: the control policies of tasks already proficient in some skills can provide explicit guidance for unmastered tasks to accelerate skills acquisition. To this end, we present a novel framework called Cross-Task Policy Guidance (CTPG), which trains a guide policy for each task to select the behavior policy interacting with the environment from all tasks' control policies, generating better training trajectories. In addition, we propose two gating mechanisms to improve the learning efficiency of CTPG: one gate filters out control policies that are not beneficial for guidance, while the other gate blocks tasks that do not necessitate guidance. CTPG is a general framework adaptable to existing parameter sharing approaches. Empirical evaluations demonstrate that incorporating CTPG with these approaches significantly enhances performance in manipulation and locomotion benchmarks.

PDF Details DOI

AAAI Conference 2024 Conference Paper

Intrinsic Action Tendency Consistency for Cooperative Multi-Agent Reinforcement Learning

  • Junkai Zhang
  • Yifan Zhang
  • Xi Sheryl Zhang
  • Yifan Zang
  • Jian Cheng

Efficient collaboration in the centralized training with decentralized execution (CTDE) paradigm remains a challenge in cooperative multi-agent systems. We identify divergent action tendencies among agents as a significant obstacle to CTDE's training efficiency, requiring a large number of training samples to achieve a unified consensus on agents' policies. This divergence stems from the lack of adequate team consensus-related guidance signals during credit assignment in CTDE. To address this, we propose Intrinsic Action Tendency Consistency, a novel approach for cooperative multi-agent reinforcement learning. It integrates intrinsic rewards, obtained through an action model, into a reward-additive CTDE (RA-CTDE) framework. We formulate an action model that enables surrounding agents to predict the central agent's action tendency. Leveraging these predictions, we compute a cooperative intrinsic reward that encourages agents to align their actions with their neighbors' predictions. We establish the equivalence between RA-CTDE and CTDE through theoretical analyses, demonstrating that CTDE's training process can be achieved using N individual targets. Building on this insight, we introduce a novel method to combine intrinsic rewards and RA-CTDE. Extensive experiments on challenging tasks in SMAC, MPE, and GRF benchmarks showcase the improved performance of our method.

PDF Details DOI

IJCAI Conference 2024 Conference Paper

Minimizing Weighted Counterfactual Regret with Optimistic Online Mirror Descent

  • Hang Xu
  • Kai Li
  • Bingyun Liu
  • Haobo Fu
  • Qiang Fu
  • Junliang Xing
  • Jian Cheng

Counterfactual regret minimization (CFR) is a family of algorithms for effectively solving imperfect-information games. It decomposes the total regret into counterfactual regrets, utilizing local regret minimization algorithms, such as Regret Matching (RM) or RM+, to minimize them. Recent research establishes a connection between Online Mirror Descent (OMD) and RM+, paving the way for an optimistic variant PRM+ and its extension PCFR+. However, PCFR+ assigns uniform weights for each iteration when determining regrets, leading to substantial regrets when facing dominated actions. This work explores minimizing weighted counterfactual regret with optimistic OMD, resulting in a novel CFR variant PDCFR+. It integrates PCFR+ and Discounted CFR (DCFR) in a principled manner, swiftly mitigating negative effects of dominated actions and consistently leveraging predictions to accelerate convergence. Theoretical analyses prove that PDCFR+ converges to a Nash equilibrium, particularly under distinct weighting schemes for regrets and average strategies. Experimental results demonstrate PDCFR+'s fast convergence in common imperfect-information games. The code is available at https: //github. com/rpSebastian/PDCFRPlus.

PDF Details DOI

AAAI Conference 2024 Conference Paper

Not All Tasks Are Equally Difficult: Multi-Task Deep Reinforcement Learning with Dynamic Depth Routing

  • Jinmin He
  • Kai Li
  • Yifan Zang
  • Haobo Fu
  • Qiang Fu
  • Junliang Xing
  • Jian Cheng

Multi-task reinforcement learning endeavors to accomplish a set of different tasks with a single policy. To enhance data efficiency by sharing parameters across multiple tasks, a common practice segments the network into distinct modules and trains a routing network to recombine these modules into task-specific policies. However, existing routing approaches employ a fixed number of modules for all tasks, neglecting that tasks with varying difficulties commonly require varying amounts of knowledge. This work presents a Dynamic Depth Routing (D2R) framework, which learns strategic skipping of certain intermediate modules, thereby flexibly choosing different numbers of modules for each task. Under this framework, we further introduce a ResRouting method to address the issue of disparate routing paths between behavior and target policies during off-policy training. In addition, we design an automatic route-balancing mechanism to encourage continued routing exploration for unmastered tasks without disturbing the routing of mastered ones. We conduct extensive experiments on various robotics manipulation tasks in the Meta-World benchmark, where D2R achieves state-of-the-art performance with significantly improved learning efficiency.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

Opponent Modeling with In-context Search

  • Yuheng Jing
  • Bingyun Liu
  • Kai Li
  • Yifan Zang
  • Haobo Fu
  • Qiang Fu
  • Junliang Xing
  • Jian Cheng

Opponent modeling is a longstanding research topic aimed at enhancing decision-making by modeling information about opponents in multi-agent environments. However, existing approaches often face challenges such as having difficulty generalizing to unknown opponent policies and conducting unstable performance. To tackle these challenges, we propose a novel approach based on in-context learning and decision-time search named Opponent Modeling with In-context Search (OMIS). OMIS leverages in-context learning-based pretraining to train a Transformer model for decision-making. It consists of three in-context components: an actor learning best responses to opponent policies, an opponent imitator mimicking opponent actions, and a critic estimating state values. When testing in an environment that features unknown non-stationary opponent agents, OMIS uses pretrained in-context components for decision-time search to refine the actor's policy. Theoretically, we prove that under reasonable assumptions, OMIS without search converges in opponent policy recognition and has good generalization properties; with search, OMIS provides improvement guarantees, exhibiting performance stability. Empirically, in competitive, cooperative, and mixed environments, OMIS demonstrates more effective and stable adaptation to opponents than other approaches. See our project website at https: //sites. google. com/view/nips2024-omis.

PDF Details DOI

AAAI Conference 2024 Conference Paper

Patch-Aware Sample Selection for Efficient Masked Image Modeling

  • Zhengyang Zhuge
  • Jiaxing Wang
  • Yong Li
  • Yongjun Bao
  • Peisong Wang
  • Jian Cheng

Nowadays sample selection is drawing increasing attention. By extracting and training only on the most informative subset, sample selection can effectively reduce the training cost. Although sample selection is effective in conventional supervised learning, applying it to Masked Image Modeling (MIM) still poses challenges due to the gap between sample-level selection and patch-level pre-training. In this paper, we inspect the sample selection in MIM pre-training and find the basic selection suffers from performance degradation. We attribute this degradation primarily to 2 factors: the random mask strategy and the simple averaging function. We then propose Patch-Aware Sample Selection (PASS), including a low-cost Dynamic Trained Mask Predictor (DTMP) and Weighted Selection Score (WSS). DTMP consistently masks the informative patches in samples, ensuring a relatively accurate representation of selection score. WSS enhances the selection score using patch-level disparity. Extensive experiments show the effectiveness of PASS in selecting the most informative subset and accelerating pretraining. PASS exhibits superior performance across various datasets, MIM methods, and downstream tasks. Particularly, PASS improves MAE by 0.7% on ImageNet-1K while utilizing only 37% data budget and achieves ~1.7x speedup.

PDF Details DOI

AAAI Conference 2023 Conference Paper

Asynchronous Event Processing with Local-Shift Graph Convolutional Network

  • Linhui Sun
  • Yifan Zhang
  • Jian Cheng
  • Hanqing Lu

Event cameras are bio-inspired sensors that produce sparse and asynchronous event streams instead of frame-based images at a high-rate. Recent works utilizing graph convolutional networks (GCNs) have achieved remarkable performance in recognition tasks, which model event stream as spatio-temporal graph. However, the computational mechanism of graph convolution introduces redundant computation when aggregating neighbor features, which limits the low-latency nature of the events. And they perform a synchronous inference process, which can not achieve a fast response to the asynchronous event signals. This paper proposes a local-shift graph convolutional network (LSNet), which utilizes a novel local-shift operation equipped with a local spatio-temporal attention component to achieve efficient and adaptive aggregation of neighbor features. To improve the efficiency of pooling operation in feature extraction, we design a node-importance based parallel pooling method (NIPooling) for sparse and low-latency event data. Based on the calculated importance of each node, NIPooling can efficiently obtain uniform sampling results in parallel, which retains the diversity of event streams. Furthermore, for achieving a fast response to asynchronous event signals, an asynchronous event processing procedure is proposed to restrict the network nodes which need to recompute activations only to those affected by the new arrival event. Experimental results show that the computational cost can be reduced by nearly 9 times through using local-shift operation and the proposed asynchronous procedure can further improve the inference efficiency, while achieving state-of-the-art performance on gesture recognition and object recognition.

PDF Details DOI

NeurIPS Conference 2023 Conference Paper

Automatic Grouping for Efficient Cooperative Multi-Agent Reinforcement Learning

  • Yifan Zang
  • Jinmin He
  • Kai Li
  • Haobo Fu
  • Qiang Fu
  • Junliang Xing
  • Jian Cheng

Grouping is ubiquitous in natural systems and is essential for promoting efficiency in team coordination. This paper proposes a novel formulation of Group-oriented Multi-Agent Reinforcement Learning (GoMARL), which learns automatic grouping without domain knowledge for efficient cooperation. In contrast to existing approaches that attempt to directly learn the complex relationship between the joint action-values and individual utilities, we empower subgroups as a bridge to model the connection between small sets of agents and encourage cooperation among them, thereby improving the learning efficiency of the whole team. In particular, we factorize the joint action-values as a combination of group-wise values, which guide agents to improve their policies in a fine-grained fashion. We present an automatic grouping mechanism to generate dynamic groups and group action-values. We further introduce a hierarchical control for policy learning that drives the agents in the same group to specialize in similar policies and possess diverse strategies for various groups. Experiments on the StarCraft II micromanagement tasks and Google Research Football scenarios verify our method's effectiveness. Extensive component studies show how grouping works and enhances performance.

PDF Details

JBHI Journal 2023 Journal Article

Improving the Quality of Fetal Heart Ultrasound Imaging With Multihead Enhanced Self-Attention and Contrastive Learning

  • Yingying Zhang
  • Haogang Zhu
  • Jian Cheng
  • Jingyi Wang
  • Xiaoyan Gu
  • Jiancheng Han
  • Ye Zhang
  • Ying Zhao

Fetal congenital heart disease (FCHD) is a common, serious birth defect affecting ∼1% of newborns annually. Fetal echocardiography is the most effective and important technique for prenatal FCHD diagnosis. The prerequisites for accurate ultrasound FCHD diagnosis are accurate view recognition and high-quality diagnostic view extraction. However, these manual clinical procedures have drawbacks such as, varying technical capabilities and inefficiency. Therefore, the automatic identification of high-quality multiview fetal heart scan images is highly desirable to improve prenatal diagnosis efficiency and accuracy of FCHD. Here, we present a framework for multiview fetal heart ultrasound image recognition and quality assessment that comprises two parts: a multiview classification and localization network (MCLN) and an improved contrastive learning network (ICLN). In the MCLN, a multihead enhanced self-attention mechanism is applied to construct the classification network and identify six accurate and interpretable views of the fetal heart. In the ICLN, anatomical structure standardization and image clarity are considered. With contrastive learning, the absolute loss, feature relative loss and predicted value relative loss are combined to achieve favorable quality assessment results. Experiments show that the MCLN outperforms other state-of-the-art networks by 1. 52–13. 61% when determining the F1 score in six standard view recognition tasks, and the ICLN is comparable to the performance of expert cardiologists in the quality assessment of fetal heart ultrasound images, reaching 97% on a test set within 2 points for the four-chamber view task. Thus, our architecture offers great potential in helping cardiologists improve quality control for fetal echocardiographic images in clinical practice.

Details DOI

AAAI Conference 2023 Conference Paper

TinyNeRF: Towards 100 x Compression of Voxel Radiance Fields

  • Tianli Zhao
  • Jiayuan Chen
  • Cong Leng
  • Jian Cheng

Voxel grid representation of 3D scene properties has been widely used to improve the training or rendering speed of the Neural Radiance Fields (NeRF) while at the same time achieving high synthesis quality. However, these methods accelerate the original NeRF at the expense of extra storage demand, which hinders their applications in many scenarios. To solve this limitation, we present TinyNeRF, a three-stage pipeline: frequency domain transformation, pruning and quantization that work together to reduce the storage demand of the voxel grids with little to no effects on their speed and synthesis quality. Based on the prior knowledge of visual signals sparsity in the frequency domain, we convert the original voxel grids in the frequency domain via block-wise discrete cosine transformation (DCT). Next, we apply pruning and quantization to enforce the DCT coefficients to be sparse and low-bit. Our method can be optimized from scratch in an end-to-end manner, and can typically compress the original models by 2 orders of magnitude with minimal sacrifice on speed and synthesis quality.

PDF Details DOI

NeurIPS Conference 2023 Conference Paper

Towards Efficient and Accurate Winograd Convolution via Full Quantization

  • Tianqi Chen
  • Weixiang Xu
  • Weihan Chen
  • Peisong Wang
  • Jian Cheng

The Winograd algorithm is an efficient convolution implementation, which performs calculations in the transformed domain. To further improve the computation efficiency, recent works propose to combine it with model quantization. Although Post-Training Quantization has the advantage of low computational cost and has been successfully applied in many other scenarios, a severe accuracy drop exists when utilizing it in Winograd convolution. Besides, despite the Winograd algorithm consisting of four stages, most existing methods only quantize the element-wise multiplication stage, leaving a considerable portion of calculations in full precision. In this paper, observing the inconsistency among different transformation procedures, we present PTQ-Aware Winograd (PAW) to optimize them collaboratively under a unified objective function. Moreover, we explore the full quantization of faster Winograd (tile size $\geq4$) for the first time. We further propose a hardware-friendly method called Factorized Scale Quantization (FSQ), which can effectively balance the significant range differences in the Winograd domain. Experiments demonstrate the effectiveness of our method, e. g. , with 8-bit quantization and a tile size of 6, our method outperforms the previous Winograd PTQ method by 8. 27\% and 5. 38\% in terms of the top-1 accuracy on ResNet-18 and ResNet-34, respectively.

PDF Details

AAAI Conference 2022 Conference Paper

DPNAS: Neural Architecture Search for Deep Learning with Differential Privacy

  • Anda Cheng
  • Jiaxing Wang
  • Xi Sheryl Zhang
  • Qiang Chen
  • Peisong Wang
  • Jian Cheng

Training deep neural networks (DNNs) for meaningful differential privacy (DP) guarantees severely degrades model utility. In this paper, we demonstrate that the architecture of DNNs has a significant impact on model utility in the context of private deep learning, whereas its effect is largely unexplored in previous studies. In light of this missing, we propose the very first framework that employs neural architecture search to automatic model design for private deep learning, dubbed as DPNAS. To integrate private learning with architecture search, we delicately design a novel search space and propose a DP-aware method for training candidate models. We empirically certify the effectiveness of the proposed framework. The searched model DPNASNet achieves state-of-theart privacy/utility trade-offs, e. g. , for the privacy budget of (, δ) = (3, 1 × 10−5 ), our model obtains test accuracy of 98. 57% on MNIST, 88. 09% on FashionMNIST, and 68. 33% on CIFAR-10. Furthermore, by studying the generated architectures, we provide several intriguing findings of designing private-learning-friendly DNNs, which can shed new light on model design for deep learning with differential privacy.

PDF Details

AAAI Conference 2022 Conference Paper

Efficient Virtual View Selection for 3D Hand Pose Estimation

  • Jian Cheng
  • Yanguang Wan
  • Dexin Zuo
  • Cuixia Ma
  • Jian Gu
  • Ping Tan
  • Hongan Wang
  • Xiaoming Deng

3D hand pose estimation from single depth is a fundamental problem in computer vision, and has wide applications. However, the existing methods still can not achieve satisfactory hand pose estimation results due to view variation and occlusion of human hand. In this paper, we propose a new virtual view selection and fusion module for 3D hand pose estimation from single depth. We propose to automatically select multiple virtual viewpoints for pose estimation and fuse the results of all and find this empirically delivers accurate and robust pose estimation. In order to select most effective virtual views for pose fusion, we evaluate the virtual views based on the confidence of virtual views using a light-weight network via network distillation. Experiments on three main benchmark datasets including NYU, ICVL and Hands2019 demonstrate that our method outperforms the state-of-the-arts on NYU and ICVL, and achieves very competitive performance on Hands2019-Task1, and our proposed virtual view selection and fusion module is both effective for 3D hand pose estimation.

PDF Details

NeurIPS Conference 2022 Conference Paper

GLIF: A Unified Gated Leaky Integrate-and-Fire Neuron for Spiking Neural Networks

  • Xingting Yao
  • Fanrong Li
  • Zitao Mo
  • Jian Cheng

Spiking Neural Networks (SNNs) have been studied over decades to incorporate their biological plausibility and leverage their promising energy efficiency. Throughout existing SNNs, the leaky integrate-and-fire (LIF) model is commonly adopted to formulate the spiking neuron and evolves into numerous variants with different biological features. However, most LIF-based neurons support only single biological feature in different neuronal behaviors, limiting their expressiveness and neuronal dynamic diversity. In this paper, we propose GLIF, a unified spiking neuron, to fuse different bio-features in different neuronal behaviors, enlarging the representation space of spiking neurons. In GLIF, gating factors, which are exploited to determine the proportion of the fused bio-features, are learnable during training. Combining all learnable membrane-related parameters, our method can make spiking neurons different and constantly changing, thus increasing the heterogeneity and adaptivity of spiking neurons. Extensive experiments on a variety of datasets demonstrate that our method obtains superior performance compared with other SNNs by simply changing their neuronal formulations to GLIF. In particular, we train a spiking ResNet-19 with GLIF and achieve $77. 35\%$ top-1 accuracy with six time steps on CIFAR-100, which has advanced the state-of-the-art. Codes are available at https: //github. com/Ikarosy/Gated-LIF.

PDF Details

NeurIPS Conference 2022 Conference Paper

PKD: General Distillation Framework for Object Detectors via Pearson Correlation Coefficient

  • Weihan Cao
  • Yifan Zhang
  • Jianfei Gao
  • Anda Cheng
  • Ke Cheng
  • Jian Cheng

Knowledge distillation(KD) is a widely-used technique to train compact models in object detection. However, there is still a lack of study on how to distill between heterogeneous detectors. In this paper, we empirically find that better FPN features from a heterogeneous teacher detector can help the student although their detection heads and label assignments are different. However, directly aligning the feature maps to distill detectors suffers from two problems. First, the difference in feature magnitude between the teacher and the student could enforce overly strict constraints on the student. Second, the FPN stages and channels with large feature magnitude from the teacher model could dominate the gradient of distillation loss, which will overwhelm the effects of other features in KD and introduce much noise. To address the above issues, we propose to imitate features with Pearson Correlation Coefficient to focus on the relational information from the teacher and relax constraints on the magnitude of the features. Our method consistently outperforms the existing detection KD methods and works for both homogeneous and heterogeneous student-teacher pairs. Furthermore, it converges faster. With a powerful MaskRCNN-Swin detector as the teacher, ResNet-50 based RetinaNet and FCOS achieve 41. 5% and 43. 9% $mAP$ on COCO2017, which are 4. 1% and 4. 8% higher than the baseline, respectively.

PDF Details

NeurIPS Conference 2022 Conference Paper

Singular Value Fine-tuning: Few-shot Segmentation requires Few-parameters Fine-tuning

  • Yanpeng Sun
  • Qiang Chen
  • Xiangyu He
  • Jian Wang
  • Haocheng Feng
  • Junyu Han
  • Errui Ding
  • Jian Cheng

Freezing the pre-trained backbone has become a standard paradigm to avoid overfitting in few-shot segmentation. In this paper, we rethink the paradigm and explore a new regime: {\em fine-tuning a small part of parameters in the backbone}. We present a solution to overcome the overfitting problem, leading to better model generalization on learning novel classes. Our method decomposes backbone parameters into three successive matrices via the Singular Value Decomposition (SVD), then {\em only fine-tunes the singular values} and keeps others frozen. The above design allows the model to adjust feature representations on novel classes while maintaining semantic clues within the pre-trained backbone. We evaluate our {\em Singular Value Fine-tuning (SVF)} approach on various few-shot segmentation methods with different backbones. We achieve state-of-the-art results on both Pascal-5$^i$ and COCO-20$^i$ across 1-shot and 5-shot settings. Hopefully, this simple baseline will encourage researchers to rethink the role of backbone fine-tuning in few-shot settings.

PDF Details

AAAI Conference 2022 Conference Paper

Towards Fully Sparse Training: Information Restoration with Spatial Similarity

  • Weixiang Xu
  • Xiangyu He
  • Ke Cheng
  • Peisong Wang
  • Jian Cheng

The 2: 4 structured sparsity pattern released by NVIDIA Ampere architecture, requiring four consecutive values containing at least two zeros, enables doubling math throughput for matrix multiplications. Recent works mainly focus on inference speedup via 2: 4 sparsity while training acceleration has been largely overwhelmed where backpropagation consumes around 70% of the training time. However, unlike inference, training speedup with structured pruning is nontrivial due to the need to maintain the fidelity of gradients and reduce the additional overhead of performing 2: 4 sparsity online. For the first time, this article proposes fully sparse training (FST) where ‘fully’ indicates that ALL matrix multiplications in forward/backward propagation are structurally pruned while maintaining accuracy. To this end, we begin with saliency analysis, investigating the sensitivity of different sparse objects to structured pruning. Based on the observation of spatial similarity among activations, we propose pruning activations with fixed 2: 4 masks. Moreover, an Information Restoration block is proposed to retrieve the lost information, which can be implemented by efficient gradient-shift operation. Evaluation of accuracy and efficiency shows that we can achieve 2× training acceleration with negligible accuracy degradation on challenging large-scale classification and detection tasks.

PDF Details

YNIMG Journal 2021 Journal Article

A slower rate of sulcal widening in the brains of the nondemented oldest old

  • Hui Tang
  • Tao Liu
  • Hao Liu
  • Jiyang Jiang
  • Jian Cheng
  • Haijun Niu
  • Shuyu Li
  • Henry Brodaty

The relationships between aging and brain morphology have been reported in many previous structural brain studies. However, the trajectories of successful brain aging in the extremely old remain underexplored. In the limited research on the oldest old, covering individuals aged 85 years and older, there are very few studies that have focused on the cortical morphology, especially cortical sulcal features. In this paper, we measured sulcal width and depth as well as cortical thickness from T1-weighted scans of 290 nondemented community-dwelling participants aged between 76 and 103 years. We divided the participants into young old (between 76 and 84; mean = 80.35±2.44; male/female = 76/88) and oldest old (between 85 and 103; mean = 91.74±5.11; male/female = 60/66) groups. The results showed that most of the examined sulci significantly widened with increased age and that the rates of sulcal widening were lower in the oldest old. The spatial pattern of the cortical thinning partly corresponded with that of sulcal widening. Compared to females, males had significantly wider sulci, especially in the oldest old. This study builds a foundation for future investigations of neurocognitive disorders and neurodegenerative diseases in the oldest old, including centenarians.

Details DOI

AAAI Conference 2020 Conference Paper

Cross-Modality Paired-Images Generation for RGB-Infrared Person Re-Identification

  • Guan-An Wang
  • Tianzhu Zhang
  • Yang Yang
  • Jian Cheng
  • Jianlong Chang
  • Xu Liang
  • Zeng-Guang Hou

RGB-Infrared (IR) person re-identification is very challenging due to the large cross-modality variations between RGB and IR images. The key solution is to learn aligned features to the bridge RGB and IR modalities. However, due to the lack of correspondence labels between every pair of RGB and IR images, most methods try to alleviate the variations with set-level alignment by reducing the distance between the entire RGB and IR sets. However, this set-level alignment may lead to misalignment of some instances, which limits the performance for RGB-IR Re-ID. Different from existing methods, in this paper, we propose to generate cross-modality paired-images and perform both global set-level and fine-grained instance-level alignments. Our proposed method enjoys several merits. First, our method can perform set-level alignment by disentangling modalityspecific and modality-invariant features. Compared with conventional methods, ours can explicitly remove the modalityspecific features and the modality variation can be better reduced. Second, given cross-modality unpaired-images of a person, our method can generate cross-modality paired images from exchanged images. With them, we can directly perform instance-level alignment by minimizing distances of every pair of images. Extensive experimental results on two standard benchmarks demonstrate that the proposed model favourably against state-of-the-art methods. Especially, on SYSU-MM01 dataset, our model can achieve a gain of 9. 2% and 7. 7% in terms of Rank-1 and mAP. Code is available at https: //github. com/wangguanan/JSIA-ReID.

PDF Details

AAAI Conference 2020 Conference Paper

M-NAS: Meta Neural Architecture Search

  • Jiaxing Wang
  • Jiaxiang Wu
  • Haoli Bai
  • Jian Cheng

Neural Architecture Search (NAS) has recently outperformed hand-crafted networks in various areas. However, most prevalent NAS methods only focus on a pre-defined task. For a previously unseen task, the architecture is either searched from scratch, which is inefficient, or transferred from the one obtained on some other task, which might be sub-optimal. In this paper, we investigate a previously unexplored problem: whether a universal NAS method exists, such that task-aware architectures can be effectively generated? Towards this problem, we propose Meta Neural Architecture Search (M-NAS). To obtain task-specific architectures, M-NAS adopts a taskaware architecture controller for child model generation. Since optimal weights for different tasks and architectures span diversely, we resort to meta-learning, and learn metaweights that efficiently adapt to a new task on the corresponding architecture with only several gradient descent steps. Experimental results demonstrate the superiority of M-NAS against a number of competitive baselines on both toy regression and few shot classification problems.

PDF Details

NeurIPS Conference 2020 Conference Paper

Revisiting Parameter Sharing for Automatic Neural Channel Number Search

  • Jiaxing Wang
  • Haoli Bai
  • Jiaxiang Wu
  • Xupeng Shi
  • Junzhou Huang
  • Irwin King
  • Michael Lyu
  • Jian Cheng

Recent advances in neural architecture search inspire many channel number search algorithms~(CNS) for convolutional neural networks. To improve searching efficiency, parameter sharing is widely applied, which reuses parameters among different channel configurations. Nevertheless, it is unclear how parameter sharing affects the searching process. In this paper, we aim at providing a better understanding and exploitation of parameter sharing for CNS. Specifically, we propose affine parameter sharing~(APS) as a general formulation to unify and quantitatively analyze existing channel search algorithms. It is found that with parameter sharing, weight updates of one architecture can simultaneously benefit other candidates. However, it also results in less confidence in choosing good architectures. We thus propose a new strategy of parameter sharing towards a better balance between training efficiency and architecture discrimination. Extensive analysis and experiments demonstrate the superiority of the proposed strategy in channel configuration against many state-of-the-art counterparts on benchmark datasets.

PDF Details

IJCAI Conference 2020 Conference Paper

Soft Threshold Ternary Networks

  • Weixiang Xu
  • Xiangyu He
  • Tianli Zhao
  • Qinghao Hu
  • Peisong Wang
  • Jian Cheng

Large neural networks are difficult to deploy on mobile devices because of intensive computation and storage. To alleviate it, we study ternarization, a balance between efficiency and accuracy that quantizes both weights and activations into ternary values. In previous ternarized neural networks, a hard threshold Δ is introduced to determine quantization intervals. Although the selection of Δ greatly affects the training results, previous works estimate Δ via an approximation or treat it as a hyper-parameter, which is suboptimal. In this paper, we present the Soft Threshold Ternary Networks (STTN), which enables the model to automatically determine quantization intervals instead of depending on a hard threshold. Concretely, we replace the original ternary kernel with the addition of two binary kernels at training time, where ternary values are determined by the combination of two corresponding binary values. At inference time, we add up the two binary kernels to obtain a single ternary kernel. Our method dramatically outperforms current state-of-the-arts, lowering the performance gap between full-precision networks and extreme low bit networks. Experiments on ImageNet with AlexNet (Top-1 55. 6%), ResNet-18 (Top-1 66. 2%) achieves new state-of-the-art.

PDF Details DOI

AAAI Conference 2020 Conference Paper

Sparsity-Inducing Binarized Neural Networks

  • Peisong Wang
  • Xiangyu He
  • Gang Li
  • Tianli Zhao
  • Jian Cheng

Binarization of feature representation is critical for Binarized Neural Networks (BNNs). Currently, sign function is the commonly used method for feature binarization. Although it works well on small datasets, the performance on ImageNet remains unsatisfied. Previous methods mainly focus on minimizing quantization error, improving the training strategies and decomposing each convolution layer into several binary convolution modules. However, whether sign is the only option for binarization has been largely overlooked. In this work, we propose the Sparsity-inducing Binarized Neural Network (Si-BNN), to quantize the activations to be either 0 or +1, which introduces sparsity into binary representation. We further introduce trainable thresholds into the backward function of binarization to guide the gradient propagation. Our method dramatically outperforms current state-ofthe-arts, lowering the performance gap between full-precision networks and BNNs on mainstream architectures, achieving the new state-of-the-art on binarized AlexNet (Top-1 50. 5%), ResNet-18 (Top-1 59. 7%), and VGG-Net (Top-1 63. 2%). At inference time, Si-BNN still enjoys the high efficiency of exclusive-not-or (xnor) operations.

PDF Details

IJCAI Conference 2019 Conference Paper

Color-Sensitive Person Re-Identification

  • Guan'an Wang
  • Yang Yang
  • Jian Cheng
  • Jinqiao Wang
  • Zengguang Hou

Recent deep Re-ID models mainly focus on learning high-level semantic features, while failing to explicitly explore color information which is one of the most important cues for person Re-ID. In this paper, we propose a novel Color-Sensitive Re-ID to take full advantage of color information. On one hand, we train our model with real and fake images. By using the extra fake images, more color information can be exploited and it can avoid overfitting during training. On the other hand, we also train our model with images of the same person with different colors. By doing so, features can be forced to focus on the color difference in regions. To generate fake images with specified colors, we propose a novel Color Translation GAN (CTGAN) to learn mappings between different clothing colors and preserve identity consistency among the same clothing color. Extensive evaluations on two benchmark datasets show that our approach significantly outperforms state-of-the-art Re-ID models.

PDF Details

AAAI Conference 2019 Short Paper

Manifold Distance-Based Over-Sampling Technique for Class Imbalance Learning

  • Lingkai Yang
  • Yinan Guo
  • Jian Cheng

Over-sampling technology for handling the class imbalanced problem generates more minority samples to balance the dataset size of different classes. However, sampling in original data space is ineffective as the data in different classes is overlapped or disjunct. Based on this, a new minority sample is presented in terms of the manifold distance rather than Euclidean distance. The overlapped majority and minority samples apt to distribute in fully disjunct subspaces from the view of manifold learning. Moreover, it can avoid generating samples between the minority data locating far away in manifold space. Experiments on 23 UCI datasets show that the proposed method has the better classification accuracy.

PDF Details

IJCAI Conference 2019 Conference Paper

Reading selectively via Binary Input Gated Recurrent Unit

  • Zhe Li
  • Peisong Wang
  • Hanqing Lu
  • Jian Cheng

Recurrent Neural Networks (RNNs) have shown great promise in sequence modeling tasks. Gated Recurrent Unit (GRU) is one of the most used recurrent structures, which makes a good trade-off between performance and time spent. However, its practical implementation based on soft gates only partially achieves the goal to control information flow. We can hardly explain what the network has learnt internally. Inspired by human reading, we introduce binary input gated recurrent unit (BIGRU), a GRU based model using a binary input gate instead of the reset gate in GRU. By doing so, our model can read selectively during interference. In our experiments, we show that BIGRU mainly ignores the conjunctions, adverbs and articles that do not make a big difference to the document understanding, which is meaningful for us to further understand how the network works. In addition, due to reduced interference from redundant information, our model achieves better performances than baseline GRU in all the testing tasks.

PDF Details

AAAI Conference 2018 Conference Paper

From Hashing to CNNs: Training Binary Weight Networks via Hashing

  • Qinghao Hu
  • Peisong Wang
  • Jian Cheng

Deep convolutional neural networks (CNNs) have shown appealing performance on various computer vision tasks in recent years. This motivates people to deploy CNNs to realworld applications. However, most of state-of-art CNNs require large memory and computational resources, which hinders the deployment on mobile devices. Recent studies show that low-bit weight representation can reduce much storage and memory demand, and also can achieve efficient network inference. To achieve this goal, we propose a novel approach named BWNH to train Binary Weight Networks via Hashing. In this paper, we first reveal the strong connection between inner-product preserving hashing and binary weight networks, and show that training binary weight networks can be intrinsically regarded as a hashing problem. Based on this perspective, we propose an alternating optimization method to learn the hash codes instead of directly learning binary weights. Extensive experiments on CIFAR10, CIFAR100 and ImageNet demonstrate that our proposed BWNH outperforms current state-of-art by a large margin.

PDF Details

AAAI Conference 2016 Conference Paper

Shoot to Know What: An Application of Deep Networks on Mobile Devices

  • Jiaxiang Wu
  • Qinghao Hu
  • Cong Leng
  • Jian Cheng

Convolutional neural networks (CNNs) have achieved impressive performance in a wide range of computer vision areas. However, the application on mobile devices remains intractable due to the high computation complexity. In this demo, we propose the Quantized CNN (Q-CNN), an efficient framework for CNN models, to fulfill efficient and accurate image classification on mobile devices. Our Q-CNN framework dramatically accelerates the computation and reduces the storage/memory consumption, so that mobile devices can independently run an ImageNet-scale CNN model. Experiments on the ILSVRC-12 dataset demonstrate 4 ∼ 6× speedup and 15 ∼ 20× compression, with merely one percentage drop in the classification accuracy. Based on the Q-CNN framework, even mobile devices can accurately classify images within one second.

PDF Details

JMLR Journal 2014 Journal Article

Bayesian Co-Boosting for Multi-modal Gesture Recognition

  • Jiaxiang Wu
  • Jian Cheng

With the development of data acquisition equipment, more and more modalities become available for gesture recognition. However, there still exist two critical issues for multi-modal gesture recognition: how to select discriminative features for recognition and how to fuse features from different modalities. In this paper, we propose a novel Bayesian Co-Boosting framework for multi-modal gesture recognition. Inspired by boosting learning and co-training method, our proposed framework combines multiple collaboratively trained weak classifiers to construct the final strong classifier for the recognition task. During each iteration round, we randomly sample a number of feature subsets and estimate weak classifier's parameters for each subset. The optimal weak classifier and its corresponding feature subset are retained for strong classifier construction. Furthermore, we define an upper bound of training error and derive the update rule of instance's weight, which guarantees the error upper bound to be minimized through iterations. For demonstration, we present an implementation of our framework using hidden Markov models as weak classifiers. We perform extensive experiments using the ChaLearn MMGR and ChAirGest data sets, in which our approach achieves 97.63% and 96.53% accuracy respectively on each publicly available data set. [abs] [ pdf ][ bib ] &copy JMLR 2014. ( edit, beta )

PDF Details

YNIMG Journal 2014 Journal Article

Non-Negative Spherical Deconvolution (NNSD) for estimation of fiber Orientation Distribution Function in single-/multi-shell diffusion MRI

  • Jian Cheng
  • Rachid Deriche
  • Tianzi Jiang
  • Dinggang Shen
  • Pew-Thian Yap

Spherical Deconvolution (SD) is commonly used for estimating fiber Orientation Distribution Functions (fODFs) from diffusion-weighted signals. Existing SD methods can be classified into two categories: 1) Continuous Representation based SD (CR-SD), where typically Spherical Harmonic (SH) representation is used for convenient analytical solutions, and 2) Discrete Representation based SD (DR-SD), where the signal profile is represented by a discrete set of basis functions uniformly oriented on the unit sphere. A feasible fODF should be non-negative and should integrate to unity throughout the unit sphere S 2. However, to our knowledge, most existing SH-based SD methods enforce non-negativity only on discretized points and not the whole continuum of S 2. Maximum Entropy SD (MESD) and Cartesian Tensor Fiber Orientation Distributions (CT-FOD) are the only SD methods that ensure non-negativity throughout the unit sphere. They are however computational intensive and are susceptible to errors caused by numerical spherical integration. Existing SD methods are also known to overestimate the number of fiber directions, especially in regions with low anisotropy. DR-SD introduces additional error in peak detection owing to the angular discretization of the unit sphere. This paper proposes a SD framework, called Non-Negative SD (NNSD), to overcome all the limitations above. NNSD is significantly less susceptible to the false-positive peaks, uses SH representation for efficient analytical spherical deconvolution, and allows accurate peak detection throughout the whole unit sphere. We further show that NNSD and most existing SD methods can be extended to work on multi-shell data by introducing a three-dimensional fiber response function. We evaluated NNSD in comparison with Constrained SD (CSD), a quadratic programming variant of CSD, MESD, and an L1-norm regularized non-negative least-squares DR-SD. Experiments on synthetic and real single-/multi-shell data indicate that NNSD improves estimation performance in terms of mean difference of angles, peak detection consistency, and anisotropy contrast between isotropic and anisotropic regions.

Details DOI

AAAI Conference 2014 Conference Paper

Recommendation by Mining Multiple User Behaviors with Group Sparsity

  • Ting Yuan
  • Jian Cheng
  • Xi Zhang
  • Shuang Qiu
  • Hanqing Lu

Recently, some recommendation methods try to improve the prediction results by integrating information from user’s multiple types of behaviors. How to model the dependence and independence between different behaviors is critical for them. In this paper, we propose a novel recommendation model, the Group-Sparse Matrix Factorization (GSMF), which factorizes the rating matrices for multiple behaviors into the user and item latent factor space with group sparsity regularization. It can (1) select out the different subsets of latent factors for different behaviors, addressing that users’ decisions on different behaviors are determined by different sets of factors; (2) model the dependence and independence between behaviors by learning the shared and private factors for multiple behaviors automatically; (3) allow the shared factors between different behaviors to be different, instead of all the behaviors sharing the same set of factors. Experiments on the real-world dataset demonstrate that our model can integrate users’ multiple types of behaviors into recommendation better, compared with other state-of-the-arts.

PDF Details

ICRA Conference 2014 Conference Paper

Robust vehicle detection using 3D Lidar under complex urban environment

  • Jian Cheng
  • Zhiyu Xiang
  • Teng Cao
  • Jilin Liu

Robust vehicle detection is one of the key task for autonomous vehicle under the complex urban environment. Using 3D Lidar, the difficulty of the task lies in that the appearance of a vehicle in the sparse range data changes greatly with the distance, the angle of view, as well as occlusions. In this paper we present a new algorithm to detect vehicles using the finely segmented 3D object points. In segmentation, RGLOS (Ring Gradient based Local Optimal Segmentation) algorithm is proposed. Instead of processing in the grid map, the point-wise segmentation method keeps the connection between points and is able to extract object points in far distance. Using the local optimal ground height, it produces much more correct object points with less wrong ground points. In feature extraction stage, three novel features: position-related shape, object height along length and reflective intensity histogram are proposed. Finally the kernel based SVM is used to finish the classification task. Experiments are carried out using the real data acquired from urban environment. The results demonstrate the superior performance comparing with previous methods, thanks to the improved segmentation and new features.

Details

UAI Conference 2001 Conference Paper

Confidence Inference in Bayesian Networks

  • Jian Cheng
  • Marek J. Druzdzel

We present two sampling algorithms for probabilistic confidence inference in Bayesian networks. These two algorithms (we call them AIS-BN-mu and AIS-BN-sigma algorithms) guarantee that estimates of posterior probabilities are with a given probability within a desired precision bound. Our algorithms are based on recent advances in sampling algorithms for (1) estimating the mean of bounded random variables and (2) adaptive importance sampling in Bayesian networks. In addition to a simple stopping rule for sampling that they provide, the AIS-BN-mu and AIS-BN-sigma algorithms are capable of guiding the learning process in the AIS-BN algorithm. An empirical evaluation of the proposed algorithms shows excellent performance, even for very unlikely evidence.

Details

UAI Conference 2000 Conference Paper

Computational Investigation of Low-Discrepancy Sequences in Simulation Algorithms for Bayesian Networks

  • Jian Cheng
  • Marek J. Druzdzel

Monte Carlo sampling has become a major vehicle for approximate inference in Bayesian networks. In this paper, we investigate a family of related simulation approaches, known collectively as quasi-Monte Carlo methods based on deterministic low-discrepancy sequences. We first outline several theoretical aspects of deterministic low-discrepancy sequences, show three examples of such sequences, and then discuss practical issues related to applying them to belief updating in Bayesian networks. We propose an algorithm for selecting direction numbers for Sobol sequence. Our experimental results show that low-discrepancy sequences (especially Sobol sequence) significantly improve the performance of simulation algorithms in Bayesian networks compared to Monte Carlo sampling.

Details
v2026.09.13