Author name cluster

Chengcheng Wang

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

6 papers

2 author rows

AAAI Conference 2026 Conference Paper

CasMoE: A Cascaded Framework for Efficient MoE Inference on Resource-constrained Devices

Chengcheng Wang
Haowen He
Liang Zhao
Xiaoheng Deng
Lixin Duan
Shaohua Wan

The Mixture-of-Experts (MoE) architecture has emerged as a key enabler for scaling large language models (LLMs), empowering increased model capacity with minimal computational overhead through gating-based dynamic expert activation. However, due to the memory demands introduced by expert modules, MoE inference on resource-constrained devices is still challenging. Existing methods such as model compression and parameter offloading provide partial alleviation but often lead to reduced accuracy or increased latency. In this paper, we propose CasMoE, a general and efficient cascaded framework for accelerating MoE inference on resource-constrained devices. CasMoE employs a two-stage offline-online approach to facilitate efficient expert prefetching. In the offline stage, a parameterized Expert Activation Predictor (EAP) is introduced to accurately predict the corresponding expert activation from the incoming prompt. In the online stage, a non-parametric Expert Activation Matcher (EAM) supporting fast expert retrieval is then integrated with the EAP to form a cascade planner that operates independently of the MoE architecture, predicting activated experts for all MoE layers in a single pass prior to decoding. A gating mechanism is also incorporated to dynamically adjust the sensitivity of the EAM and EAP, enabling a flexible trade-off between inference efficiency and quality. Extensive experiments on diverse downstream tasks demonstrate CasMoE’s effectiveness in accelerating inference while preserving high accuracy.

PDF Details DOI

AAAI Conference 2026 Conference Paper

PocketLLM: Ultimate Compression of Large Language Models via Meta Networks

Ye Tian
Chengcheng Wang
Jing Han
Yehui Tang
Kai Han

As Large Language Models (LLMs) continue to grow in size, storing and transmitting them on edge devices becomes increasingly challenging. Traditional methods like quantization and pruning struggle to achieve extreme compression of LLMs without sacrificing accuracy. In this paper, we introduce PocketLLM, a novel approach to compress LLMs in a latent space via meta-networks. A simple encoder network is proposed to project the weights of LLMs into discrete latent vectors, which are then represented using a compact codebook. A lightweight decoder network is employed to map the codebook's representative vectors back to the original weight space. This method allows for significant compression of the large weights in LLMs, consisting solely of a small decoder, a concise codebook, and an index. Extensive experiments show that PocketLLM achieves superior performance even at significantly high compression ratios, e.g., compressing Llama 2-7B by 10x with a negligible drop in accuracy.

PDF Details DOI

JBHI Journal 2025 Journal Article

MTFuseSyn: A Multi-Task Fusion Framework for Drug Synergy Prediction Integrating Cell Line Multi-Omics Data

Yuanyuan Zhang
Ciao Zhang
Qihao Wang
Wensheng An
Chengcheng Wang
Shaoqiang Wang

With the continuous rise in cancer incidence and mortality, drug resistance has emerged as a critical challenge in cancer therapy. Conventional monotherapy often fails to address tumor cell heterogeneity and multiple drug resistance, resulting in limited efficacy, whereas combination therapy—through the synergistic effects of multiple drugs—can significantly enhance treatment outcomes and delay resistance development. However, accurately predicting drug synergism remains a formidable task due to the complex interplay of factors such as drug molecular features, drug–drug interactions, target proteins, and cell line characteristics, with current methods falling short in integrating these multidimensional data. To address this challenge, we propose a multi-task learning framework—MTFuseSyn—which constructs and integrates multiple tasks, including drug–target interactions and drug–drug interactions. To obtain richer drug features, the framework incorporates a graph aggregation module that leverages an adaptive attention mechanism to automatically identify and focus on key molecular substructures highly correlated with synergistic effects. Additionally, the framework integrates multimodal cell line data to learn richer and context-relevant cellular feature representations, thereby providing robust biological support for prediction. To effectively integrate knowledge across tasks, we design a task fusion attention module to dynamically capture potential associations among multiple tasks. Experimental results on the authoritative DrugCombDB and Oncology-Screen datasets demonstrate that MTFuseSyn significantly outperforms existing methods in both classification and regression tasks, underscoring the importance of multidimensional information fusion in drug synergy prediction. Ablation studies and case analyses further validate the efficacy of the proposed modules.

Details DOI

ICML Conference 2024 Conference Paper

Data-efficient Large Vision Models through Sequential Autoregression

Zhiwei Hao 0001
Jianyuan Guo
Chengcheng Wang
Yehui Tang 0001
Han Wu 0009
Han Hu 0001
Kai Han 0002
Chang Xu 0002

Training general-purpose vision models on purely sequential visual data, eschewing linguistic inputs, has heralded a new frontier in visual understanding. These models are intended to not only comprehend but also seamlessly transit to out-of-domain tasks. However, current endeavors are hamstrung by an over-reliance on colossal models, exemplified by models with upwards of 3B parameters, and the necessity for an extensive corpus of visual data, often comprising a staggering 400B tokens. In this paper, we delve into the development of an efficient, autoregression-based vision model, innovatively architected to operate on a limited dataset. We meticulously demonstrate how this model achieves proficiency in a spectrum of visual tasks spanning both high-level and low-level semantic understanding during the testing phase. Our empirical evaluations underscore the model’s agility in adapting to various tasks, heralding a significant reduction in the parameter footprint, and a marked decrease in training data requirements, thereby paving the way for more sustainable and accessible advancements in the field of generalist vision models. The code is available at https: //github. com/ggjy/DeLVM.

Details

NeurIPS Conference 2023 Conference Paper

Gold-YOLO: Efficient Object Detector via Gather-and-Distribute Mechanism

Chengcheng Wang
Wei He
Ying Nie
Jianyuan Guo
Chuanjian Liu
Yunhe Wang
Kai Han

In the past years, YOLO-series models have emerged as the leading approaches in the area of real-time object detection. Many studies pushed up the baseline to a higher level by modifying the architecture, augmenting data and designing new losses. However, we find previous models still suffer from information fusion problem, although Feature Pyramid Network (FPN) and Path Aggregation Network (PANet) have alleviated this. Therefore, this study provides an advanced Gatherand-Distribute mechanism (GD) mechanism, which is realized with convolution and self-attention operations. This new designed model named as Gold-YOLO, which boosts the multi-scale feature fusion capabilities and achieves an ideal balance between latency and accuracy across all model scales. Additionally, we implement MAE-style pretraining in the YOLO-series for the first time, allowing YOLOseries models could be to benefit from unsupervised pretraining. Gold-YOLO-N attains an outstanding 39. 9% AP on the COCO val2017 datasets and 1030 FPS on a T4 GPU, which outperforms the previous SOTA model YOLOv6-3. 0-N with similar FPS by +2. 4%. The PyTorch code is available at https: //github. com/huawei-noah/Efficient-Computing/tree/master/Detection/Gold-YOLO, and the MindSpore code is available at https: //gitee. com/mindspore/models/tree/master/research/cv/Gold_YOLO.

PDF Details

NeurIPS Conference 2023 Conference Paper

Species196: A One-Million Semi-supervised Dataset for Fine-grained Species Recognition

Wei He
Kai Han
Ying Nie
Chengcheng Wang
Yunhe Wang

The development of foundation vision models has pushed the general visual recognition to a high level, but cannot well address the fine-grained recognition in specialized domain such as invasive species classification. Identifying and managing invasive species has strong social and ecological value. Currently, most invasive species datasets are limited in scale and cover a narrow range of species, which restricts the development of deep-learning based invasion biometrics systems. To fill the gap of this area, we introduced Species196, a large-scale semi-supervised dataset of 196-category invasive species. It collects over 19K images with expert-level accurate annotations (Species196-L), and 1. 2M unlabeled images of invasive species (Species196-U). The dataset provides four experimental settings for benchmarking the existing models and algorithms, namely, supervised learning, semi-supervised learning and self-supervised pretraining. To facilitate future research on these four learning paradigms, we conduct an empirical study of the representative methods on the introduced dataset. The dataset will be made publicly available at https: //species-dataset. github. io/.

PDF Details