Author name cluster

Zhiwei Dong

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

3 papers

2 author rows

AAAI Conference 2025 Conference Paper

AtomNet: Designing Tiny Models from Operators Under Extreme MCU Constraints

Zhiwei Dong
Mingzhu Shen
Shihao Bai
Xiuying Wei
Jinyang Guo
Ruihao Gong
Song-Lu Chen
Xianglong Liu

Tiny machine learning (TinyML) has attracted heightened attention for its ability to provide low-cost and instantaneous performance on edge devices. Particularly, the commonly used microcontroller unit (MCU) imposes extreme constraints on peak memory (SRAM) and storage (Flash). Existing TinyML methods often rely on a customized and hard-to-obtain inference libraries, as well as necessitate a time-consuming search for a deployable architecture using advanced Neural Architecture Search (NAS) algorithms. To solve these problems, we fully exploit the resources on MCU and deduce hardware-oriented guidelines for designing models under extreme MCU constraints. In detail, we delve into thorough information about the atom operators by collecting the runtime data of Flash, SRAM, and latency to build a dataset named AtomDB. Based on AtomDB, several critical operator guidelines are established to fully utilize limited Flash and SRAM, while minimizing latency. By transferring the guidelines to analyze blocks, we propose a hybrid pattern that organizes appropriate blocks at different network stages to form the AtomNet, a more hardware-oriented architecture, to handle the former SRAM bottleneck and the latter Flash bottleneck. Extensive experiments demonstrate the effectiveness of the exploitation of the hardware characteristics. Remarkably, AtomNet pioneeringly achieve 3.5% accuracy enhancement and more than 15% latency reduction on 320KB MCU using readily available official inference libraries for ImageNet tasks, surpassing the current state-of-the-art method.

PDF Details DOI

IROS Conference 2025 Conference Paper

Data-Bootstrapped, Physics-Informed Framework for Object Rearrangement

Alex Wong
Zhiwei Dong

Object rearrangement, which involves arranging objects step-by-step to achieve tidy states, is critical in robotic applications. Progress in this area is often constrained by issues such as high-cost data collection and physically infeasible trajectory prediction. To address these challenges, we propose the Data-Bootstrapped, Physics-Informed Rearrangement (DPR) framework, which leverages a transformer for sequential decision making. Specifically, DPR integrates Enhanced Data Generation with a Physics Reward Feedback Transformer. Enhanced Data Generation consists of Random Trajectory Reverse for producing high-quality training data and Bootstrapped Trajectory Synthesis, which leverages the transformer’s sequence modeling to diversify training trajectories. To ensure the feasibility of the generated trajectories and to improve the transformer’s performance, we incorporate a Physical Reward Feedback mechanism into the transformer. Experiments on ball and room rearrangement tasks show that DPR significantly outperforms existing methods in terms of both efficiency and effectiveness. Code will be released soon.

Details

ICLR Conference 2024 Conference Paper

QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Models

Jing Liu 0048
Ruihao Gong
Xiuying Wei
Zhiwei Dong
Jianfei Cai 0001
Bohan Zhuang

Large Language Models (LLMs) have demonstrated unparalleled efficacy in natural language processing. However, their high computational demands and memory overheads hinder their broad deployment. To address this, two quantization strategies emerge, including Quantization-Aware Training (QAT) and Post-Training Quantization (PTQ). For LLMs, the billions of parameters make the QAT impractical due to the prohibitive training cost and thus PTQ becomes more prevalent. In existing studies, activation outliers in particular channels are identified as the biggest challenge to PTQ accuracy. They propose to transform the magnitudes from activations to weights, which however offers limited alleviation or suffers from unstable gradients, resulting in a severe performance drop at low-bitwidth. In this paper, we propose QLLM, an accurate and efficient low-bitwidth PTQ method designed for LLMs. QLLM introduces an adaptive channel reassembly technique that reallocates the magnitude of outliers to other channels, thereby mitigating their impact on the quantization range. This is achieved by channel disassembly and channel assembly, which first breaks down the outlier channels into several sub-channels to ensure a more balanced distribution of activation magnitudes. Then similar channels are merged to maintain the original channel number for efficiency. Additionally, an adaptive strategy is designed to autonomously determine the optimal number of sub-channels for channel disassembly. To further compensate for the performance loss caused by quantization, we propose an efficient tuning method that only learns a small number of low-rank weights while freezing the pre-trained quantized model. After training, these low-rank parameters can be fused into the frozen weights without affecting inference. Extensive experiments on LLaMA-1 and LLaMA-2 show that QLLM is able to obtain accurate quantized models efficiently. For example, QLLM quantizes the 4-bit LLaMA-2-70B within 10 hours on a single A100-80G GPU, outperforming the previous state-of-the-art method by 7.89% on the average accuracy across five zero-shot tasks. Code is available at [ZIP Lab](https://github.com/ziplab/QLLM) and [ModelTC](https://github.com/ModelTC/QLLM).

Details