E4: Energy-Efficient DNN Inference for Edge Video Analytics via Early Exiting and DVFS

Deep neural network (DNN) models are increasingly popular in edge video analytic applications. However, the computeintensive nature of DNN models pose challenges for energyefficient inference on resource-constrained edge devices. Most existing solutions focus on optimizing DNN inference latency and accuracy, often overlooking energy efficiency. They also fail to account for the varying complexity of video frames, leading to sub-optimal performance in edge video analytics. In this paper, we propose an EnergyEfficient Early-Exit (E4) framework that enhances DNN inference efficiency for edge video analytics by integrating a novel early-exit mechanism with dynamic voltage and frequency scaling (DVFS) governors. It employs an attentionbased cascade module to analyze video frame diversity and automatically determine optimal DNN exit points. Additionally, E4 features a just-in-time (JIT) profiler that uses coordinate descent search to co-optimize CPU and GPU clock frequencies for each layer before the DNN exit points. Extensive evaluations demonstrate that E4 outperforms current state-of-the-art methods, achieving up to 2.8× speedup and 26% average energy saving while maintaining high accuracy.

Authors

• Ziyang Zhang UbiCIS, Faculty of Computing, Harbin Institute of Technology National Key Laboratory of Smart Farm Technologies and Systems
• Yang Zhao International Research Institute for Artificial Intelligence, Harbin Institute of Technology (Shenzhen)
• Ming-Ching Chang Department of Computer Science, University at Albany - SUNY
• Changyao Lin UbiCIS, Faculty of Computing, Harbin Institute of Technology National Key Laboratory of Smart Farm Technologies and Systems
• Jie Liu

Keywords

No keywords are indexed for this paper.