Credal Ensemble Distillation for Uncertainty Quantification

Deep ensembles (DE) have emerged as a powerful approach for quantifying predictive uncertainty and distinguishing its aleatoric and epistemic components, thereby enhancing model robustness and reliability. However, their high computational and memory costs during inference pose significant challenges for wide practical deployment. To overcome this issue, we propose credal ensemble distillation (CED), a novel framework that compresses a DE into a single model, CREDIT, for classification tasks. Instead of a single softmax probability distribution, CREDIT predicts class-wise probability intervals that define a credal set, a convex set of probability distributions, for uncertainty quantification. Empirical results on out-of-distribution detection benchmarks demonstrate that CED achieves superior or comparable uncertainty estimation compared to several existing baselines, while substantially reducing inference overhead compared to DE.

Authors

• Kaizheng Wang Department of Computer Science, KU Leuven, Belgium Flanders Make@KU Leuven, Belgium
• Fabio Cuzzolin School of Engineering, Computing and Mathematics, Oxford Brookes University, U.K.
• David Moens Flanders Make@KU Leuven, Belgium Department of Mechanical Engineering, KU Leuven, Belgium
• Hans Hallez Department of Computer Science, KU Leuven, Belgium

Keywords

No keywords are indexed for this paper.