Variable Importance Using Decision Trees

Decision trees and random forests are well established models that not only offer good predictive performance, but also provide rich feature importance information. While practitioners often employ variable importance methods that rely on this impurity-based information, these methods remain poorly characterized from a theoretical perspective. We provide novel insights into the performance of these methods by deriving finite sample performance guarantees in a high-dimensional setting under various modeling assumptions. We further demonstrate the effectiveness of these impurity-based methods via an extensive set of simulations.

Authors

• Jalil Kazemitabar University of California, Los Angeles
• Arash Amini
• Adam Bloniarz Google
• Ameet Talwalkar Carnegie Mellon University

Keywords

No keywords are indexed for this paper.