Evaluating and Aggregating Feature-based Model Explanations

A feature-based model explanation denotes how much each input feature contributes to a model's output for a given data point. As the number of proposed explanation functions grows, we lack quantitative evaluation criteria to help practitioners know when to use which explanation function. This paper proposes quantitative evaluation criteria for feature-based explanations: low sensitivity, high faithfulness, and low complexity. We devise a framework for aggregating explanation functions. We develop a procedure for learning an aggregate explanation function with lower complexity and then derive a new aggregate Shapley value explanation function that minimizes sensitivity.

Authors

• Umang Bhatt New York University The Alan Turing Institute
• Adrian Weller The Alan Turing Institute University of Cambridge
• José M. F. Moura Department of Electrical and Computer Engineering at Carnegie Mellon University, Pittsburgh, PA, USA

Keywords

• AI Ethics: Explainability
• Machine Learning: Explainable Machine Learning
• Machine Learning: Interpretability