Title: Conditional Permutation Importance Revisited

Author: Dries Debeer

Affiliation: University of Zürich

Abstract:
Although they were originally developed for prediction purposes, Random
Forests (RFs; Breiman & Cutler, 2003) have become a popular tool for assessing
the relevance of the predictor variables in a prediction setting. A multitude
of so-called RF-based variable importance measures have been proposed and
applied to assess the relevance of the predictor variables. In this
presentation we revisit the Conditional Permutation Importance (CPI; Strobl,
et al., 2008), which, we argue, can be interpreted as a quantification of a
more partial importance. After explaining the rationale behind the CPI, we
will demonstrate some issues with its current implementation in the R-package
party. In an attempt to mitigate these issues we propose a new implementation,
with an accompanying R-package: permimp. Differences between the old and new
CPI the implementation are explained. The practical consequences pertaining to
the issues we raised are illustrated using simulated data.