RF_MSU: A multivariate symmetrical uncertainty-based random forest

Abstract

Decision Trees (DTs) have become very popular classifiers due to their good performance and, most of all, their interpretability. In addition, the machine learning community is also paying attention to Random Forests (RFs) since they defy the interpretability-accuracy tradeoff. Most RFs strategies are based on univariate measures, a fact that may limit the capability of identifying the interaction among more than two features. In order to overcome this problem many multivariate approaches have been proposed. However, most of them are based on finding linear or non-linear combinations of features. In this work, we propose a novel univariate RF strategy that builds DTs using the Multivariate Symmetrical Uncertainty (MSU) measure as splitting criterion. The proposal, referred to as RF_MSU, was tested on high-dimensional datasets and compared to state-of-the-art univariate and multivariate DTs and RFs classifiers. Results suggest that RF_MSU is capable of finding simpler rules than other RFs approaches while keeping a high predictive power equivalent to that of multivariate approaches. The DT strategies considered obtained simpler models than RF_MSU, but at the expense of degrading the classifier. Thus, we can conclude that RF_MSU is a RF-based classifier that achieves a good trade-off between the performance and the complexity of the model.