Contents
The short answer is no. It does not affect prediction accuracy. Multicollinearity does not affect the accuracy of predictive models, including regression models.
Does multicollinearity affect random forests?
Random Forest uses bootstrap sampling and feature sampling, i.e row sampling and column sampling. Therefore Random Forest is not affected by multicollinearity that much since it is picking different set of features for different models and of course every model sees a different set of data points.
Why are feature correlations biased in random forests?
In my last post, I investigated claims by Altmann, et al. that feature importance scores from Random Forests (RFs) were biased for categorical variables. I demonstrated that the bias was due to the encoding scheme. Altmann, et al. also claimed that RFs are biased in the presence of correlation between variables.
Why are random forests important in machine learning?
Random forest feature importance. Random forests are among the most popular machine learning methods thanks to their relatively good accuracy, robustness and ease of use. They also provide two straightforward methods for feature selection: mean decrease impurity and mean decrease accuracy.
How to calculate feature importance in random forests?
Several measures are available for feature importance in Random Forests: Gini Importance or Mean Decrease in Impurity (MDI) calculates each feature importance as the sum over the number of splits (accross all tress) that include the feature, proportionaly to the number of samples it splits.
How does feature correlation and feature importance bias work?
The same pattern emerges – as the number of correlated variables increases, the feature importance score for each variable decreases. This pattern should be expected, however. The correlated variables provide similar information. When the decision trees are searching for optimal splits, the correlated variables are interchangeable.