Contents
Can random forest be built without decision trees?
The success of a random forest highly depends on using uncorrelated decision trees. If we use same or very similar trees, overall result will not be much different than the result of a single decision tree. Random forests achieve to have uncorrelated decision trees by bootstrapping and feature randomness.
Are trees independent in random forest?
1 Answer. No, they are not. In Random Forest (and bootstrap aggregation in general), individual decision trees are connected only in the sense that their training samples partially overlap. After their training datasets are sampled, they learn independently.
What’s the optimal number of trees in the random forest?
What I’ve found: the best performance is obtained by selecting the optimal tree number with single tree precision (I’ve tried to tune the number of trees with 100 trees and single tree step precision), the optimal number of trees in the Random Forest depends on the number of rows in the data set.
How is the prediction of the random forest?
The prediction of the Random Forest is the average from all trees in the subset (I’m doing manually what is done internally in predict_proba in the Random Forest). As ealier, the final response is the average over all 5 models (from internal CV). Notice, that each model from internal CV can have (and have) different number of trees.
Why are 10 trees better than 500 trees?
In your case you observe that in a single experiment on a single test set a forest of 10 trees performs better than a forest of 500 trees. This may be due to statistical variance. If this would happen systematically, I would hypothesize that there is something wrong with the implementation.
How do you determine the number of trees?
If you do not have any concern regarding the computation times, the more trees you have, the better (reliable) estimates you get from out-of-bag predictions. You can use OOB error rate to determine the number of trees.