Contents
When to use random forest over SVM and vice versa?
Indeed as @Ianenok, SVMs tend to be unusable beyond 10 000 data points. However, SVMs are known to perform better on some specific datasets (images, microarray data…). So, once again, cross validation is indeed the best way to know which method performs best.
How many columns are dense in a random forest?
Out of this 650 columns, 500 are dense (nonzero over 10%) and 150 are sparse (nonzero less than 10%). As expected, the random forest has difficulty selecting the sparse features for any splits, while they are expected to hold at least some predictive power (A forest with only these features can obtain a gini of 0.2).
Can a sparse feature be included in a dense feature?
A second thought was to apply PCA to the sparse features to obtain a smaller set of features, and include those together with the dense features in a random forest. However, because of the large amount of zero’s, one component already captures >99.99% of the variance.
How big of a problem can a SVM solve?
Therefore, as a rule of thumb, SVM is hardly scalable beyond 10^5 points. Large number of features (homogeneous features with meaningful distance, pixel of image would be a perfect example) is generally not a problem. For a classification problem Random Forest gives you probability of belonging to class.
Which is better a random forest or a support vector machine?
Recall the table from the article about time complexity. What we can see is that the computational complexity of Support Vector Machines (SVM) is much higher than for Random Forests (RF). This means that training a SVM will be longer to train than a RF when the size of the training data is higher.
When to use a random forest vs a RF?
This means that training a SVM will be longer to train than a RF when the size of the training data is higher. This has to be considered when chosing the algorithm. Typically, SVMs tend to become unusable when the number of rows exceeds 20 000. Therefore, random forests should be prefered when the data set grows larger.