Contents
Is it possible to get feature importance from weights of hyperplane in logistic regression?
Logistic Regression An inherently binary classification algorithm, it tries to find the best hyperplane in k-dimensional space that separates the 2 classes, minimizing logistic loss. The k dimensional weight vector can be used to get feature importance.
When would you use an SVM and when logistic regression?
SVM try to maximize the margin between the closest support vectors whereas logistic regression maximize the posterior class probability. SVM is deterministic (but we can use Platts model for probability score) while LR is probabilistic. For the kernel space, SVM is faster.
How to use weight vector of SVM and..?
Both classifier provide a weight vector which is of the size of the number of features. I can use this weight vector to select the 10 most important features by just selecting the 10 features with the highest weights. Should I use the absolute values of the weights, i.e. selecting the 10 features with the highest absolute values?
Can you use weight vectors in logistic regression?
Therefore when feature are multicollinear we cannot use weight vectors to get feature importance in Logistic Regression. Does multicollinearity affect Support Vector Machines ?
How is linear kernel of support vector machine similar to logistic regression?
Linear Kernel of Support vector Machines is very similar to Logistic Regression, and hence the effect of multicollinearity has a very similar effect in case of Linear Kernel of SVM. We have to remove multicollinearity , if we want to use weight vectors directly for feature importance.
Why is the weight vector of a classifier important?
As I answered to similar question, weight vector of any linear classifier indicates feature importance: simply because final value is a linear combination of feature values with weights as coefficients, so the bigger weight, the more impact to the final value is caused by the corresponding summand.