Contents
What does the varimp function in caret package actually compute?
Your help is very much appreciated! For these models, they are regression the coefficients for the final Model. Big coefficients are associated with larger effects. Using scale = FALSE is good here so you can also get the signs too. There are always pitfalls with these measures depending on how you want to measure importance.
How to find the difference between varimp and importance?
I computed a simple RF classification model and when computing variable importance, I found that the “ranking” of predictors was not the same for both functions:
Which is a wrapper around the function varimp.gbm?
Boosted Trees: varImp.gbm is a wrapper around the function from that package (see the gbm package vignette) Multivariate Adaptive Regression Splines: MARS models include a backwards elimination feature selection routine that looks at reductions in the generalized cross-validation (GCV) estimate of error.
How is the varimp function used in Mars?
Currently, the varImp is a wrapper to the evimp function in the earth package. There are three statistics that can be used to estimate variable importance in MARS models. Using varImp (object, value = “gcv”) tracks the reduction in the generalized cross-validation statistic as terms are added.
How does Lasso compare to caret feature selection?
Lasso, on the other hand, is somewhat indifferent to very correlated predictors, and will tend to pick one and ignore the rest. caret actually looks at the final coefficients of the fit and then takes the absolute value to rank the coefficients. Then the ranked coefficients are stored as variable importance.
How does caret look at the final coefficients?
caret actually looks at the final coefficients of the fit and then takes the absolute value to rank the coefficients. Then the ranked coefficients are stored as variable importance.
Is it good to use scale = false in caret?
Using scale = FALSE is good here so you can also get the signs too. There are always pitfalls with these measures depending on how you want to measure importance. They don’t measure lack of fit at all, so if your model is 51% accurate, they are not very reflective of the data.