What is the consistency of the LASSO model?
On the model selection consistency front, Meinshausen and Buhlmann (2006) have shown that under a set of conditions, Lasso is consistent in estimating the dependency between Gaussian variables even when the number of variables p grow faster than n. Leng et al.
What do you need to know about variable selection?
The paper is intended for the practicing statistician, whose routine work includes the development, estimation, and interpretation of statistical models for observational data to help answering research questions in life sciences.
Who are the authors of the variable selection paper?
This paper is based on a lecture held in the series “Education for Statistics in Practice” by Georg Heinze and Daniela Dunkler at the conference of the “Deutsche Arbeitsgemeinschaft Statistik” in Göttingen, Germany, March 2016.
Which is the best method for stability selection?
In order to put stability selection in context, we \\frst describe several existing methods for performing model selection. Cross-validation. A method for choosing the value of \hat minimizes the generalization error on a held-out test validation set. This approach only works for supervised problems such as regression.
What’s the difference between SVM and stability selection?
They both build on top of other (model based) selection methods such as regression or SVM, building models on different subsets of data and extracting the ranking from the aggregates. As a wrap-up I’ll run all previously discussed methods, to highlight their pros, cons and gotchas with respect to each other.
Why is stability selection useful for data interpretation?
This means stability selection is useful for both pure feature selection to reduce overfitting, but also for data interpretation: in general, good features won’t get 0 as coefficients just because there are similar, correlated features in the dataset (as is the case with lasso).
When is the lasso estimator not variable selection consistent?
One well-known result is that the Lasso estimator is not variable selection consistent if the irrepresentable condition fails (Meinshausen and Buhlmann, 2006; Zhao and Yu, 2007), which means the correct sparse subset of the relevant variables can not be identi\\fed asymptotically with large probabil- ity.
Is the group lasso an asymp Totic upper bound?
Our main theorem shows that the group Lasso achieves estimation consis- tency under a mild condition and an asymp- totic upper bound on the number of selected variables can be obtained.