Contents
Solution: If two variables are highly correlated, keeping only one will help reduce dimensionality without much loss of information.
Why is dimensionality reduction important in machine learning and predictive modeling?
Dimensionality reduction refers to techniques for reducing the number of input variables in training data. When dealing with high dimensional data, it is often useful to reduce the dimensionality by projecting the data to a lower dimensional subspace which captures the “essence” of the data.
When to remove a feature to reduce dimensionality?
We can always revisit this technique and consider removing features with low variance. Features that are highly correlated or co-linear can cause overfitting. When a pair of variables are highly correlated we can remove one to reduce dimensionality without much loss of information. Which one should we keep?
What are the two main methods of dimensionality reduction?
There are two key methods of dimensionality reduction: Feature selection: Here, we select a subset of features from the original feature set. Feature extraction: With this technique, we generate a new feature set by extracting and combining information from the original feature set.
When to use PCA in dimensionality reduction?
PCA can be useful in many situations, but especially in cases with excessive multicollinearity or explanation of predictors is not a priority. Here we will apply PCA and keep 90% of the variance:
How does Recursive feature selection and dimensionality reduction work?
Recursive feature selection works by eliminating the least important features. It continues recursively until the specified number of features is reached. Recursive elimination can be used with any model that assigns weights to features, either through coef_ or feature_importances_ Here we will use Random Forest to select the 100 best features: