Contents
What are the consequences of multicollinearity?
Statistical consequences of multicollinearity include difficulties in testing individual regression coefficients due to inflated standard errors. Thus, you may be unable to declare an X variable significant even though (by itself) it has a strong relationship with Y.
Multicollinearity occurs when independent variables in a regression model are correlated. This correlation is a problem because independent variables should be independent. If the degree of correlation between variables is high enough, it can cause problems when you fit the model and interpret the results.
What would be then consequences for the OLS estimator?
Correct! The consequences of autocorrelation are similar to those of heteroscedasticity. The OLS estimator will be inefficient in the presence of autocorrelation, which implies that the standard errors could be sub-optimal.
How do you know if there is a correlation between two variables?
The correlation coefficient is measured on a scale that varies from + 1 through 0 to – 1. Complete correlation between two variables is expressed by either + 1 or -1. When one variable increases as the other increases the correlation is positive; when one decreases as the other increases it is negative.
One of the most commonly used methods is Variation Inflation Factor (VIF). A VIF value of > 10 means that the variable is more than 90% correlation with other variables in the dataset and we can choose to remove these variables and re-run the model.
However, when independent variables are correlated, it indicates that changes in one variable are associated with shifts in another variable. The stronger the correlation, the more difficult it is to change one variable without changing another.
Which is an example of a highly correlated variable?
For example, highly correlated variables might cause the first component of PCA to explain 95% of the variances in the data. Then, you can simply use this first component in the model. Random forests can also be used for feature selection by looking at the feature importances of the variable.
How to do PCA of highly correlated variables?
Any help is appreciated. Perform a PCA or MFA of the correlated variables and check how many predictors from this step explain all the correlation. For example, highly correlated variables might cause the first component of PCA to explain 95% of the variances in the data. Then, you can simply use this first component in the model.