Contents
Can PCA be used for regression analysis?
In statistics, principal component regression (PCR) is a regression analysis technique that is based on principal component analysis (PCA). In PCR, instead of regressing the dependent variable on the explanatory variables directly, the principal components of the explanatory variables are used as regressors.
Are principal components latent variables?
A principal component is a linear combination of weighted observed variables. Principal components are uncorrelated and orthogonal. A latent construct can be measured indirectly by determining its influence to responses on measured variables. Principal component scores are actual scores.
How are principal components used in PC regression?
To perform principal components (PC) regression, we transform the independent variables to their principal components. Mathematically, we write X’X =PDP’=Z’Z where is a diagonal matrix of the eigenvalues of D X’X, P is the eigenvector matrix of X’X, and Z is a data matrix (similar in structure to X) made up of the principal components.
Why do we need control variables in regression analysis?
Why do we need control variables? ¶ A major strength of regression analysis is that we can control relationships for alternative explanations. You’ve probably heard the expression “correlation is not causation.” It means that just because we can see that two variables are related, one did not necessarily cause the other.
As you must have noticed we don’t have any of the original numeric variables in this model but for the uncorrelated principal components i.e. Comp 1 and Comp 2. Moreover, we have run the stepwise regression to remove insignificant variables and components.
Do you use raw variables in principal component analysis?
Moreover, during operationalization of models, principal components add another level of complexity. Hence, it is a good idea if possible, to build the model with the original raw variables. You may remember this table from the previous part of this article on principal component analysis.