Contents
What is orthogonal in PCA?
PCA is defined as an orthogonal linear transformation that transforms the data to a new coordinate system such that the greatest variance by some scalar projection of the data comes to lie on the first coordinate (called the first principal component), the second greatest variance on the second coordinate, and so on.
Are principal components eigenvectors?
The eigenvectors and eigenvalues of a covariance (or correlation) matrix represent the “core” of a PCA: The eigenvectors (principal components) determine the directions of the new feature space, and the eigenvalues determine their magnitude.
Are principal components orthonormal?
The principal components are the eigenvectors of a covariance matrix, and hence they are orthogonal. Importantly, the dataset on which PCA technique is to be used must be scaled. The results are also sensitive to the relative scaling.
Why do eigenvectors have to be orthogonal in PCA?
In the latter case, the eigenspace for that repeated eigenvalue can contain eigenvectors which are not orthogonal. So your instinct to question why the eigenvectors have to be orthogonal was a good one; if there are repeated eigenvalues they may not be!
Why are the eigenvectors of a symmetric matrix orthogonal?
This has repeated eigenvalue 1 and any two non-zero vectors are eigenvectors, orthogonal or not. (Thinking out such special cases is often a good way to spot counter-examples.) If a symmetric matrix has a repeated eigenvalue, we can choose to pick out orthogonal eigenvectors from its eigenspace.
Why are computed eigenvectors of the covariance orthonormal?
The computed eigenvectors are the columns of Z so we can see LAPACK guarantees they will be orthonormal (if you want to know quite how the orthogonal vectors of T are picked, using a Relatively Robust Representations procedure, have a look at the documentation for DSYEVR ).
Which is the vector of eigenvalues of T?
Here Λ is the vector of eigenvalues (of T and also of A – they work out the same) and since Z is the product of two orthogonal matrices it is also orthogonal.