Contents
Can PCA handle categorical data?
While it is technically possible to use PCA on discrete variables, or categorical variables that have been one hot encoded variables, you should not. Simply put, if your variables don’t belong on a coordinate plane, then do not apply PCA to them.
Can we apply scaling to categorical variables?
Encoded categorical variables contain values on 0 and 1. Therefore, there is even no need to scale them. However, scaling methods will be applied to them when you choose to scale your entire dataset prior to using your data with scale-sensitive ML models.
When should I scale before PCA?
Yes, it is necessary to normalize data before performing PCA. The PCA calculates a new projection of your data set. And the new axis are based on the standard deviation of your variables.
How to deal with categorical values before PCA?
In the preprocessing phase, I converted all the categorical values in numerical, so that the software can deal with them (basically I created dummy variables). Now, in order to apply PCA I have to scale the data matrix such that I have mean equal to 0.
When to scale before or after PCA data?
If you are getting a number of PCA components for multiple features it is best to scale them as with features of different size, your algorithm might interpret one as more important than others without any real reason.
How to calculate PCA for categorical variables in Python?
The following publication shows great and meaningful results when computing PCA on categorical variables treated as simplex vertices: Niitsuma H., Okada T. (2005) Covariance and PCA for Categorical Variables. In: Ho T.B., Cheung D., Liu H. (eds) Advances in Knowledge Discovery and Data Mining.
Which is better a bigger scale or PCA?
A bigger scale would create bigger values and square of a big value grow faster. But it does not mean that the price carry significantly more information compared to for instance location. In this example, however, PCA would give a very high weight to the price feature and perhaps the weights of categorical features would almost drop to 0.