Contents
Why is standard scaler used?
StandardScaler removes the mean and scales each feature/variable to unit variance. This operation is performed feature-wise in an independent way. StandardScaler can be influenced by outliers (if they exist in the dataset) since it involves the estimation of the empirical mean and standard deviation of each feature.
Why do we use MIN MAX scaler?
Transform features by scaling each feature to a given range. This estimator scales and translates each feature individually such that it is in the given range on the training set, e.g. between zero and one.
What is scaler fit?
The idea behind StandardScaler is that it will transform your data such that its distribution will have a mean value 0 and standard deviation of 1. In case of multivariate data, this is done feature-wise (in other words independently for each column of the data).
Why do I use scaler instead of X _ test?
If you would fit the scaler using x_test you would be using information from your test set and would be leaking data. This is information that you would not have if your model was in production and can therefore not use when fitting your model.
Why do you fit Scaler on all training folds?
Again, remember to fit the scaler on all the training folds and apply it to the test fold for every iteration. The main reason for that is the assumption that we make that our model has never seen our test data or have any information about it.
When to use scaler.fit _ transform on total?
If you are worried that your training data does not adequately reflect the true distribution, you can fall back to a cross-validation approach with multiple folds so you can estimate the effect across multiple splits of the data. Again, remember to fit the scaler on all the training folds and apply it to the test fold for every iteration.
What happens when I normalize the train and test data?
After which I will train and test the model ( A, B as features, C as Label) and get some accuracy score. Now my doubt is, what happens when I have to predict the label for new set of data. Say, Because when I normalize the column the values of A and B will be changed according to the new data, not the data which the model will be trained on.