What is feature scaling in linear regression?
Feature Scaling is a technique to standardize the independent features present in the data in a fixed range. If feature scaling is not done, then a machine learning algorithm tends to weigh greater values, higher and consider smaller values as the lower values, regardless of the unit of the values.
Why is feature scaling used?
Feature scaling is a method used to normalize the range of independent variables or features of data. In data processing, it is also known as data normalization and is generally performed during the data preprocessing step.
When do we need to perform feature scaling?
We need to perform Feature Scaling when we are dealing with Gradient Descent Based algorithms (Linear and Logistic Regression, Neural Network) and Distance-based algorithms (KNN, K-means, SVM) as these are very sensitive to the range of the data points. This step is not mandatory when dealing with Tree-based algorithms.
Do I need to do feature scaling for simple linear regression?
The exception, of course, is when you apply regularization. Then linear scaling can change the results dramatically. That’s actually another reason to do feature scaling, but since you asked about simple linear regression, I won’t go into that.
How does feature scaling affect the gradient descent?
Having features on a similar scale can help the gradient descent converge more quickly towards the minima. Distance algorithms like KNN, K-means, and SVM are most affected by the range of features. This is because behind the scenes they are using distances between data points to determine their similarity.
When do you need to scale features in regularization?
When you start introducing regularization, you will again want to scale the features of your model. The penalty on particular coefficients in regularized linear regression techniques depends largely on the scale associated with the features.