Contents
Does gradient descent require feature scaling?
In data processing, it is usually used in data pre-processing. Because in the original data, the range of variables is very different. Feature scaling is a necessary step in the calculation of stochastic gradient descent.
Is gradient descent used in practice?
In actual practice we use an approach called Mini batch gradient descent. This approach uses random samples but in batches. What this means is that we do not calculate the gradients for each observation but for a group of observations which results in a faster optimization.
How does feature scaling affect the gradient descent?
Having features on a similar scale can help the gradient descent converge more quickly towards the minima. Distance algorithms like KNN, K-means, and SVM are most affected by the range of features. This is because behind the scenes they are using distances between data points to determine their similarity.
When do you use feature scaling in machine learning?
Machine learning algorithms like linear regression, logistic regression, neural network, etc. that use gradient descent as an optimization technique require data to be scaled. Take a look at the formula for gradient descent below: The presence of feature value X in the formula will affect the step size of the gradient descent.
How is gradient descent used in machine learning?
Machine learning algorithms like linear regression, logistic regression, neural network, etc. that use gradient descent as an optimization technique require data to be scaled. Take a look at the formula for gradient descent below:
Which is better feature scaling or tree based algorithms?
Scaling has brought both the features into the picture and the distances are now more comparable than they were before we applied scaling. Tree-based algorithms, on the other hand, are fairly insensitive to the scale of the features. Think about it, a decision tree is only splitting a node based on a single feature.