Contents
Does squared loss function sensitive to outliers justify your answer?
The MSE loss function penalizes the model for making large errors by squaring them. This property makes the MSE cost function less robust to outliers. Therefore, it should not be used if our data is prone to many outliers.
Is exponential loss sensitive to outliers?
The exponential loss is convex and grows exponentially for negative values which makes it more sensitive to outliers.
Which ML models are sensitive to outliers?
Many machine learning models, like linear & logistic regression, are easily impacted by the outliers in the training data. Models like AdaBoost increase the weights of misclassified points on every iteration and therefore might put high weights on these outliers as they tend to be often misclassified.
What is the loss function after removing outliers?
To be exact the loss function after removing outliers (say 5% extremes, so 2.5% each side) looks like that: where is the loss function, and is the error (difference between prediction and target). I mean to say that there is still a cost, which is not zero. This is on the math side, but we can argue further.
What does a high value for the loss function mean?
A high value for the loss means our model performed very poorly. A low value for the loss means our model performed very well. Selec t ion of the proper loss function is critical for training an accurate model.
Which is the output of the loss function?
The output of the loss function is called the loss which is a measure of how well our model did at predicting the outcome. A high value for the loss means our model performed very poorly. A low value for the loss means our model performed very well.
Why do we use Mae for larger loss values?
Using the MAE for larger loss values mitigates the weight that we put on outliers so that we still get a well-rounded model. At the same time we use the MSE for the smaller loss values to maintain a quadratic function near the centre. This has the effect of magnifying the loss values as long as they are greater than 1.