Why is my loss value so high?
Loss is a value that represents the summation of errors in our model. It measures how well (or bad) our model is doing. If the errors are high, the loss will be high, which means that the model does not do a good job. Otherwise, the lower it is, the better our model works.
Is high loss bad?
A loss is a number indicating how bad the model’s prediction was on a single example. Higher loss is the worse(bad prediction) for any model. The loss is calculated on training and validation and its interpretation is how well the model is doing for these two sets. Unlike accuracy, a loss is not a percentage.
Why does my neural network have a high loss value?
Since the total loss is the average you get a high loss on that set even though it is performing very well on all the points but one. Your loss might be hijacked by a few outliers, check the distribution of your loss function on individual samples of your validation set. If there are a cluster of values around the mean then you are overfitting.
What causes a neural network to underfit?
Augmentation has a regularizing effect. Too much of this combined with other forms of regularization (weight L2, dropout, etc.) can cause the net to underfit. 14. Check the preprocessing of your pretrained model If you are using a pretrained model, make sure you are using the same normalization and preprocessing as the model was when training.
Can a neural network still learn if there is an outlier?
In that sense it is possible that there are one or a few outliers that are classified extremely badly and that are making the loss explode, but at the same time your model is still learning on the rest of the dataset. In the following example I use a very simple dataset in which there is an outlier in the test data.
What should I do when my neural network doesn’t learn?
Residual connections can improve deep feed-forward networks. To achieve state of the art, or even merely good, results, you have to have to have set up all of the parts configured to work well together. Setting up a neural network configuration that actually learns is a lot like picking a lock: all of the pieces have to be lined up just right.