Contents
Why training loss is more than validation loss?
If your training loss is much lower than validation loss then this means the network might be overfitting . Solutions to this are to decrease your network size, or to increase dropout. For example you could try dropout of 0.5 and so on. If your training/validation loss are about equal then your model is underfitting.
What is validation loss in deep learning?
“Validation loss” is the loss calculated on the validation set, when the data is split to train / validation / test sets using cross-validation.
What to do if your training loss is lower than Your Validation loss?
If your training loss is much lower than validation loss then this means the network might be overfitting. Solutions to this are to decrease your network size, or to increase dropout. For example you could try dropout of 0.5 and so on. If your training/validation loss are about equal then your model is underfitting.
What to do about validation loss in machine learning?
Really a fundamental question in machine learning. If validation loss >> training loss you can call it overfitting. If validation loss > training loss you can call it some overfitting. If validation loss < training loss you can call it some underfitting. If validation loss << training loss you can call it underfitting.
When does training loss go down to zero?
The training-loss goes down to zero. That means your model is sufficient to fit the data. If the training-loss would get stuck somewhere, that would mean the model is not able to fit the data. So, your model is flexible enough.
When does validation loss start to go down?
Furthermore the validation-loss goes down first until it reaches a minimum and than starts to rise again. If your validation set is large enough and representative you can argue that your model starts to overfit the data.