Contents
Is the validation accuracy less than the training accuracy?
It is not overfitting since your validation accuracy is not less than the training accuracy. In fact, it sounds like your model is underfitting since your validation accuracy > training accuracy.
How is it possible that validation loss is possible?
For example, if an image of a cat is passed into two models. Model A predicts {cat: 0.9, dog: 0.1} and model B predicts {cat: 0.6, dog: 0.4}. Both model will score the same accuracy, but model A will have a lower loss. Because of this the model will try to be more and more confident to minimize loss.
What does validation accuracy mean for binary classification?
Your validation accuracy on a binary classification problem (I assume) is “fluctuating” around 50%, that means your model is giving completely random predictions (sometimes it guesses correctly few samples more, sometimes a few samples less). Generally, your model is not better than flipping a coin.
Why is the validation loss more stable in machine learning?
The reason the validation loss is more stable is that it is a continuous function: It can distinguish that prediction 0.9 for a positive sample is more correct than a prediction 0.51. For accuracy, you round these continuous logit predictions to { 0; 1 } and simply compute the percentage of correct predictions.
Why does the loss / accuracy fluctuate during the training?
For batch_size=2 the LSTM did not seem to learn properly (loss fluctuates around the same value and does not decrease). Upd. 4: To see if the problem is not just a bug in the code: I have made an artificial example (2 classes that are not difficult to classify: cos vs arccos). Loss and accuracy during the training for these examples:
Which is better validation data or training data?
Even though the data is under fitted, the validation data may perform well under circumstances that the validation data fits better in your model than does training data. Increase more convolution layers and loosen up on Dropout layers, lesser dropout layers or lower percentage of units dropped out.
How many epochs does it take to reach 100% accuracy?
And here are the loss&accuracy during the training: (Note that the accuracy actually does reach 100% eventually, but it takes around 800 epochs.) I thought that these fluctuations occur because of Dropout layers / changes in the learning rate (I used rmsprop/adam), so I made a simpler model: