Contents
How to use keras stateful LSTM for validation loss?
I had multiple inputs: one called seq_model_in that is a time series, and one called feat_in that is not a time series (and so is concatenated in to the model after the LSTM but before the classification step). As my classes are highly imbalanced, I also used Keras’s class_weights function.
How does the sequential model in keras work?
I’m implementing a neural network with Keras, but the Sequential model returns nan as loss value. I have sigmoid activation function in the output layer to squeeze output between 0 and 1, but maybe doesn’t work properly.
Why is the NaN value in loss in keras?
The shape your logits is [None, 1], so keras assumes that the number of your classes is 1, but your are feeding more than one classes ( 0 or 1) and that is causing the error. The right activation function here is sigmoid (tanh also could work by altering the dataset target to be either -1 or 1).
Why are ReLU and softmax activations bad for keras?
Sometimes with Keras the combination of Relu and Softmax causes numerical troubles as Relu can produce large positive values corresponding to very small probabilities. Softmax activation is not the right choice here. You have only one neuron on the output layer.
Why does the validation loss return Nan for Val loss?
Why does the validation loss return NaN? No idea about the tensor board stuff, but the NAN for val loss could be being caused by an unexpected very large or very small number. As your training dataset is very small there is a high chance that the validation data has a very different underlying distribution.
What happens if validation samples fall too far away?
If the latent space of your NN isn’t continuous and validation samples fall too far away from the distribution of the training samples, the NN will not deal with them well. In a classification problem this would result in misclassification (most likely).