Contents
When to use binary cross entropy in autoencoder?
I am working on an autoencoder for non-binary data ranging in [0,1] and while I was exploring existing solutions I noticed that many people (e.g., the keras tutorial on autoencoders, this guy) use binary cross-entropy as the loss function in this scenario.
Which is an example of a cross entropy loss?
Any loss consisting of a negative log-likelihood is a crossentropy between the empirical distribution defined by the training set and the probability distribution defined by model. For example, mean squared error is the cross-entropy between the empirical distribution and a Gaussian model.
Is the NNL the same as the cross entropy?
From what I’ve googled, the NNL is equivalent to the Cross-Entropy, the only difference is in how people interpret both. However when I go on wikipedia on the Cross-Entropy page, what I find is: Question 1: Why are they raising the estimated outcome to the power of the (N * training outcome).
Why is binary cross entropy used in VAE?
This discussion suggests that binary cross entropy is used in VAE case mainly for better optimization behavior. Another reason it works well is that MNIST dataset roughly follows multivariate Bernoulli distribution – the pixel values are close to either zero or one and binarization does not change it much.
How does binary cross entropy loss work in machine learning?
As for the loss function, it comes back to the values of input data again. If the input data are only between zeros and ones (and not the values between them), then binary_crossentropy is acceptable as the loss function. Otherwise, you need to use other loss functions such as ‘mse’ (i.e. mean squared error) or ‘mae’ (i.e. mean absolute error).
When to use binary crossentropy in binray classification?
I know that binary crossentropy can be used in binray classification problems where the ground-truth labels (i.e. y) are either 0 or 1 and therefore when predictions (i.e. p) are correct, in both cases, the loss value would be zero:
When does binary crossentropy have a value of zero?
However, binary crossentropy does not have a value of zero when neither of its arguments are both zero or one, which is the case for an autoencoder with ground-truth labels in range [ 0, 1] (i.e. assuming the input data has been normalized in this range).