Contents
What does training error mean?
There are two important concepts used in machine learning: the training error and the test error. Training Error: We get the by calculating the classification error of a model on the same data the model was trained on (just like the example above).
Can we reduce the training error to zero?
If boosting does succeed in reducing the error on fresh test data, it often does so in a spectacular way. One very surprising finding is that performing more boosting iterations can reduce the error on new data long after the classification error of the combined classifier on the training data has dropped to zero.
Do we need zero training loss a er achieving zero training error?
Overparameterized deep networks have the capacity to memorize training data with zero \emph{training error}. Even after memorization, the \emph{training loss} continues to approach zero, making the model overconfident and the test performance degraded.
Will happen with bias and variance as you increase the size of training data?
25) What do you expect will happen with bias and variance as you increase the size of training data? As we increase the size of the training data, the bias would increase while the variance would decrease. Question Context 26: Consider the following data where one input(X) and one output(Y) is given.
Is it possible to get zero training error?
The accepted answer is a bit complicated, so here’s a simpler answer: You can get zero training error by chance, with any model. Say your biased classifier always predicts zero, but your dataset happens to be all labeled zero.
Why are we unable to use the bias-variance tradeoff?
The simple answer is that we are unable to use this approach because there is no guarantee that the model with the lowest training MSE will also be the model with the lowest test MSE. Why is this so? The answer lies in a particular property of statistical machine learning methods known as the bias-variance tradeoff.
What do you mean by zero sum bias?
The zero-sum bias is a cognitive bias that causes people to mistakenly view certain situations as being zero-sum, meaning that they incorrectly believe that one party’s gains are directly balanced by other parties’ losses.
When to use k = 1 for training error?
When K = 1, you’ll choose the closest training sample to your test sample. Since your test sample is in the training dataset, it’ll choose itself as the closest and never make mistake. For this reason, the training error will be zero when K = 1, irrespective of the dataset.