Contents
What happens when the model complexity increases?
When you increase complexity of your model, it is more likely to overfit, meaning it will adapt to training data very well, but will not figure out general relationships in the data. In such case, performance on a test set is going to be poor.
What typically happens to bias and variance when the model complexity increases decreases?
“More generally, as the model complexity of our procedure is increased, the variance tends to increase and the squared bias tends to decrease.”
What happens when the complexity of a model goes up?
When you increase complexity of your model, it is more likely to overfit, meaning it will adapt to training data very well, but will not figure out general relationships in the data. In such case, performance on a test set is going to be poor.
Why are training errors decreasing but test errors increase?
On final iteration results for each dataset you can observe that model with fever training samples producing lower training error and higher test error, and with increase in size of data, training errors are increasing but test errors are decreasing. I’m really puzzled…
What happens when Model Complexity goes up in deep learning?
You can use more complex model, like polynominal of degree equal to size of your training set. Your model will be great at memorizing a training set, but will most likely fail to model a linear relationship properly where a 1st degree polynominal does a great job. This leads to poor test set performance.
Why are both training and testing loss decrease?
So by increasing training data, training loss is increased but test loss decreases and that is what expected for Prediction. Reduce model complexity: Same applies here when you reduce model complexity, your model can’t remember all training data. So training loss increases and test loss decreases.