Contents
Why is my AUC so high?
One possible reason you can get high AUROC with what some might consider a mediocre prediction is if you have imbalanced data (in favor of the “zero” prediction), high recall, and low precision.
What does ROC mean in statistics?
receiver operating characteristic curve
An ROC curve (receiver operating characteristic curve) is a graph showing the performance of a classification model at all classification thresholds.
Should AUC be high or low?
The Area Under the Curve (AUC) is the measure of the ability of a classifier to distinguish between classes and is used as a summary of the ROC curve. The higher the AUC, the better the performance of the model at distinguishing between the positive and negative classes.
What’s the difference between ROC AUC and accuracy?
The big question is when. The first big difference is that you calculate accuracy on the predicted classes while you calculate ROC AUC on predicted scores. That means you will have to find the optimal threshold for your problem. Moreover, accuracy looks at fractions of correctly assigned positive and negative classes.
How does AUC relate to true positive rate?
AUC measures how true positive rate (recall) and false positive rate trade off, so in that sense it is already measuring something else. More importantly, AUC is not a function of threshold. It is an evaluation of the classifier as threshold varies over all possible values. It is in a sense a broader metric,…
When is AUC is not computable for a problem?
AUC is not computable if you truly only have a black-box classifier, and not one with an internal threshold. These would usually dictate which of the two is even available to a problem at hand.
Is the AUC aggregate over confidence threshold good or bad?
AUC aggregate over confidence threshold, for good and bad. For good, you get a weight result for all confidence level. The bad is that you are usually care only about the confidence level you will actually use and the rest are irrelevant. However, I want to remark about choosing a proper performance measure for a model.