What is a good area under the precision-recall curve?

What is a good area under the precision-recall curve?

An ideal PR-curve goes from the topleft corner horizontically to the topright corner and straight down to the bottomright corner, resulting in a PR-AUC of 1.

Why is ROC bad for Imbalanced Data?

Although widely used, the ROC AUC is not without problems. For imbalanced classification with a severe skew and few examples of the minority class, the ROC AUC can be misleading. This is because a small number of correct or incorrect predictions can result in a large change in the ROC Curve or ROC AUC score.

Why you should stop using the ROC curve?

“You should stop using the ROC-curve, you should use Average-Precision instead.” Because, when we rely on a metric — such as ROC-Area or Average-Precision — we are assuming that the many facets of a model performance can be enclosed in a single number.

Which is more robust ROC or precision recall?

An ROC curve tends to be more robust to class imbalanace that a precision-recall curve. In this animation [below], both distributions start with 1000 outcomes. The blue one is then reduced to 50. The precision-recall curve changes shape more drastically than the ROC curve, and the AUC value mostly stays the same.

How are precision recall curves and ROC curves related?

The Precision-Recall AUC is just like the ROC AUC, in that it summarizes the curve with a range of threshold values as a single score. The score can then be used as a point of comparison between different models on a binary classification problem where a score of 1.0 represents a model with perfect skill.

Which is better precision recall or precision recall?

Typically, a model that produces a precision-recall curve that is closer to the top-right corner is better than a model that produces a precision-recall curve that is skewed towards the bottom of the plot. Class imbalance happens when the number of outputs in one class is different from the number of outputs in another class.

How is AUC different from Roc and precision?

The precision-recall curve changes shape more drastically than the ROC curve, and the AUC value mostly stays the same. We also observe this behaviour when the other disribution is reduced to 50. Here’s the same, but now with the red distribution shrinking to just 50 samples.