How to deal with the class imbalance in binary classification?
Here’s a brief description of my problem: I am working on a supervised learning task to train a binary classifier. I have a dataset with a large class imbalance distribution: 8 negative instances every one positive. I use the f-measure, i.e. the harmonic mean between specificity and sensitivity, to assess the performance of a classifier.
Which is more important in an imbalanced classification problem?
When working with an imbalanced classification problem, the minority class is typically of the most interest. This means that a model’s skill in correctly predicting the class label or probability for the minority class is more important than the majority class or classes.
How is the degree of class imbalance determined?
Since class labels are required in order to determine the degree of class imbalance, class imbalance is typically gauged with respect to the training distribution. — Page 16, Imbalanced Learning: Foundations, Algorithms, and Applications, 2013. It is common to describe the imbalance of classes in a dataset in terms of a ratio.
How to describe the imbalance of classes in a dataset?
Another way to describe the imbalance of classes in a dataset is to summarize the class distribution as percentages of the training dataset. For example, an imbalanced multiclass classification problem may have 80 percent examples in the first class, 18 percent in the second class, and 2 percent in a third class.
Can you have a class imbalance on a multi class classification problem?
You can have a class imbalance problem on two-class classification problems as well as multi-class classification problems. Most techniques can be used on either. The remaining discussions will assume a two-class classification problem because it is easier to think about and describe.
Is there a limit to the accuracy of binary classification?
The limitation here is that you are making absolute trade-offs. Any modification in the cutoff will in turn decrease the accuracy of predicting the other class. If you have exceedingly high probabilities for the majority of your common classes (e.g. most above 0.85) you are more likely to have success with this method.