Contents
What is exclusive feature bundling?
Exclusive Feature Bundling (EFB) is a near-lossless method to reduce the number of effective features. In a sparse feature space many features are nearly exclusive, implying they rarely take nonzero values simultaneously. One-hot encoded features are a perfect example of exclusive features.
What makes LightGBM fast?
There are three reasons why LightGBM is fast: Histogram based splitting. Gradient-based One-Side Sampling (GOSS) Exclusive Feature Bundling (EFB)
How does light gradient boost work?
Gradient boosting refers to a class of ensemble machine learning algorithms that can be used for classification or regression predictive modeling problems. Ensembles are constructed from decision tree models. Trees are added one at a time to the ensemble and fit to correct the prediction errors made by prior models.
Why does LightGBM use exclusive feature bundling?
1)It is possible that LightGBM Framework can find out that we give the features as one-hot-encoded from the sparsity, it is possible that the algorithm does not treat one-hot-encoded with EFB. 2)It is also possible that LightGBM uses EFB on one-hot-encoded samples but it may be harmful, or not good as EFB on direct categorical inputs.
How does the exclusive feature bundling algorithm work?
I’m currently studying GBDT and started reading LightGBM’s research paper. In section 4. they explain the Exclusive Feature Bundling algorithm, which aims at reducing the number of features by regrouping mutually exclusive features into bundles, treating them as a single feature.
Why does LightGBM use EfB on one hot encoded samples?
2)It is also possible that LightGBM uses EFB on one-hot-encoded samples but it may be harmful, or not good as EFB on direct categorical inputs. (I go for this one) But still, I do not think that EFB will reverse one-hot-encoding since EFB is explained as a unique way of treating the categorical features.
What makes an EFB dataset more efficient?
EFB (Exclusive Feature Bundling) Datasets with a high number of features are likely to have sparse features (i.e. lots of zero values). These sparse features are usually mutually exclusive which means they do not have non-zero values simultaneously. Consider the case of one-hot encoded text data.