Why LightGBM is fast?
There are three reasons why LightGBM is fast: Histogram based splitting. Gradient-based One-Side Sampling (GOSS) Exclusive Feature Bundling (EFB)
What is Goss LightGBM?
What is GOSS? GOSS (Gradient Based One Side Sampling) is a novel sampling method which down samples the instances on basis of gradients. In a nutshell GOSS retains instances with large gradients while performing random sampling on instances with small gradients.
What is LightGBM algorithm?
Light GBM is a fast, distributed, high-performance gradient boosting framework based on decision tree algorithm, used for ranking, classification and many other machine learning tasks.
Why does LightGBM use exclusive feature bundling?
1)It is possible that LightGBM Framework can find out that we give the features as one-hot-encoded from the sparsity, it is possible that the algorithm does not treat one-hot-encoded with EFB. 2)It is also possible that LightGBM uses EFB on one-hot-encoded samples but it may be harmful, or not good as EFB on direct categorical inputs.
How does the exclusive feature bundling algorithm work?
I’m currently studying GBDT and started reading LightGBM’s research paper. In section 4. they explain the Exclusive Feature Bundling algorithm, which aims at reducing the number of features by regrouping mutually exclusive features into bundles, treating them as a single feature.
Why does LightGBM use EfB on one hot encoded samples?
2)It is also possible that LightGBM uses EFB on one-hot-encoded samples but it may be harmful, or not good as EFB on direct categorical inputs. (I go for this one) But still, I do not think that EFB will reverse one-hot-encoding since EFB is explained as a unique way of treating the categorical features.
Why do we use exclusive feature bundling in decision trees?
In section 4. they explain the Exclusive Feature Bundling algorithm, which aims at reducing the number of features by regrouping mutually exclusive features into bundles, treating them as a single feature. The researchers emphasize the fact that one must be able to retrieve the original values of the features from the bundle.