Contents
What is minimal cost complexity pruning?
Minimal cost complexity pruning recursively finds the node with the “weakest link”. The weakest link is characterized by an effective alpha, where the nodes with the smallest effective alpha are pruned first. As alpha increases, more of the tree is pruned, which increases the total impurity of its leaves.
What is the difference between overfitting noisy data and pruning?
In machine learning and data mining, pruning is a technique associated with decision trees. Overfitting happens when a model memorizes its training data so well that it is learning noise on top of the signal. Underfitting is the opposite: the model is too simple to find the patterns in the data.
Which is the minimal cost-complexity pruning algorithm?
Minimal Cost-Complexity Pruning is one of the types of Pruning of Decision Trees. This algorithm is parameterized by α (≥0) known as the complexity parameter. The complexity parameter is used to define the cost-complexity measure, R α (T) of a given tree T: Rα(T)=R (T)+α|T|
How to prune a decision tree with cost complexity?
In the following plot, the maximum effective alpha value is removed, because it is the trivial tree with only one node. Next, we train a decision tree using the effective alphas. The last value in ccp_alphas is the alpha value that prunes the whole tree, leaving the tree, clfs [-1], with one node.
How to calculate the complexity of a tree?
Let α ≥ 0 be a real number called the complexity parameter and define the cost-complexity measure R α ( T) as: The more leaf nodes that the tree contains the higher complexity of the tree because we have more flexibility in partitioning the space into smaller pieces, and therefore more possibilities for fitting the training data.
Which is the penalized version of the cost complexity measure?
In the end, the cost complexity measure comes as a penalized version of the resubstitution error rate. This is the function to be minimized when pruning the tree. Which subtree is selected eventually depends on α .