What Is Pruning in a Decision Tree Algorithm?

Pruning is the process of eliminating weight connections from a network to speed up inference and reduce model storage size. Decision trees and neural networks, in general, are overparameterized. Pruning a network entails deleting unneeded parameters from an overly parameterized network.

Pruning mostly serves as an architectural search inside the tree or network. In fact, because pruning functions as a regularizer , a model will often generalize slightly better at low levels of sparsity. The trimmed model will match the baseline at higher levels. If you push it too far, the model will start to generalize worse than the baseline, but with greater performance.