Editing Random forest (section)

===Preliminaries: decision tree learning===
{{main|Decision tree learning}}
Decision trees are a popular method for various machine learning tasks. Tree learning is almost "an off-the-shelf procedure for data mining", say [[Trevor Hastie|Hastie]] ''et al.'', "because it is invariant under scaling and various other transformations of feature values, is robust to inclusion of irrelevant features, and produces inspectable models. However, they are seldom accurate".<ref name="elemstatlearn">{{ElemStatLearn}}</ref>{{rp|352}}

In particular, trees that are grown very deep tend to learn highly irregular patterns: they [[overfitting|overfit]] their training sets, i.e. have [[Bias–variance tradeoff|low bias, but very high variance]]. Random forests are a way of averaging multiple deep decision trees, trained on different parts of the same training set, with the goal of reducing the variance.<ref name="elemstatlearn"/>{{rp|587–588}} This comes at the expense of a small increase in the bias and some loss of interpretability, but generally greatly boosts the performance in the final model.