Editing Feature selection (section)

==Regularized trees==
The features from a [[decision tree]] or a tree [[Ensemble learning|ensemble]] are shown to be redundant. A recent method called regularized tree<ref name="DengRunger2012">H. Deng, G. Runger, "[https://arxiv.org/abs/1201.1587 Feature Selection via Regularized Trees]", Proceedings of the 2012 International Joint Conference on Neural Networks (IJCNN), IEEE, 2012</ref> can be used for feature subset selection. Regularized trees penalize using a variable similar to the variables selected at previous tree nodes for splitting the current node. Regularized trees only need build one tree model (or one tree ensemble model) and thus are computationally efficient.

Regularized trees naturally handle numerical and categorical features, interactions and nonlinearities. They are invariant to attribute scales (units) and insensitive to [[outlier]]s, and thus, require little [[data preprocessing]] such as [[Normalization (statistics)|normalization]]. Regularized random forest (RRF)<ref name="RRF">[https://cran.r-project.org/web/packages/RRF/index.html RRF: Regularized Random Forest], [[R (programming language)|R]] package on [[CRAN (R programming language)|CRAN]]</ref> is one type of regularized trees. The guided RRF is an enhanced RRF which is guided by the importance scores from an ordinary random forest.