Editing Feature selection (section)

=== Quadratic programming feature selection ===
mRMR is a typical example of an incremental greedy strategy for feature selection: once a feature has been selected, it cannot be deselected at a later stage. While mRMR could be optimized using floating search to reduce some features, it might also be reformulated as a global [[quadratic programming]] optimization problem as follows:<ref name="QPFS">{{cite journal |first1=I. |last1=Rodriguez-Lujan |first2=R. |last2=Huerta |first3=C. |last3=Elkan |first4=C. |last4=Santa Cruz |title=Quadratic programming feature selection |journal=[[Journal of Machine Learning Research|JMLR]] |volume=11 |pages=1491–1516 |year=2010 |url=http://jmlr.csail.mit.edu/papers/volume11/rodriguez-lujan10a/rodriguez-lujan10a.pdf}}</ref>

:<math>
\mathrm{QPFS}: \min_\mathbf{x}  \left\{ \alpha \mathbf{x}^T H \mathbf{x} -  \mathbf{x}^T F\right\} \quad \mbox{s.t.} \ \sum_{i=1}^n x_i=1, x_i\geq 0
</math>

where <math>F_{n\times1}=[I(f_1;c),\ldots, I(f_n;c)]^T</math> is the vector of feature relevancy assuming there are {{mvar|n}} features in total, <math>H_{n\times n}=[I(f_i;f_j)]_{i,j=1\ldots n}</math> is the matrix of feature pairwise redundancy, and <math>\mathbf{x}_{n\times 1}</math> represents relative feature weights. QPFS is solved via quadratic programming. It is recently shown that QFPS is biased towards features with smaller entropy,<ref name="CMI" /> due to its placement of the feature self redundancy term <math>I(f_i;f_i)</math> on the diagonal of {{mvar|H}}.