Editing Feature selection (section)

=== Conditional mutual information ===

Another score derived for the mutual information is based on the conditional relevancy:<ref name="CMI">Nguyen X. Vinh, Jeffrey Chan, Simone Romano and James Bailey, "Effective Global Approaches for Mutual Information based Feature Selection". Proceedings of the 20th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD'14), August 24–27, New York City, 2014. "[http://people.eng.unimelb.edu.au/baileyj/papers/frp0038-Vinh.pdf]"</ref>

:<math>
\mathrm{SPEC_{CMI}}: \max_{\mathbf{x}} \left\{\mathbf{x}^T Q \mathbf{x}\right\} \quad \mbox{s.t.}\ \|\mathbf{x}\|=1, x_i\geq 0
</math>

where <math>Q_{ii}=I(f_i;c)</math> and <math>Q_{ij}=(I(f_i;c|f_j)+I(f_j;c|f_i))/2, i\ne j</math>.

An advantage of {{math|SPEC<sub>CMI</sub>}} is that it can be solved simply via finding the dominant eigenvector of {{mvar|Q}}, thus is very scalable. {{math|SPEC<sub>CMI</sub>}} also handles second-order feature interaction.