Editing Gene expression programming (section)

=====Fitness functions for regression=====
In [[Regression analysis|regression]], the response or dependent variable is numeric (usually continuous) and therefore the output of a regression model is also continuous. So it's quite straightforward to evaluate the fitness of the evolving models by comparing the output of the model to the value of the response in the training data.

There are several basic [[fitness function]]s for evaluating model performance, with the most common being based on the error or residual between the model output and the actual value. Such functions include the [[mean squared error]], [[root mean squared error]], [[mean absolute error]], relative squared error, root relative squared error, relative absolute error, and others.

All these standard measures offer a fine granularity or smoothness to the solution space and therefore work very well for most applications. But some problems might require a coarser evolution, such as determining if a prediction is within a certain interval, for instance less than 10% of the actual value. However, even if one is only interested in counting the hits (that is, a prediction that is within the chosen interval), making populations of models evolve based on just the number of hits each program scores is usually not very efficient due to the coarse granularity of the [[fitness landscape]]. Thus the solution usually involves combining these coarse measures with some kind of smooth function such as the standard error measures listed above.

Fitness functions based on the [[Pearson product-moment correlation coefficient|correlation coefficient]] and [[R-square]] are also very smooth. For regression problems, these functions work best by combining them with other measures because, by themselves, they only tend to measure [[Correlation and dependence|correlation]], not caring for the range of values of the model output. So by combining them with functions that work at approximating the range of the target values, they form very efficient fitness functions for finding models with good correlation and good fit between predicted and actual values.