Nerds on Wall Street: Math, Machines, and Wired Markets

8.8. Fitness Functions for Forecasting Models

There are many aspects to what makes one forecasting model better than another. Ultimately, we don't really know what a model's true value will be, since only the future will tell. We use a variety of holdback samples, statistical and financial measures, and heuristics to identify what we consider promising models. The ability of the GA to incorporate these diverse elements (in contrast to simpler statistical tools) is part of its appeal in this context.

• Risk and return. The obvious candidate for fitness in this context is financial return. This is used in most simple GA trading strategies, and certainly makes sense. In more complex strategies, there are other aspects to consider. Measures of risk-adjusted returns, such as the Sharpe ratio, are more appropriate.

• Statistical fitness. In many contexts, the performance of an investment strategy will depend on multiple models working together; there may not even be a sensible measure of the return to a single model in a complex strategy using combined forecasts and portfolio optimization. In these situations, a statistical measure of the fitness of the model is appropriate. A common choice is the information coefficient, simply the correlation of the predicted and actual outcomes. This is sensible, but hides some subtle errors. Correlations and other measures based on squared errors don't distinguish between a forecast that is in the right direction but wrong by the same amount as another ...