Inside the Black Box: The Simple Truth About Quantitative Trading

Chapter 12. Evaluating Quants and Quant Strategies

The avocation of assessing the failures of better men can be turned into a comfortable livelihood, providing you can back it up with a Ph.D.

—Nelson Algren

In this chapter we discuss methods of assessing quant strategies and practitioners to separate the good from the mediocre and the mediocre from the poor. As I have said throughout this book, a great deal of the work that quants do has very natural analogues in discretionary trading. There are also significant parallels in the work of a quant trader to the work of a corporate CEO or any other person involved in the allocation of resources. In this regard, the framework presented in this chapter can be used successfully to judge the work of such decision makers. Indeed, one person I trained in this method of assessing quants has adapted it for trading credit markets and now uses the same method to provide a framework for judging the merit of various corporate bond offerings and the companies behind them.

The first challenge an evaluator of quants faces is to pierce the walls of secrecy that quants build around their methods. Though it is fair to say that quants are often secretive, I have had a rather different experience. The vast majority of quants I have evaluated—and there have been many hundreds of them—have been willing to answer most or all of my innumerable questions. The difference is due, at least in part, to the questions we ask at my firm. It also owes to how we ...