Inside the Black Box: The Simple Truth About Quantitative Trading

8.3. SOURCES OF DATA

One can get data from many sources. Most direct, but also perhaps most challenging, is to get raw data from the primary sources. In other words, a quant would get price data for stocks traded on the New York Stock Exchange directly from the NYSE. This has the benefit of allowing the quant maximum control over the cleaning and storing of data, and it can also have significant benefits in terms of speed. However, there is also a massive cost to doing things this way. It would require building connectivity to every primary source, and if we are speaking about trading multiple types of instruments (e.g., stocks and futures) across multiple geographical markets and exchanges, the number of data sources can explode. With each, software must be built to translate the primary sources' unique formats into something usable by the quant's trading systems.

Examples of the kinds of primary sources and data types include these:

• Exchanges. Prices, volumes, timestamps, open interest, short interest, order book data.

• Regulators. Financial statements from individual companies, filings related to large owners of individual stocks as well as insider buying and selling activities.

• Governments. Macroeconomic data, such as employment, inflation, or GDP data.

• Corporations. Announcements of financial results and other relevant developments (e.g., changes in dividends).

• News agencies. Press releases or news articles.

• Proprietary data vendors (or data generators). House-created data that ...