Efficient C++ Performance Programming Techniques

System Architecture

The cost of memory access is far from being uniform. On one particular RISC architecture, data access takes one CPU cycle if it resides in the data cache, 8 cycles if the data resides in main memory (cache miss), and 400,000 cycles if the data is on disk (page fault). Although the exact number of cycles may vary, the general picture is the same across processor architectures: The speed difference between a cache hit, cache miss, and page fault is measured in orders of magnitude.

The data cache is the first place that is searched when data is accessed. If the data is not in the cache, a cache miss is generated by the hardware. The cache miss results in this data being loaded from RAM or disk into the cache. The cache is loaded ...