Use the Visitor pattern to separate data traversal from data handling.
Early in my career as a programmer, I did a lot of scientific programming with data acquisition systems. These were systems that recorded data at a sampling interval of 3 microseconds—in other words, 333,333 samples per second. That came out to 38 megabytes for every minute of information! For long recording sessions, a file could easily get into gigabytes. Needless to say, we had problems recording and storing that much information without any hiccups.
Another problem had to do with analyzing this data. How do you analyze a multigigabyte file when the machine doing the work has only 128 MB of memory? The answer is to chunk the data. Chunking means reading in the file by sections, swapping out the data you don't need and swapping in what you do need.
That said, you'd think that these scientific algorithms were tough enough without worrying about swapping in and out big chunks of data. To solve these problems—and do it elegantly—we used the Visitor pattern. One object would handle getting the data in and out of memory, and another object would handle processing the data when it was in memory.
Figure 7-12 shows a
RecordList object that contains a list of
Records. It has an
iterate() function that, when given another
function, calls the passed-in function on each record.
With this approach, the data processing function—passed in to
iterate()—doesn't have to understand ...