Big Data Analytics with R

To the memory limits and beyond

We will start off by introducing you to three very useful and versatile packages which facilitate out-of-memory data processing: ff, ffbase, and ffbase2.

Data transformations and aggregations with the ff and ffbase packages

Although the ff package authored by Adler, Glaser, Nenadic, Ochlschlagel, and Zucchini, is several years old it still proves to be a popular solution to large data processing with R. The title of the package Memory-efficient storage of large data on disk and fast access functions roughly explains what it does. It chunks the dataset, and stores it on a hard drive, while the ff data structure (or ffdf data frame), which is held in RAM, like the other R data structures, provides mapping to the partitioned ...