Apache Spark 2.x Machine Learning Cookbook

Creating RDDs with Spark 2.0 using internal data sources

There are four ways to create RDDs in Spark. They range from the parallelize() method for simple testing and debugging within the client driver code to streaming RDDs for near-realtime responses. In this recipe, we provide you with several examples to demonstrate RDD creation using internal sources. The streaming case will be covered in the streaming Spark example in Chapter 13, Streaming Machine Learning System, so we can address it in a meaningful way.