Developing Spark with Eclipse
Eclipse is the IDE of choice for Java developers. The best way is to install the Scala version (2.11.8) of Eclipse from http://scala-ide.org/. This will enable one to develop the Scala and Java programs.
Let's first create a Scala project like so:
From Spark 2.0.0 onwards, they have changed the packaging, so we have to include spark-2.0.0/assembly/target/scala-2.11/jars in Add External Jars… as shown in the following screenshot:
Get Fast Data Processing with Spark 2 - Third Edition now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
