Coding our first Spark Streaming job
In this section, we will code and execute our first Spark Streaming job in Scala. We will also simulate the streaming data by creating a temporary stream.
Creating a stream producer
Perform the following steps to create a stream producer which continuously reads the input data provided by the user from the console and then further submits that data to a socket:
- Open and edit your
Spark-Examples
project and create a new Scala package and class namedchapter.nine.StreamProducer.java
. - Next, edit
StreamProducer.java
and add the following piece of code:import java.net.*; import java.io.*; public class StreamProducer { public static void main(String[] args) { if (args == null || args.length < 1) { System.out.println("Usage ...
Get Real-Time Big Data Analytics now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.