Hadoop: Data Processing and Modelling

Time for action – the first run

Let's now perform the initial execution of this algorithm on our starting representation of the graph:

1. Put the previously created graph.txt file onto HDFS:

   $ hadoop fs -mkdirgraphin
   $ hadoop fs -put graph.txtgraphin/graph.txt
   

2. Compile the job and create the JAR file:

   $ javac GraphPath.java
   $ jar -cvf graph.jar *.class
   

3. Execute the MapReduce job:

   $ hadoop jar graph.jarGraphPathgraphingraphout1
   

4. Examine the output file:

   $ hadoop fs –cat /home/user/hadoop/graphout1/part-r00000
   12,3,40D
   21,41C
   31,5,61C
   41,21C
   53,6-1P
   63,5-1P
   76-1P
   

What just happened?

After putting the source file onto HDFS and creating the job JAR file, we executed the job in Hadoop. The output representation of the graph shows a few changes, as follows: ...