Time for action – causing task failure
Let's cause a task to fail; before we do, we will need to modify the default timeouts:
- Add this configuration property to
mapred-site.xml
:<property> <name>mapred.task.timeout</name> <value>30000</value> </property>
- We will now modify our old friend WordCount from Chapter 3, Understanding MapReduce. Copy
WordCount3.java
to a new file calledWordCountTimeout.java
and add the following imports:import java.util.concurrent.TimeUnit ; import org.apache.hadoop.fs.FileSystem ; import org.apache.hadoop.fs.FSDataOutputStream ;
- Replace the
map
method with the following one:public void map(Object key, Text value, Context context ) throws IOException, InterruptedException { String lockfile = "/user/hadoop/hdfs.lock" ; Configuration ...
Get Hadoop: Data Processing and Modelling now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.