Loading data from a local machine to HDFS
In this recipe, we are going to load data from a local machine's disk to HDFS.
Getting ready
To perform this recipe, you should have an already Hadoop running cluster.
How to do it...
Performing this recipe is as simple as copying data from one folder to another. There are a couple of ways to copy data from the local machine to HDFS.
- Using the
copyFromLocalcommand- To copy the file on HDFS, let's first create a directory on HDFS and then copy the file. Here are the commands to do this:
hadoop fs -mkdir /mydir1 hadoop fs -copyFromLocal /usr/local/hadoop/LICENSE.txt /mydir1
- To copy the file on HDFS, let's first create a directory on HDFS and then copy the file. Here are the commands to do this:
- Using the
putcommand- We will first create the directory, and then put the local file in HDFS:
hadoop fs -mkdir /mydir2 hadoop fs -put /usr/local/hadoop/LICENSE.txt ...
- We will first create the directory, and then put the local file in HDFS:
Get Hadoop Real-World Solutions Cookbook - Second Edition now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
