A Hadoop Distributed File System (HDFS) is a Java-based, distributed, scalable, and portable file system for the Hadoop framework. With PDI, pulling data back out from the Hadoop File System is really easy. In fact, we can treat it just like any other flat file source. Here are the steps for reading a file from Hadoop:
- Open Spoon and create a new Transformation.
- From the Big Data category, drag to the work area, the Hadoop File Input step.
- Edit the Hadoop File Input step. This will be used to pull the input file information.
- Under Environment, select <Static>.
- For the File/Folder field, enter the connection information to get to the file. For instance, if the data is stored in a file named samplefile.csv under the ...
