O'Reilly logo

Learning Hadoop 2 by Garry Turkington, Gabriele Modena

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Hadoop filesystems

Until now, we referred to HDFS as the Hadoop filesystem. In reality, Hadoop has a rather abstract notion of filesystem. HDFS is only one of several implementations of the org.apache.hadoop.fs.FileSystem Java abstract class. A list of available filesystems can be found at https://hadoop.apache.org/docs/r2.5.0/api/org/apache/hadoop/fs/FileSystem.html. The following table summarizes some of these filesystems, along with the corresponding URI scheme and Java implementation class.

Filesystem

URI scheme

Java implementation

Local

file

org.apache.hadoop.fs.LocalFileSystem

HDFS

hdfs

org.apache.hadoop.hdfs.DistributedFileSystem

S3 (native)

s3n

org.apache.hadoop.fs.s3native.NativeS3FileSystem

S3 (block-based)

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required