File formats
Hive supports a number of file formats out of the box. In this section, we will inspect some of these file formats and their utilities.
Compressed files
For some use cases, storing files in a compressed format within HDFS is advantageous. This strategy not only uses less storage, it also can reduce query times. Hive provides importing files stored in GZIP and BZIP2 formats directly into tables. During query execution, these files are decompressed and given as inputs to Map tasks. However, files compressed with GZIP and BZIP2 compression schemes cannot be split and are processed within a single Map task.
In practice, files stored in these compressed file formats are loaded into a table whose underlying data format is a Sequence file. ...
Get Hadoop: Data Processing and Modelling now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
