Tuning memory size for MemStores

As we described in recipe Avoiding update blocking on write-heavy clusters, HBase write operations are applied in the hosting region's MemStore at first, and then flushed to HDFS to save memory space when MemStore size reaches a threshold. MemStore flush runs on background threads using a snapshot of the MemStore. Thus HBase keeps handling writes even when the MemStores are being flushed. This makes HBase writes very fast. If the write spike is so high that the MemStore flush cannot catch up, the speed writes fill MemStores and memory used by MemStores will keep growing. If the size of all MemStores in a region server reaches a configurable threshold, updates are blocked and flushes are forced.

We will describe ...

Get HBase Administration Cookbook now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.