Scaling CouchDB

Chapter 1. Defining Scaling Goals

Before you can scale CouchDB, you need to define your scaling goals. Once you have defined your goals then you can design your system. You should test the scalability of your system before it is deployed. See Chapter 6 for information about how to perform distributed load testing on your system using Tsung. When your system has been deployed to production, you should continue to monitor its performance and resource utilization using a tool such as Munin (a CouchDB plugin for Munin is available at https://github.com/strattg/munin-plugin-couchdb) or Nagios. You can monitor an individual node by issuing a GET HTTP request to /_stats which will return various statistics about the CouchDB node.

What is Scalability?

It’s important to note the distinctions between performance, vertical scaling, and horizontal scaling. Performance typically refers to properties of a system such as response time or throughput. Vertical scaling (or scaling up) means adding computing capacity to a single node in a system. This could be through added memory, a faster CPU, or larger hard drives. While memory gets cheaper, CPUs get faster, and hard drives get larger every year, at any given moment there’s an upward limit to vertical scaling. Also, the hardware for high capacity nodes is usually more expensive per unit of computing capacity (as defined by your application) than commodity hardware.

When someone uses the word “scalability,” they are often referring to horizontal scalability. Horizontal scaling (or scaling out) is when a system’s computing capacity is scaled by adding new nodes. In theory, each new node adds the entire amount of that node’s computing capacity to the system. In practice, true horizontal scalability is rarely, if ever, achieved. The network overhead of nodes communicating with each other can detract from the overall computing capacity of the system. Also, few systems are configured such that there is no redundant computing work between nodes.