ZeroMQ

Serialization Libraries

The msgpack.org site says this about the MessagePack serialization library:

It’s like JSON, but fast and small. MessagePack is an efficient binary serialization format. It lets you exchange data among multiple languages like JSON, but it’s faster and smaller. For example, small integers (like flags or error code) are encoded into a single byte, and typical short strings only require an extra byte in addition to the strings themselves.

I’m going to make the perhaps unpopular claim that “fast and small” are features that solve non-problems. The only real problem that serialization libraries solve is, as far as I can tell, the need to document the message contracts and actually serialize data to and from the wire.

Let’s start with the “fast and small” claim. It’s based on a two-part argument: first, that making your messages smaller and reducing CPU cost for encoding and decoding will make a significant difference to your application’s performance; and second, that this will be equally valid across the board, for all messages.

But most real applications tend to fall into one of two categories: either the speed of serialization and size of encoding are marginal compared to other costs, such as database access or application code performance, or network performance really is critical, and then all significant costs occur in a few specific message types.

Thus, aiming for “fast and small” across the board is a false optimization. You get neither ...