Let’s start looking at how the features we just described have contributed to some of Erlang’s successes. Ericsson’s first major Erlang product was the AXD301 ATM switch; more recently, Erlang has been the key to implementing the CouchDB schema-free, document-oriented database. Finally, we report on a Motorola-based research project comparing the productivity of Erlang and C++ head on.
The AXD301, a telephony-class 10–160 Gbps ATM switch, was designed and implemented from scratch in less than three years. At the heart of the AXD301 are more than 1.5 million lines of Erlang code, handling all the complex control logic, and overseeing operations and maintenance. This integrates with about half a million lines of C/C++ implementing low-level protocol and device drivers, much of it coming from third-party sources.
This ATM switch has been installed in networks all over the world, but the installation that shot to prominence was used by British Telecom to build what was at the time the largest “Voice over ATM” backbone in the world. According to an Ericsson press release issued at the end of the trial period, “Since cut-over of the first nodes in BT’s network in January 2002 only one minor fault has occurred, resulting in 99.9999999% availability.” The director of Ericsson’s Next Generation Systems program, Bernt Nilsson, confirmed that “the network performance has been so reliable that there is almost a risk that our field engineers do not learn maintenance skills.”
Experiences with the AXD301 suggest that “five nines” availability, downtime for software upgrades included, is a more realistic assessment. For nonstop operations, you need multiple computers, redundant power supplies, multiple network interfaces and reliable networks, cooling systems that never fail, and cables that system administrators cannot trip over, not to mention engineers who are well practiced in their maintenance skills. Considering that this target has been achieved at a fraction of the effort that would have been needed in a conventional programming language, it is still something to be very proud of.
How did Erlang contribute to the success of the AXD301? It supports incremental development, with the absence of side effects, making it easier to add or modify single components. Support for robustness and concurrency is built into the language and available from the start.
Erlang was very popular with the programming teams that found they were building much more compact code, thus dramatically improving their productivity. Experience from the project, although not scientifically documented, suggests that the Erlang code was 4 to 10 times shorter than similar systems written in C/C++, Java, and PLEX, while the fault rate per thousand lines of code was the same.
Ericsson has gone on to use Erlang on other projects across the company, including a SIP telephony stack, control software for wireless base stations, telephony gateway controllers, media gateways, broadband solutions, and in GPRS and 3G data transmission. And these are just a few of the many we are allowed to talk about.
When Damien Katz decided to implement CouchDB, he wanted to be the one developing “cool stuff.” He wanted to see whether he was good enough to develop something from scratch, pushing the code base to new levels. CouchDB is an open source database that provides a schema-less replicated document store, storing objects in JSON format and accessed through a RESTful interface.
He wrote the first version of CouchDB in C++. His system consisted of three components: a storage engine, a view engine, and a query language. The complexity of his components increased, and when he started hitting concurrency issues, he felt like he had hit a wall. He stumbled upon Erlang, downloaded it, and quickly came to the realization that it would solve his problems.
From the world Damien was coming from, Erlang initially sounded very complicated, and he believed it would be hard to learn. But when he got down to the details, what instead struck him was the simplicity of the language. Getting something to work with Erlang took extra effort compared to Java, as there were fewer tools and IDEs available, but to get something working reliably ended up taking much less talent and time than any of the other languages he knew.
Erlang gave Damien the features he needed for CouchDB at a fraction of the effort of using conventional languages. When migrating CouchDB to Erlang, he focused on the concurrency aspects and integrating it with his existing C++ components. He ended up replacing the entire C++ code base, as Erlang had all of the qualities he was looking for in a database application. They included support for intensive I/O, high reliability, and facilities for dealing with failure gracefully. The first benchmarks on the code, even before it was profiled, allowed in excess of 20,000 simultaneous connections. This compared pretty favorably with the 500 he expected to get on the C++ version!
Once it was released, CouchDB started getting lots of attention in the open source community. Damien made the decision to release the code under the Apache license, giving him the freedom he needed to continue development. Today, CouchDB is one of the best known Erlang open source applications currently being used in production systems worldwide.
What happened to Damien? He got a job with IBM, allowing him to continue developing CouchDB as an open source project. In Damien’s words, now he is indeed “the guy who gets paid to work on cool stuff.” You can read out more about CouchDB at http://www.couchdb.org.
Most experienced Erlang programmers will confirm that the Erlang programs they have written are substantially shorter than their counterparts in other mainstream programming languages used by the industry. Indeed, this was an urban legend among Ericsson programmers long before Erlang was released as open source. But until recently, there was very little scientific evidence to back up these claims. Quicksort (Chapter 9) using list comprehensions or remote procedure call server examples (Chapter 11), both of which we cover in this book, were used to argue the case. When comparing programming languages, however, you must benchmark whole systems in the application domain for which those languages were designed, not code snippets or simple functions.
Heriot-Watt University in the United Kingdom received an EPSRC grant to study the impact of distributed functional programming languages in the telecom sector. When we first heard about this grant, our reaction was, why not speak with Ericsson and get it over with? We quickly changed our minds when we realized the research project was being done in cooperation with Motorola Labs, one of Ericsson’s competitors. Although Heriot-Watt might have taken Ericsson’s word that Erlang was suitable for programming telecom applications, Motorola wasn’t having any of it.
The focus of the study consisted of two C++-based systems referred to as the Data Mobility (DM) component and the Dispatch Call Controller (DCC). These systems handled digital communication streams for pocket radio systems as used by emergency services. The DM been written with fault tolerance and reliability in mind. The implementation was done by good C++ programmers who based their development work on proprietary Motorola libraries. The DCC was an internal research prototype intended for evaluating the use of C++ and CORBA to gain scalability.
The Erlang rewrites were implemented by Jan Henry Nyström, an experienced Erlang programmer with an academic background. Two Erlang rewrites were done of the DM and only one of the DCC. The first DM implementation interfaced with Motorola’s libraries, and the second was a pure Erlang implementation. The DCC was a pure Erlang implementation. Comparisons were made of the performance, robustness, productivity, and impact of the programming language constructs.
The interesting conclusions of this research came with the pure Erlang implementations. In the DM, there was an 85% reduction in code. This was explained by the fact that 27% of the C++ code consisted of defensive programming, 11% of memory management, and 23% of high-level communication, all features which in Erlang are part of the semantics of the language or are implemented in the OTP libraries. The DCC’s code base was more in line with the folklore and urban legends, namely that it was about 70% smaller than its C++ counterpart.
The Erlang DM resulted in a 100% performance increase when compared to the C++ version, which crashed when severely overloaded. Although the throughput might sound surprising at first, it was a result of Erlang and its lightweight concurrency model being the right tool for the task. The mobility application in question had lots of concurrency, short messages, and little in terms of heavy processing and number crunching. The C++ implementation was never implemented to handle the loads it was subjected to, so as a result, the conclusion was that these load results might not be relevant and are certainly unfair to the C++ implementation. They do, however, demonstrate an important property of Erlang-based systems, which are stable under heavy loads and recover automatically when the load drops.
Although Erlang pioneers argued their case of shorter and more compact code based on experience, this study has finally provided empirical data to support the claims. A full report that fully confirms the “Erlang advantage” is available in “High-level distribution for the rapid production of robust telecoms software: comparing C++ and Erlang.”
 PLEX is a proprietary language developed by Ericsson and used extensively in the AXE-10 switches. Just like Erlang, many of its features were ahead of its time. It was never released to the public.
 The Engineering and Physical Sciences Research Council provides U.K. government support for science research in universities.
 Nyström, J.H., P.W. Trinder, and D.J. King. Concurrency and Computation: Practice & Experience, 20(8), 2008.