Beautiful Testing

Chapter 23. Testing Network Services in Multimachine Scenarios

Isaac Clerencia

The Need for an Advanced Testing Tool in eBox

We had been developing eBox for a bit over a year when we started to struggle to get new releases out. The main problem we faced every time we had to release a new version was the testing and quality assurance (QA) process, which had quickly become the lengthiest and most dreaded task among eBox developers, although it did face tight competition from documentation writing in that honor.

The eBox platform is a complex open source web tool to manage corporate networks. It integrates some homegrown services, such as network configuration, a firewall, or traffic shaping, and well-known existing services such as Squid, Samba, or OpenLDAP. It is released as a set of independent modules that have to be thoroughly tested to prevent regressions and to verify that every new feature works as expected.

Even though eBox just offers a simplified interface to these services, the number of test cases still grew overwhelmingly fast as new modules were added and existing ones got more features that required new tests.

Another problem we faced was the complexity of the scenarios required to perform the tests. In the beginning, most modules needed quite simple scenarios, such as the proxy one, which only required two machines, one client to try to browse from, and the proxy server managed by eBox that we wanted to test.

Later in the development process, new modules requiring far ...