Beautiful Testing

The First Steps

The first step for success is simple: write great tests. Many test teams set themselves up for failure by holding the quality bar for their test code to much lower standards than their product code. Teams that write poor tests in an effort to save time almost always end up putting a large effort into maintaining their tests and investigating failed tests.

Test code must be easily maintainable. Whether the tests are going to be around for 10 days or 10 years, on a large team, chances are that someone other than the author will have to read, debug, or modify the code at some point. On many teams I’ve observed, once authoring of an automated test and integration of that test into the system are complete, that test remains in the system forever (unless, of course, the team removes the component under test from the product). Whether or not the “once a test, always a test” concept is appropriate (as well as mitigation techniques for the issue) is a topic for a different essay.

Code reviews and static analysis of the test code are also important. Hardcoded paths to test servers or hardcoded usernames and passwords are frequent causes of fragile tests, as are hardcoded resource strings that fail the first time the test runs on non-English builds. Tools or scripts that analyze source code at or extremely close to compile time are essential for detecting many common programming errors such as these.

Finally, it’s critical that test code is under source control and builds ...