Secure Coding: Principles and Practices

6.1. Why Test?

If you've read this far into the book, perhaps you can anticipate our answer. For us, the purpose of testing is to determine whether software is secure enough. It's not to ensure that the application is unqualifiedly secure. And it's not to find all the vulnerabilities. The great Dr. Dijkstra said it best:

Program testing can be quite effective for showing the presence of bugs, but is hopelessly inadequate for showing their absence.

The message here is very important. Although testing is a necessary process—and indeed should be done throughout a development project—it must be handled carefully. Testing is not a substitute for sound design and implementation. It is not a cure-all to be applied at the end of a development project. And there's a trap: we have observed engineers who test a program against a publicly distributed attack tool and then declare that their code is "secure" when the attack tool fails. In fact, all that they've proven is that their program can resist one specific attack tool.

Testing is a kind of analysis, discovering what is and comparing it to what should be. The odd thing about security testing is that the yardstick we would like to hold up to our software is, in one sense, always changing in size.

Consider again the thought experiment we posed in Chapter 5: imagine that you carefully develop a complex e-commerce application and, as a last step, test it against all the savage attacks you can devise (or find out about). Let's assume that ...