Beautiful Testing

Testing in the Regulated World

In 2001, my boss Jim Kandler explained how to test software in a regulated world. His explanation to me (referring to the FDA) was: tell them what you going to do, do it, and then prove you did it. “Gee, that’s it?”, I can recall thinking. But it’s a lot harder to do than it sounds.

What Jim meant by “tell them what you going to do” is this: document the process. Document it for multiple people—the team who will execute the process, the company so that they know the team has a process, and of course, for the FDA. I was taught that an FDA auditor will review the process documentation as one of the first parts of an audit and then ask for evidence that you’ve followed the process. Some of the other documents they request first are defect reports, the trace matrix, and the final validation report.

What I’ve found is that detailing the process you believe the team follows is harder than it sounds when you get down to the nuances, not to mention the exceptions that occur in real life.

In the regulated waterfall approach to software development that I’ve seen, requirements are drafted. Requirements are from a business perspective, the patient perspective, the customer perspective, and from a systems point of view. Often people write them without thinking about testing or about how they are going to prove the requirement. Often people write them without a software tester’s involvement.

The requirements evolve into design specifications, ...