Beautiful Testing

Can’t Handle the Load? Change the UI

Very shortly after the “memory leak that wasn’t” incident, it was decision time for the financial planning application. The application was functioning, but we simply did not believe that we could improve the performance enough to handle the Super Bowl commercial–inspired peak. I hadn’t been included in the discussion of options until at least two weeks after both the client and the development team had become very concerned. I’m not exactly sure why I was invited to the “what are we going to do” meeting with the client that day, but it turned out that whoever invited me, intentionally or accidentally, was probably glad they did.

The short version of the problem was that the application gave all indications of being completely capable of handling the return user load, but that, if the projections were close to being correct, there was no way the architecture could handle the peak new user load generated by the Super Bowl advertising campaign. What I didn’t know until I got to the meeting on that day was that we’d reached the point of no return in terms of hardware and infrastructure. The application was going to run in the existing environment. The question now was, what to do about the usage peak generated by the marketing campaign?

For about 30 minutes, I listened to one expensive and/or improbable idea after another get presented and rejected. The most likely option seemed to be to lease four identical sets of hardware and find a data center to host them, which was estimated to cost enough that every time it came up, Ted shook his head and mumbled something about not wanting to lose his job.

Finally, I spoke up. I pointed out that there were really only two things that we couldn’t handle large numbers of users doing all at once. One was “Generate your personalized retirement savings plan,” and the other was “Generate your personalized college savings plan.” I also pointed out that for someone to get to the point where those plans were relatively accurate, they’d have to enter a lot of information that they probably didn’t have at their fingertips. I then speculated that if we redesigned the UI so that those options weren’t available until users had at least clicked through to the end of the questionnaire (as opposed to making it available on every page of the questionnaire, virtually encouraging folks to click the button after each piece of information they entered so they could watch the charts and graphs change), that might reduce the number of plan generations enough to get through the marketing campaign. I further commented that we could put the plan generation links back on each page after the campaign was over.

The looks of shock and the duration of stunned silence lasted long enough that I actually became uncomfortable. Eventually, a woman I didn’t know started scribbling on a sheet of paper, then pulled out a calculator to do some calculations, then scribbled some more before very quietly saying, “It might just work.” Everyone turned their stunned stares to her when Ted asked her what she’d said. She repeated it, but added, “Well, not exactly what he said, but what if we….”

To be honest, I partly don’t remember what she said, and I partly never understood it, because she seemed to be speaking in some secret financial planning language to Ted. Regardless of the details, as soon as she was done explaining, Ted said, “Do it!” and everyone started smiling and praising me for solving the problem.

A few weeks later, I got a new build with a modified UI and usage model to test. It took a couple of iterations of minor UI modifications and tuning, but we achieved our target load. I found out much later that the marketing campaign was a success and that the system held up without a glitch. In fact, the campaign went so well that a big bank bought the company, Ted got a promotion, and of course, the big bank had their developers rebuild the entire application to run on their preferred vendor’s hardware and software.

The obvious beauty here is that the project was successful and that the solution we came up with was not prohibitively complicated or expensive. The less obvious beauty lies in the often-overlooked value of involving people with several different perspectives in problem solving work groups.