This is a question that many software companies will ask themselves when it comes to testing their code. To answer this question we need to discuss what automation is, and then discuss possible solutions.
What do you think when you think about automation? Most people think that it is record and playback. Fairly, this is where some software automation tools started, but this type of testing is very limited. If the workflow of an application is dynamic, the playback scripts running against them are very fragile and prone to error. A lot of effort goes into determining if the application was at fault or if it was the scripts.
If you are a developer, you might think that automation is very similar to unit testing. That is also fair, since both types of testing can use similar tools. However, the goal for unit testing is usually to test the smallest testable part of code, and each test is designed to run in isolation.
If you are in Quality Assurance and do not have an in-depth technical background, you could be staring down a deep rabbit hole. Automation tools have come a long way in order to provide the modularity and flexibility needed to test a dynamic application. Automation for functional testing of software needs to be modular; however, scripts need to work together to test the workflow of the application. It also needs to be flexible enough that if the workflow changes, major overhauls are not needed. As well, if multiple people will be creating scripts then they need to be stored on a server or in a repository and not on a local machine.
Buy vs. build
The draw of licensed products is that some of the technical aspect is already done. A licensed product provides a framework that includes logging, reporting, and a structure for the scripts. The drawbacks of a licensed product are the potential costs, the use of proprietary scripting languages. Moreover, limited configuration options can restrict flexibility to some degree.
The draw of open source products is that the up-front cost is minimal, if any, and flexibility is optimized. However, the technical aspect of implementing open source tools can be that rabbit hole I mentioned earlier. When attempting to use open source automation tools, you will soon find that you need several components to make tests run.
First, you need the driver that drives a web browser or application. A good book to read to get familiar with one of the more popular open source automation engines, Selenium, can be found here Selenium 2 Testing Tools Beginner’s Guide.
Next you will need some type of test runner that finds test case classes within your scripts in order to run them. One of the popular test runner applications is TestNG. Tips on how to use TestNG with Selenium can be found here Time for action – using Selenium with TestNG from TestNG Beginner’s Guide.
From here, you will want to have a utility that can parse logs and output meaningful reporting. You will probably want to set this up with some scheduling to run automatically at certain times. You will also need to decide on which programming language to use for the framework and for the scripts. Putting all of these components into place is part of building the framework.
With all of this in mind, I will say that automation takes dedication and commitment by the company. It is not a part-time task that a QA engineer can perform when they are not manually testing on other projects. In most cases, it will be best to either purchase a licensed tool to get the built-in framework, or commission a developer to build one with requirements coming from the QA personnel that will be creating and maintaining the scripts. Functional Automation Testing is not a silver bullet that can be built and left to its own devices. However, careful planning and dedication to this type of tool will pay off. If done correctly it will provide a good sense of stability in your application.