The automation of software testing tasks is an emerging yet elusive proposition. In fact, it is a concept that has become increasingly important in the context of ISEB and ISTQB certifications. All companies want to automate as many aspects of their business as possible, but most are simply not at the point where they know how to execute this idea.
The first factor to consider is the cost / benefit ratio. Automation tools can solve just about any problem you can imagine, but there is a clear correlation between the complexity of the problem and the cost of the solution. At some point, there is a point of financial diminishing returns. Additionally, you will want to consider what the tool can and cannot do, and factor that information into your cost / benefit analysis. For example, if the tool is not able to automatically debug its own scripts, then you can count on an increase in your time estimate of at least 30% in order to develop and debug said scripts.
Another key consideration is the availability of technical support and technical documentation for the tool in question. Obviously the expectation is that the provider of the tool should offer solid customer service, but beyond that, you’ll want to have some assurance that lots of other people or companies are using the tool. For example, the presence of forums or other online communities focused on users of the tool is a good sign, as this means the tool actually works and you will have access to people and information that will help to quickly answer any tool-related question you might have.
Finally, you will want to make sure that the tool provides the analytical reporting functionality necessary to measure the results. If the tool in question is weak on the analytics side of the equation, walk away because you cannot improve what you cannot measure.
In summary, when selecting an automated software testing tool, make sure that your organization considers the costs and benefits, including the covert costs of the tool not performing 100% of the required tasks; make sure the tool has good technical and customer support, along with lots of communities of users; and make sure the tool can thoroughly measure any and all results. Good luck and happy testing!
Filed Under: Software Testing