Software testing, especially testing which closely follows ISEB and ISTQB best practices, is absolutely necessary in this day and age. However, despite its necessity, software testing does have some limitations that are important to be cognizant of. Specifically, successful testing does not guarantee a bug free software product upon release. Read on for a few examples of why this is the case.
First and foremost, it is simply not feasible to identify every possible scenario or testing attribute because at some point this effort will be subject to diminishing returns. This is generally known as the fallacy of exhaustive testing. It’s also akin to the 80/20 rule. The objective of testing is typically to identify and test as many important features and functionalities as possible, not necessarily all of them.
Another reason is that the participants of the testing effort, especially in terms of usability testing, are typically not representative of the population of end users as a whole. Everyone is different, and as such it is not possible to test a software application in a way that covers all possible user behaviors. Thus, much like with the fallacy of exhaustive testing, the goal is to identify the majority or most important classifications of users, not necessarily all of them.
Additionally, the environment within which the testing occurs is usually fabricated. Most times, testing occurs in a lab or testing environment that has been created for this specific purpose. Thus, it may or may not be completely representative of the actual environments within which end-users will use the software application.
The bottom line is that, although software testing seeks to ensure the highest probability of success, there is no way to guarantee with 100% certainty that the software will be glitch free in the ‘real world.’ That said, software testing – when conducted accurately, during the correct phase of the software development lifecycle, and as part of a user-oriented methodology – can dramatically reduce the risk of deploying faulty software into the market. Simply put, in nearly all cases, it is better to test than not to test.
Filed Under: Software Testing