You are here: » Software Testing » The Fallacy of Exhaustive Testing (and What to Do About It)

The Fallacy of Exhaustive Testing (and What to Do About It)

If you’re a perfectionist, then software testing is not the ideal career for you. Perfectionists want and need to do it all, every time. But in the context of software testing, this is simply not possible, and in fact this exact principle is a core attribute of the ISEB Foundation / ISTQB Foundation (or CTFL) certification.

Except for the simplest of applications, it is impossible to test every variable, every scenario, and every systemic interaction the universe has ever known. Software applications in this day and age are simply too complex to test every single feature and permutation. To do so would cost so much time and money, that it would prevent companies from ever launching anything. In software testing, the old 80/20 rule holds true; that is, 80% of the results can be obtained from 20% of the effort, and anything outside of this is simply not cost-effective.

So how should companies and software testers deal with this cruel reality? For one thing, companies should not try and be all things to all people. It is not always possible or even desirable to offer variations of the product for every possible market. Pick the niches that are the best opportunities and focus on those and those alone.

Another option is to make products less complicated. Again, remember the 80/20 rule. In this case, 20% of all possible features would probably satisfy 80% of the market, so just focus on attributes with broader appeal.

Finally you could accept the fact that the product will have some bugs when launched. Obviously this is not ideal, but when faced with the tradeoff of endless testing versus delivering the product to paying customers, sometimes this can be the wisest and most profitable course of action.

In the final analysis, have enough discipline to draw that line in the sand. Avoid the temptation to try and identify every possible bug. To do otherwise would hurt not only company profitability, but your sanity as well!


Related posts:

  1. Overview of ISEB and ISTQB Company Licenses
  2. Top 6 Software Testing Blunders of 2010

Filed Under: Software Testing

Leave a Reply