Testing Principles 1 - 5
1. “Absence of proof is not proof of absence”. This is true here. Testing is about evidence. We say what we saw and can extrapolate to provide an indication of confidence in the software quality, or otherwise, but we cannot say that all bugs have been found.
2. In testing we do not try to test everything – it would take too long and some scenarios may not be feasible in the real world. Instead, we consider the priorities (perhaps the most viable product), the risks that need to be addressed and the techniques that we can use to expose these defects.
3. As soon as we have something to test, we should test it. This includes testing the requirements, the source code, the executing code etc as soon as they are available.This is because, as you might already know, the earlier we catch the issue, the cheaper it is (usually) to fix it then, thereby reducing the total cost of quality.
4. There is something known as the Pareto Principle – that 80% of effects arise from 20% of causes. This principle is based on this idea. In testing, you may have noticed that defects seem to cluster in parts of the system, hence defect clustering. If we can predict these, perhaps based on previous experience or knowledge of something new or complex being added, then we can prepare for it by targeting our testing to these areas.
5. When testing is started, hopefully bugs will be found and dealt with. Rerunning the same tests many times will eventually yield fewer and fewer defects, perhaps leading to a false sense of security – our tests may no longer capture the current requirements; not use appropriate data nor even reflect the way things are done now. Thus, we should make sure that our workflows and data sets are up-to-date. In regression testing however, repeatability of the testing baseline is beneficial, especially when automating the regression testing.