How to Define the Completion Criteria for our Software Testing Activities
The completion criteria are what we use to determine if we can stop the testing or if we have to go on to reach the objective of the testing.
The completion criteria are derived from the strategy and should be based on a risk analysis; the higher the risk, the stricter the completion criteria; the lower the risk the less demanding and specific the completion criteria. It is quite significant step to decide up front which completion criteria should be fulfilled before the test may be stopped.
The completion criteria guide the specification of the test and the selection of different techniques for test case design. These techniques are exploited to provide the test cases that satisfy the completion criteria.
The most appropriate completion criteria differ among various test levels.
Completion criteria for the test may be specified as under:
a) When the specified coverage has been successfully attained
b) When the specified number of failures detected against every testing effort has been successfully attained
c) When we are not able to detect any known serious defects
d) When we are able to identify that system benefits exceed the number of problems.
e) When the project time has run out
The last one is not an official completion criterion and should never be used as such; it is nonetheless often encountered in real life! Coverage is a very often used measurement and completion criteria in testing. Test coverage is the degree, expressed as a percentage, to which the coverage items have been exercised by a test.
The above mentioned completion criteria may be combined and the completion criteria for a test be defined as a combination of more individual completion criteria.
Few examples of combinations of completion criteria for different test levels can be defined as under:
A) Component testing
# 100% statement coverage
# 95% decision coverage
# No known faults
B) Integration testing (both for components and systems)
# 90% parameter coverage
# 60% interface coverage
# No known faults
C) System testing
# 90% requirement coverage
# 100% equivalence class coverage for specific requirements
# No known failures of criticality 1 or 2
# Stable number of failures per test hour for more than 20 test hours
D) Acceptance testing
# 100% business procedure coverage
# No known failures of criticality 1