Objectivity of Entry & Exit Criteria at Varying Levels of Software Testing
In software testing terms, entry and exit criteria represent the objective criteria that must be met before a test level is entered or exited. Entry and exit criteria can be applied to every test level. The exit criteria from the final test level (typically user acceptance testing) may be known as acceptance criteria or completion criteria, but the difference is in name only and the criteria serve the same purpose as any other exit criteria.
To ensure that the software projects are properly defined and to have a successful outcome, testing plans need to have concrete objectives.
Thus we need to answer three key questions coming to the mind of any test planner.
Q. 1: What do we mean by �Objective Criteria�?
Objective criteria are not subject to opinion or judgement; they are defined in a way that is either directly measurable or otherwise testable so that there is no doubt or uncertainty associated with whether the criterion is
achieved.For example, if all components must have zero known defects before integration testing can start then zero known component defects is an entry criterion for integration testing. The criterion is an objective one provided defects are formally reported and cleared so that we have an unambiguous way to determine whether the criterion has been met. It might be more realistic to expect zero critical defects rather than zero defects; this is still objective as long as �critical� is defined and is part of the reporting mechanism for defects.
Q. 2: What is the Importance of Objectivity?
Objectivity is important because objective criteria are not open to opinion or manipulation; they are either met or not met and, if they are based on some numerical measure, we can tell how far from meeting the criterion we are. Thus a system with 50 components, of which two have a single critical defect outstanding each, appears to be close to meeting the criteria. We can only tell how close when we have examined the outstanding defects and determined how much time and effort it will take to correct them. Contrast this with a system that has 50 components and 30 of them each have more than one critical defect.
In the first case identified the idea of accepting the risk and proceeding to integration seems reasonable and could be justified as long as the risk associated could be shown to be small and the outstanding defect corrections had been scheduled into a revised plan. In the second case, the risk could be addressed by reprioritizing appropriate test activities, after determining how long it will take to correct and retest all the outstanding defects.
If we set objective criteria for every level of testing and we do this at the beginning of the project, then we have a reliable means of determining how close we are to meeting our quality objectives, how much work will be entailed in reaching the objectives from where we are, and how much risk there would be in proceeding without meeting all of the criteria. In other words, we have a way to take the guesswork and emotion out of decision making. This still does not guarantee that we will always make the right decisions but it will make it easier to understand the implications of decisions.
Q. 3: What are the benefits of managing testing projects by objective criterion?
There are two important advantages to managing testing projects by making use of objective entry and exit criteria.
1) The criteria for successful completion are set at the beginning of the project, well before the pressures associated with delivery are felt, and can be based on a consensus of all the stakeholders.
2) Decisions are visible and simple at the transitions from one test level to another and at delivery.
These advantages do not, in themselves, provide any guarantee of a successful project. What they do achieve, however, is to make the decision-making process more open throughout the project. Every stakeholder can see how well the project is progressing against its quality criteria and if any decisions are required to authorize progress beyond a milestone for which the criteria have not been met the implications are clear to all.
At the point of delivery a final release decision will be based on criteria that are met, nearly met or not met. Where the criteria are met the release is straightforward. Where the criteria are nearly met a risk-based decision can be made based on the gap between the criteria and the actual achievement. Where criteria are not met the decision may be more tricky, but even here the existence of objective criteria can help to quantify the level of risk and select the best action plan.
Testing plans must have concrete objectives to ensure projects are properly defined and that successful outcomes can be differentiated from failed projects.
One of the most important mechanisms is the use of exit criteria to set specific goals for an activity. Exit criteria can be used to define success at the end of any activity, test level or project, so this concept has multiple uses within software testing projects.
Entry criteria can also be used where it is necessary to decide entry to an activity based on factors other than success of a previous activity.
Many More Articles on Test Planning & Management
An expert on R&D, Online Training and Publishing. He is M.Tech. (Honours) and is a part of the STG team since inception.