This question can have a significant impact on the cost and schedule of any project. Experts view that, "While there is no magic way to select a sufficient set of practical tests in software testing effort, the objective is to test reasonably completely all valid classes for normal operation and to exhaustively test unusual behavior and illegal conditions".
The question of how many tests are needed must be answered early on so adequate resources (people and equipment) can be arranged and accurate and realistic schedules developed.
The test estimate measure reflects the number of tests needed based on factors such as the following:
a) Features and functions defined in the SRS and related documents;
b) Act-Like-A-Customer (ALAC) testing;
c) Achieving a test coverage goal;
d) Achieving a software reliability goal.
Test estimates should be based on and tied to specific sections of the SRS and other related documents. Starting with the SRS, review each requirement and, based on past experience, estimate the number of tests needed to determine whether the software has met the requirement. In addition to tests that are tied directly to the SRS, we should also develop a reasonable number of ALAC tests that are representative of customer use of the product.
Act-like-a-customer (ALAC) software testing is a method in which tests are developed based on knowledge of how customers use the software product. ALAC tests are based on the principle that complex software products have many bugs. ALAC tests allow us to focus on finding those bugs that customers are most likely to find. Acting like a customer also means developing tests that:
a) Do it wrong;
b) Use wrong or illegal combinations of input;
c) Do not do enough;
d) Do nothing;
e) Do too much.
ALAC software testing method is generally used for validation testing, regression testing & functional acceptance testing to verify that the software meets the SRS.
The test estimate should also reflect the complexity of tests as well as manual versus automated tests.
Developing tests should be viewed as an investment. The time and effort required to identify, write, and debug a test can be more than recouped based on the costs required to find and fix bugs once a product has been released. Building up a large suite of good regression tests is like having money in the bank.
Like most estimating tasks, the first time we make a test estimate we may find that our estimate and the actual number of tests developed are very different. At the end of a project, we must do a postmortem and understand why there was a discrepancy. We must try to learn from past experience, and our estimates will continually get better.
The test estimate metric is measured in units that are the number of tests to be written. To improve our ability to accurately estimate the magnitude of the software validation-testing task, we use this measure to compare the estimated number of tests to the actual number written.