software testing genius

How to Know when should we Stop our Testing

Welcome to “Software Testing Genius”. Subscribe to my RSS feed for latest content on Software Testing.

How to Know when should we Stop our Testing

All testers come across this typical question as to “when to stop” testing. Fact is that testing can never be considered complete. We can never be able to prove scientifically that our software system is free from errors now.

Most Common Criteria Practiced in the software industry goes by the statement

1) Stop the testing when the committed / planned testing deadlines are about to expire.

style="TEXT-INDENT: -0.25in; MARGIN: 5pt 0.5in 5pt 0.25in; tab-stops: list .25in; mso-list: l0 level1 lfo2">2) Stop the testing when we are not able to detect any more errors even after execution of all the planned test Cases.

We can see that both the above statements do not carry any meaning and are contradictory since we can satisfy the first statement even by doing nothing while the second statement is equally meaningless since it can not ensure the quality of our test cases.


Pin pointing the time - when to stop testing is difficult. Many modern software applications are so complex and run in such an Interdependent environment, that complete testing can never be done.

Most common factors helpful in deciding when to stop the testing are:

1) Stop the Testing when deadlines like release deadlines or testing deadlines have reached

2) Stop the Testing when the test cases have been completed with some prescribed pass percentage.

3) Stop the Testing when the testing budget comes to its end.

4) Stop the Testing when the code coverage and functionality requirements come to a desired level.

5) Stop the Testing when bug rate drops below a prescribed level

6) Stop the Testing when the period of beta testing / alpha testing gets over.

Keeping a Track on the Progress of Testing

Testing metrics can help the testers to take better and accurate decisions; like when to stop testing or when the application is ready for release, how to track testing progress & how to measure the quality of a product at a certain point in the testing cycle.

The best way is to have a fixed number of test cases ready well before the beginning of test execution cycle. Subsequently measure the testing progress by recording the total number of test cases executed using the following metrics which are quite helpful in measuring the quality of the software product

1) Percentage Completion: (Number of executed test cases) / (Total number of test cases)

2) Percentage Test cases Passed: Defined as (Number of passed test cases) / (Number of executed test cases)

3) Percentage Test cases Failed: Defined as (Number of failed test cases) / (Number of executed test cases)

A test case is declared - Failed even when just one bug is found while executing it, otherwise it is considered as - Passed

Scientific Methods to decide when to stop testing:

1) Decision based upon Number of Pass / Fail test Cases:

a) Preparation of predefined number of test cases ready before test execution cycle.

b) Execution of all test cases In every testing cycle.

c) Stopping the testing process when all the test cases get Passed

d) Alternatively testing can be stopped when percentage of failure in the last testing cycle is observed to be extremely low.


2) Decision based upon Metrics:

a) Mean Time Between Failure (MTBF): by recording the average operational time before the system failure.

b) Coverage metrics: by recording the percentage of instructions executed during tests.

c) Defect density: by recording the defects related to size of software like “defects per 1000 lines of code” or the number of open bugs and their severity levels.


Finally How to Decide:
Stop the testing, If:


1) Coverage of the code is good
2) Mean time between failure is quite large
3) Defect density is very low
4) Number of high severity Open Bugs is very low.

Here 'Good', 'Large', 'Low' and 'High' are subjective terms and depend on the type of product being tested. Ultimately, the risk associated with moving the application into production, as well as the risk of not moving forward, must be taken into consideration.

Broad / Universal statement to define the time to stop testing is when:

All the test cases, derived from equivalent partitioning, cause-effect analysis & boundary-value analysis are executed without detecting errors.

Many More Articles on Best Practices in Manual Testing 

Largest Database of Sample Papers - 1000+ Unique Questions for ISTQB Foundation Exam

ISTQB Foundation Exam - Full Crash Course for Download

ISTQB Advanced CTAL Test Analysts Exam - Full Crash Course for Download

ISTQB Advanced CTAL Test Manager Exam - Full Crash Course for Download

Consolidated Study Material - All ISTQB Certification Exams

What Successful Testers say about the Quality of this website

If you want to keep track of further articles on Software Testing,
I suggest you to subscribe my
RSS feed

You can also Subscribe by E-mail
and get All New articles delivered directly to your Inbox.

Quick Navigation of Software Testing Genius

Get your Absolutely Free Copy of Several MS PowerPoint Presentations & E-Books related to ISTQB, HP Load Runner, IBM RFT, HP QTP & QC Certification Exams, prepared by Popular Writers & Trainers, by writing to:

Study Material for Certification Exams on Other Automation Tools:

Download Full Study Material - HP QTP & QC Certification Exams

Practical Roadmap to QTP Certification

Rehearsal of QTP in 1 Hr. -  Interview Questions

Study Material - HP LoadRunner Certification Exams for All Modules

Rehearsal of LoadRunner in 1 Hr. -  Interview Questions

Study Material - IBM RFT Certification Exam

Study Material to prepare for Manual Testing & QA:

Practical Roadmap to CSTE Certification

Consolidated Study Material - Testing & QA


Comments :


Leave Your Comments: (*) Marked Fields are Mandatory

You can apply basic formatting to the text

Name *
Email Address *
Speak your mind
sex hikayeleri