more than thirteen times. After around thirteen executions, automation works out to be cheaper than manual testing.
For example, suppose for any test case that the total effort in writing the test case comes in at fifteen minutes and manual execution needs fifteen minutes. So the total time required for executing this test case thirteen times is 6.5 hours. Now automation requires around 6 hours of work. Automated script for the test case runs in less than three minutes. So the time required for writing the test case, automating the test case, and running it thirteen times is around 6 hours and 54 minutes.
B) Based upon the type of tests against production Instance:
Sanity checks, performance checks, and some other tests that are run against production instance of any system are the best candidates for automation. Production environments do not change often, and on average some patches may be applied twice per year. This means that they are the most stable systems against which automation scripts may not have to be changed often. Most of the organizations run these sanity checks on a nightly basis to ensure that the production systems are running smoothly. If any problem occurs, then it is caught in the nightly run of the sanity check and the problem is fixed quickly before users start using the system in the office hours.
C) Based upon the number of regression tests: When a system is being built, many features are being added in future releases. Whenever any defects are found in previous releases, they are also fixed in the next release or a patch. Regression of existing features is tested in all of these next releases or patches. Whenever those features are touched upon in these new releases, they should be regression tested. So most of the features get tested by the software testing engineers again and again. So regression testing is also a good candidate for automation. In fact, the number of regression tests increases over time with new releases. But software testing resources are limited and do not increase over time. So you end up having fewer resources for executing these regression tests. That is why it makes sense to automate them so that resource requirements can be reduced significantly.
Strategies for Manual Testing:
If you believe that since you have plenty of automation experts and automation tools at your disposal, hence you can automate 100% of your testing; it is not a wise decision.A) Based upon the capability of breaking the system apart:
Automation is for repeated execution of test cases, which are in most cases for system sanity checks (in production instances) or for regression (in the case of new versions). In no way are they capable of breaking the system apart. Manual tests do exactly that. Domain experts use their experience to test workflows, integration touch points, and other functional aspects of the application to check if the system breaks.
B) Based upon the type of testing suitable exclusively for human beings:
Automating regression and sanity tests free these domain experts to concentrate on the kinds of software testing that only human beings can do and require a lot of intuitive thinking. Providing ample scheduled time and resources for manual software testing will ensure a better and more effective software test effort.
Five Broad Software Testing Strategies Devised by Experts
Apart from basic strategies on Automation & Manual Testing, expert software testing managers use following five broad testing strategies that are viewed as major focus towards the improvement effort.
Strategy-1: On Specification and Design Validation
Successful testing process depends on the existence of good requirements and specifications for what is intended. Assuring that the specifications and design meet the quality standard is a central strategy and it strengthens the reviews and walkthroughs by emphasizing testability (i.e., early definition of system test data and acceptance criteria) as a major emphasis.
Strategy-2: Maintaining a Test Case Library
Test case development must begin with tests defined as the requirements and specifications are validated. Other tests must be developed over the project life. Software testing managers focus on the development of a test case library that stores and controls all the test data effectively. This could include performance test data, functional test data, and the unit module structured tests. It must be updated as changes to the system take place and can be taken as an extension or subset of the data dictionary for the system.
Strategy-3: On Top-Down TestingThe choice of the order for integrating modules and checking out subsystems has a big impact on the system testing process. Software testing managers aim to have a methodology that tests and checks out a high level system first and then adds modules in an organized top-down manner.
Strategy-4: On Structured Module TestingTechniques for unit testing of modules need be developed to provide a basis for determining how much software testing is required and producing the test cases needed in a straightforward and structured manner. These software testing techniques must have a theoretical foundation and must have proven themselves both in terms of effectiveness and efficiency. These techniques must be introduced as a normal practice for unit module testing.
Strategy-5: On Test Planning and ReportingA comprehensive test plan that defines the software testing goals, strategy, approach, and organization must be a regular practice. The plan must elaborately spell out how the results would be captured and tracked, and must provide a means for feeding back quality information to individual team members and to the management.
Detailed stand-alone procedures for each of the above five key strategy areas must be developed & implemented.