Various approaches of Regression Testing

Various approaches of Regression Testing

Regression testing is done to ensure that enhancements or defect fixes made to the software works properly and does not affect the existing functionality. It is usually done during the maintenance phase. As a software system ages, the cost of maintaining the software dominates the overall cost of developing the software.

Regression testing is a testing process that is used to determine if a modified program still meets its specifications or if new errors have been introduced. Improvements in the regression testing process would help reduce the cost of software.

Points of differences between Regression Testing and Normal Testing:

Differences between regression testing and the normal testing process are as under.

Types of Regression Testing:

We have following four types of regression testing techniques.

1) Corrective Regression Testing: Corrective regression testing applies when specifications are unmodified and test cases can be reused.

2) Progressive Regression Testing: Progressive regression testing applies when specifications are modified and new test cases must be designed.

3) The Retest-All Strategy: The retest-all strategy reuses all tests, but this strategy may waste time and resources due to execution of unnecessary tests. When the change to a system is minor, this strategy would be rather wasteful.

4) The Selective Strategy: The selective strategy uses a subset of the existing test cases to reduce the retesting cost. In this strategy, a test unit must be rerun if and only if any of the program entities, e.g. functions, variables etc., it covers have been changed. The challenge is to identify the dependencies between a test case and the program entities it covers.

Rothermel and Harrold suggested a typical selective regression testing process, which includes following steps.

Step – 1: Identify affected software components after program, P, has been modified to P’.

Step – 2: Select a subset of test cases, T’, from an existing test suite, T that covers the software components that are affected by the modification.

Step – 3: Test modified program P’ with T’ to establish the correctness of P’ with respect to T’.

Step – 4: Examine test results to identify failures.

Step – 5: Identify and correct the fault(s) that caused a failure. Step 6. Update the test suite and test history for P’.

With the help of above mentioned steps we are able to establish following characteristics of selective regression testing

(1) Identifying the program components that must be retested and finding those existing tests that must be rerun are essential.

(2) When selected test cases satisfy retest criterion, new test cases are not needed.

(3) Once regression testing is done, it is necessary to update and store the test information for reuse at a later time.

Where do we apply Regression Testing?

Regression Testing is generally applied at all three levels of testing.

Some experts suggest applying specification based (system) testing before structure based (unit) testing to get more test case reuse because system testing cases could be reused in unit testing but the converse is not true. McCarthy suggested applying regression unit testing first to find faults early.

a) Regression Testing at Unit Level: Unit testing is the process of testing each software module to ensure that its performance meets its specifications.

b) Regression Testing at Integration Level: Integration testing is the testing applied when individual modules are combined to form larger and larger working units until the entire program is created. Integration testing detects failures that were not discovered during unit testing. Integration testing is important because approximately 40% of software errors can be traced to module integration problem discovered during integration testing.

“Leung” and “White” introduced the firewall concept to regression testing at the integration level. A firewall is used to separate the set of modules affected by program changes from the rest of the code. The modules enclosed in the firewall could be those that interact with the modified modules or those that are direct ancestors or direct descendants of the modified modules.

The firewall concept is simple and easy to use, especially when the change to a program is small. By retesting only the modules and interfaces inside the firewall, the cost of regression integration testing can be reduced.

c) Regression Testing at System Level: System testing is testing of the entire software system against the system specifications. It must verify that all system elements have been property integrated and perform allocated functions. It can be performed without the knowledge of the software implementation at all.

Test tube is a system developed at AT&T Bell laboratories to perform system level regression testing. Test tube is an example of a selective retesting technique. Test tube partitions a software system into basic code entities, then monitors the execution of a test case, analyzes its relationship with the system under test and determines which subset of the code entities, the test covers. There are plans to extend test tube to non-deterministic systems such as real-time telecommunications software.

Regression Testing of Global Variables
A global variable is an output parameter in a module where the variable is defined and an input parameter for a module that uses the variable. The regression testing of global variables is very time consuming hence very costly.

The global variables can be retested at the unit level by running the test that exercise the changed code and the instruction referencing the global variable.

A global variable can be retested at the integration level. If any of its defining modules have been changed then all of its using modules must be retested.

Comparison among different Selective Regression Testing Techniques

Following four criterions form a framework for evaluation of various selective regression testing techniques.

1) Inclusiveness: It measures the extent to which a technique chooses tests that will expose faults caused by program changes.

2) Precision: It measures the ability of a technique to avoid choosing tests that will not reveal the faults caused by the changes.

3) Efficiency: It measures the computational cost of a technique.

4) Generality: It measures the ability of a technique to handle different language constructs and testing applications.

Many More Articles on Basics of Testing