Strategies & Considerations for Test Automation as adopted by Software Testing Managers
It is quite natural for the business managers to aim to automate every activity that is performed by the human beings. All of us hear a lot about software test automation and its benefits. The reasons for building such common beliefs can be the following:
a) Automation saves on labor costs
b) Automation eliminates human error
c) Automation increases productivity
d) Automation brings information online and in real time etc. etc.
But apart from the above the biggest fact that remains live & can�t be forgotten is that, not all the activities can be automated. It is amazing to note the fact about the amazingly small number of activities involved in software testing that can actually be automated.
Let us analyze the activities involved in a typical GUI level regression test.
1) Analyzing the specification documents and all other documents for any error or ambiguity.
2) Analyzing the source code with a view to find any error.
3) Designing of test cases.
4) Creating the test data.
5) Executing the tests for the 1st time.
6) Evaluating the 1st result.
7) Reporting a defect from the very 1st run.
8) Debugging the tests.
9) Saving the code.
Saving the results.
11) Documenting the tests.
12) Building a traceability matrix (i.e. tracing back the test cases to specifications or requirements).
13) Selecting the test cases for execution.
14) Executing the tests.
15) Recording the results.
16) Evaluating the results.
17) Measuring the results (like functional test results or performance test results,).
18) Reporting and analyzing the errors.
19) Maintaining, updating, and debugging the tests.
Of all the activities listed above, only the activities mentioned at sr. 14 & 15 can be automated. The rest of the activities will always depend on human testers and cannot be automated. Sr. 18 can be partially automated, as analyzing the results is always a human activity but error reporting can be automated fully.
In the case of software test management, some amount of automation has been found to be successful in areas like test case execution, test case management, requirements management etc. etc.
Evolution of different Frameworks:
Testing can be effectively done through the use of scientific framework approach like
1) Keyword-Driven Automation Framework
2) Data-Driven Automation Framework
To understand these current generation frameworks we need to firstly know the history of evolution of different frameworks for software development.
Phase-1: Procedure-driven Application Framework: The framework for software development started with the procedural model. Source code used to be organized inside procedures. These procedures could be called anywhere inside the software application. This used to facilitate code reuse, as the procedure could be passed with different kinds of data to generate different results using the same procedure. A limitation of the procedure-driven application framework was that it would not allow different implementations of the same procedure. This is required to eliminate the large number of procedures needed to do similar kinds of work in the code.
Phase – 2: Object-oriented Programming Framework: Then came the object-oriented programming framework. Here a class, once created, could be implemented in different ways to do similar kinds of work. This reduced the need to create a large pool of classes to be created for the software application. This framework also allowed the possibility of child classes to be part of parent classes. This resulted in creating packages of classes that together could do many kinds of tasks by replacing a child inside a parent class with another child. These measures ensured that code reuse could be enhanced to the maximum extent possible. Object-oriented programming also helps in maintaining the code. If any change in definition of any class is required, then it is done in that class; the rest of the software changes automatically, as wherever that class is called, the changed class behavior reflects the output of that class to the whole application. So the need to change the code in a lot of places is eliminated.
So object-oriented programming had made a huge impact in improving productivity of software developers in recent years.
Phase – 3: Keyword-driven Automation Framework: In actual practice, test cases contain many steps that are identical or similar to steps in other test cases. A limitation of the record-and-play type of automation is that these similar steps are recorded again and again during recording time. Thus, for the same steps across many test cases, separate scripts are generated even though they are the same pieces of code. Now if the application that is being tested gets changed, then you will have to make changes at all of these places.
Now suppose you devise a method so that the generated scripts do not repeat for the same pieces of code. Instead a placeholder for these pieces of code is placed and the code in fact is kept at just one place. The code gets called at these placeholders and gets executed. In such a scenario, maintaining the code base of the automation script will become very easy. Whenever any changes occur in the application under test, you just have to change the code base once at places where you are keeping the code. Now when that piece of code is called at placeholder places, this changed code gets executed. Thus maintenance effort will be significantly reduced and managing the automation script base will become a lot easier.
To achieve this goal, some approaches have been devised. The best one is known as the keyword-driven automation framework.
Typical Steps involved in Creating Keyword Framework: For creating a keyword driven framework we don’t need to make changes in the way test cases are written for manual execution. Typical steps are:
Step-1: Design and write your test cases.
Step-2: Find out which test cases have steps that are the same for other test cases.
Step-3: Make an inventory of these steps.
Step-4: Use your automation tool to record these steps.
Step-5: Input appropriate names for each of these scripts. These will be the functions that can be called inside the main script.
Step-6: Go back to your test cases and record for the entire test case suite.
Step-7: Delete the generated scripts for the steps for which you had done a separate recording.
Step-8: Put a placeholder in place of deleted code (function calls).
Step-9: Call the appropriate function at these places.
That’s all – Now you are ready.
The keyword framework is similar to procedural programming. So test automation has not reached the stage that object-oriented programming has reached. Still, using keyword framework, you are promoting code reuse and increasing productivity.
Phase – 4: Data-driven automation framework:
When you run your test case, you repeat the test case with checks for boundary values, acceptable values, and unacceptable values to ensure that the software application under test is working fine for all kinds of user inputs.
In data-driven automation testing, you iterate the test case by supplying different values to the same test case.
The test data can be taken from a database or a data table, captured from a GUI, input manually in input boxes in the application, and so on.