How to Select the Scripting Technique-Out of Data-driven and Keyword-driven
Software testing experts practice the use of many available scripting techniques, which are not mutually exclusive, rather in some sense these are used together. Every technique has some benefit & pitfalls affecting the time and effort needed for their implementation and in the maintenance of the supported test cases.
In this article we�ll focus on more matured techniques like data driven & keyword driven techniques that are largely exploited by the software testing engineers these days. The discussion would help software testing engineers to make a suitable decision in choosing a technique that seem to suit their application.
Data-driven Scripting Technique:
In this scripting technique a separate data file stores all the test inputs instead of script storing it. All the controlling information such as navigation menu etc. are retained in the script only. During the execution of tests, all inputs are accessed directly from the data file instead of being read from
the script.
Benefits of Data-driven Scripting Approach:
At the data-driven scripting level we can draw many significant advantages from test automation that are:
1) Same script can be used to run many different tests with very little extra effort. Here the software testing engineers just need to specify a new set of input data and corresponding expected results for every additional test case. Thus similar tests can be added very quickly without the need to implement any more scripts.
For example, a test for a banking application needing entering the particulars of a new account and verify that the database has been correctly updated. Another test would do exactly the same except it would work upon a different bank account. It needs the same instructions but different inputs and different expected outcomes (the values that describe a different bank account). Both these tests can be implemented with the help of the same test script and one data file.
2) We can add many more test cases around every original regression tests & automate them, since these are expected to use exactly the same script instructions, except areas of frequent change being the input data and the expected outcomes. Ref figure below.
3) We can easily customize the format of the data file to suit the needs of the software-testing engineer.
For example, the tester can add comments to the data file that the script will be made to ignore thereby making the data file much more understandable and maintainable.
4) Testers can specify the test input from a spreadsheet package being from an entirely different input format. Once the spreadsheet is completed it can be converted to a neutral format like a comma-separated text file. The spreadsheet is kept as the master version and any changes required are made directly to the spreadsheet instead in the data file. Whenever changes are made the data file is regenerated from the spreadsheet.
5) Testers can easily choose the format and layout of their test data and can comfortably add the test data without much of the programming knowledge of the scripting language supported by the tool.
6) The scripts become comparatively simpler & lighter due to the removal of the test inputs and the expected results from the script and transfer to the data file. Every expected result is directly associated with a particular test input, thus when the inputs are in the data file, the results to compare are also parked in the data file.
Many automation tools are available that encourage this by providing effective means for capturing information on the screen and storing it elsewhere as expected results for comparison with new results generated by a later run of the test.
7) This doesn�t call for any additional effort towards maintenance of comprehensive set of test scripts especially when we have hundreds rather than dozens of tests.
8) For large systems involving large number of tests which are expected to last long and change frequently, organizations tend to realize tangible gains of saved efforts.
Few Shortcomings of Data-driven Scripting Approach:
Despite great simplicity, data-driven scripts offer few stumbling blocks that are given below.
1) Navigation and actions performed need to be the same for every test case, and the logical knowledge of what the tests are is built into both the data file and the control script, hence both these need to be synchronized, otherwise there will be a collapse.
2) Writing the control scripts needs involvement of technically competent persons with programming related background. It is all the ways better that the software testing engineers in the teams are comfortable with programming of automated test scripts.
3) Initial set-up takes of lot of effort, consumes considerable time & upfront costs.
4) For small systems, wherein many tests aren�t available, this approach seems to be more cumbersome rather than being helpful.
Keyword-driven Scripting Technique:
‘Keyword-driven scripts’ are nothing but a logical extension of the more advanced version of the data-driven technique. Keyword-driven scripts incorporate the application knowledge
The keyword-driven is a combination of data-driven technique and the ability to efficiently describe the automated test cases with sufficient clarity. This involves expansion of the data file to describe the test cases that testers want to automate with the help of set of few keywords.
The control script is written with sufficient intelligence that it�s able to interpret the keywords but this is implemented outside of the control script itself. This additional level of separation from the implementation (tool) level does require an additional layer of technical implementation. Although this sounds as though it would be more difficult, we believe that it is actually much easier in the long run. The trick is just to identify the right keywords for the job.
Here the test cases are described by a test file rather than a data file. The test file describes as to what is done by the test case; and not how it is done. The control script reads each keyword in the test file turn by turn and calls the associated supporting script. The supporting script may require more information from the test file, and this can either be read directly or passed on by the controlling script. The control script is no longer tied to a particular feature of the software under test, nor indeed to a particular application or system.
Keyword-driven scripting permits software-testing engineers to adopt a descriptive approach. To implement an automated test case they just need to provide a description of the test case more or less in a similar manner as we would do for an intelligent manual tester.
This descriptive approach helps us in building knowledge of the software under test into our test automation environment. This knowledge becomes a part and parcel of the supporting scripts.
Due to this we can develop the test cases separately from the test scripts and vice versa. Persons having business knowledge and tester like skills can concentrate on the test files while persons having technical knowledge can concentrate on the supporting scripts. Of course such persons can�t work completely independently since the supporting scripts have to support the keywords used in the test files.
Benefits of Keyword-driven Scripts:
‘Keyword-driven scripts’ have immense advatages, few of them are listed below.
1) Since here the number of scripts remain proportional to the size of the software under test rather than the number of tests. Thus large number of tests can be implemented without increasing the number of scripts, once the basic application supporting the scripts are available.
2) Huge reduction in script maintenance cost and at the same time speeding up the implementation of automated tests.
3) Few software-testing engineers with little or no knowledge of programming can implement several thousand test cases with the help of just a few hundred scripts.
4) Automated tests can be implemented in a way that is independent of the test tool or even the platform. In the eventuality of changing the tool due to any reasons, our investment in the test cases remain safe & we would possibly need re-implementation of the supporting scripts.
5) In case our software under test is to be tested on another hardware platforms that are not supported by the available test tool, we don�t have to change our test cases again. What we need is merely the re-implementation of the supporting scripts.
6) The way in which tests are implemented can be customized to suit the software-testing engineers rather than to the testing tool. This can be done with the help of the format and tools with which the testing teams are most comfortable with e.g. text editor, spreadsheet, or word processor.
7) We can optimize the format so that new automated tests can be implemented as quickly as possible in a safer way and avoiding many of the mistakes that could call for excessive debugging of tests.
8) Simple tools can be easily implemented to perform static analysis on the automated test definitions.
9) We can do the separation of the tester concerns from the technical concerns.
Conclusion:
Keyword driven scripting & automation technique certainly scores an edge over the competing data-driven technique due its several underlying benefits & many companies are formulating their own frameworks being in line with the automation tool available.
Many More articles on Test Automation Frameworks
An expert on R&D, Online Training and Publishing. He is M.Tech. (Honours) and is a part of the STG team since inception.