Tricks involved in Selection and Implementation of Test Automation Tools
Smart testing managers use comprehensive selection criteria wherein a small-scale pilot study remains helpful in reinforcing the decision & understanding the value of a particular tool in a better manner.
Successful tool selection is no guarantee of its successful implementation within the organization. Majority of the organizations buy the testing tools that become mere show pieces or in other words �shelfware�, simply because the organizations could not derive intended benefits out of their investments due to one reason or the other.
The cost of buying a software testing tool remains quite small compared to the cost of the internal efforts required for its implementation. Testing managers spend sizable amount of their time & energy in promoting the tool internally, organizing training & support, and creating suitable infrastructure to support the test automation regime.
Tool Implementation & Iceberg Analogy:
Tool implementation process is analogous to an iceberg. Like an iceberg, nine-tenth of efforts on software testing tool remains below the surface.
Purchasing a test execution tool is the easiest job, wherein we are shown the tip of the iceberg that is the rosy picture of the test automation. Majorities of the organizations
fail in test automation due to the poor visibility of the iceberg below the surface.
The tool implementation comprises of following 3 key elements:
1) Infrastructure or an automated testing regime;
2) Internal promotion activity usually performed by the �tool champion�;
3) Continuous support from the implementation team & the management;
The quantum of the 3 elements above mentioned is not precisely known in the beginning at the tool acquisition stage. The buying price of the tool is a smaller fragment of what we land up spending on above 3 elements for getting the true success.
Implementing a test execution tool is a process of �management of change� which slowly widens the acceptance and use of the software testing tool within the organization.
Personnel involved in the tool implementation & change process:
There are varieties of roles to be played in implementing a tool, not necessarily by separate individuals in smaller organizations.
The first key person is known as �tool champion�, who remains the focal point in implementing the tool. He/she is essentially an enthusiastic person having passion for implementing the test automation in the organization & has firm belief in the benefits of introducing the test automation.
The second person is known as �change agent�, who is responsible for planning & managing the day-to-day changes to the ways of working of the people. He/she is supposed to be the person having extensive experience of software testing with high analytical abilities & be well aware of latest technologies in testing. In smaller organization, �tool champion� can perform the role of �change agent� as well, however bigger organizations can have different persons for these functions.
The �testing manager� or sponsor is the third senior level person, a part of the top management, who provides the high-level support to the �tool champion� & the �change agent�. The same person in small organization can perform the roles of �tool champion� & �testing manager�.
The forth person is known as �tool custodian� who is generally responsible for the technical aspects of the tool such as implementing new versions from the tool vendor and building the automated regime and testware. He/she is supposed to hold the ownership of the standards for the process of use of the software testing tool.
The implementation team is drawn from all of those who are expected to use the tool. They are a means of communication about tool requirements and about how best to use the tool.
The first step of the implementation process is to prepare grounds for it. This requires management commitment that is both visible and material. It is important that management expectations are set at realistic levels. The next step is internal publicity and internal market research (by the implementation team).
It is a very good idea to implement a new software testing tool first of all on a small-scale pilot project. Quantified objectives are set and results are evaluated against the business case of the pilot. If the results are found encouraging, then the tool can be rolled out on a wider scale. Evaluate the tool use against the original and continuing quantified goals, and continue to improve.
Risks Involved in the mismanagement of people issues:
Managing change is primarily resolving people related issues, because it is the way people work that is being changed. It is extremely important to be aware of �people� related issues. This is one reason why lot of tool implementations fail. A smart software testing manager must remain sensitive to these issues, but generally technical persons are not aware of the emotional and psychological effects of technical change on the personnel.
If you adventure changing the way people work in the wrong way, you may risk inviting suspicion about the value of the change, and resistance to the change. Resistance is a natural part of the process of change, but if it is not handled well, it can lead to resentment or even sabotage of the change effort. An attitude of �We tried it once and we are certainly not going to try it again�, can not make any future change attempts that are much more difficult.
There are some people who tend to jump on any bandwagon that comes along – they have a very low threshold of change, and are easily dissatisfied with the status quo.
At the other extreme we have people who would say �I� have done it like this for the last 25 years and I am not going to do anything differently, no matter what happens. Such people have a very high threshold of change and will be very resistant to any new ideas.
Motivating people to change the conventional way of working:
It is very difficult to change someone’s psychological personality. However, the way to encourage people to change the way they work is to concentrate on the 3 things like:
1) Make them more dissatisfied with the current way software testing is done (For example saying �You mean you are still doing all that testing by hand?�).
2) Give them a fuller vision of the way things could be in the future (For example saying �Come and see what we have been able to do in the pilot project, and see if some of this might be of interest to you?�).
3) Explain the first easy step toward a complete change (For example saying �Let me help you automate a few key tests for you and you can carry forward on the next project�).
This is probably the most important thing you can do: to give them more detailed information about the steps they need to take from where they are now to where you want them to be in the future. People don’t like taking large steps into the dark, but if you illuminate the way, they can usually be convinced to take a small step forward. Note that this is the one reason why you need to plan the implementation, so that you have these steps mapped out in advance.
Successful change gives birth to more changes:
Once people begin to change the way they work, change gives birth to more change, provided the experience is a good one. Once they have taken the first step and found that it was beneficial, they will be more willing to take the next step, i.e. their threshold of change will be lower.
For example, if someone currently spends too much time in some tedious and error-prone task, such as populating a database before a test can be run, you could set up the test automation tool to do just this one job for them. In the first instance you may need to do it for them if they are not confident to do it themselves. In any case, they will need your support (�hand holding in the beginning�) as they have their first experience of using the software testing tool. If you can ensure that this experience is both pleasant and beneficial for them, you probably have changed the mindset of these persons, who would go on using the tool the ways it could provide benefits for them.
Evaluation of results of tool implementation:
Once the tool has been rolled out to sizable number of active users, the original business case needs a review to verify if the intended objectives have been achieved or not.
Probably we may find that few aspects have not come up to the desired expectations while at the same time we had been able to derive some other tangible benefits.
The test managers need to reassess the use of the tool based upon its performance and set newer targets for continuing automation & install some measures to measure the effectiveness of the infrastructure or the automation regime. Once you enter an auto mode of continuous analysis & subsequent improvement of the automation regime, it must never come to a halt. If continuous attention is not paid to the automation regime, it will decay over a period of time.
Ten concluding tips for realizing benefits out of a good tool:
1) Test managers need to ensure the presence of standardized ways of use of the tool, as different users are likely to use the tool in different ways.
2) While selecting the tool software testing managers must ensure that ‘ease of use� must not be the focus at the cost of good functionality. Otherwise the tool is likely to be easily misused.
3) After tool purchase the top management must display a strong commitment towards tool support for the software testing effort and it must be maintained throughout the process of implementation.
4) The top management need to display enough patience so as to provide enough time for learning & overcoming initial teething troubles, otherwise the tool is liable to head towards a premature death.
5) The change agents must keep on pumping in a continuous dose of publicity about the new tool, by publicizing the success stories with the user masses.
6) The testing managers must ensure to address all issues related to test automation in project plans, quality & test plans.
7) The testing managers must continuously strive to create an effective infrastructure for the automation regime
8) To understand the usefulness of the tool & to realize tangible benefits out of test automation project we must periodically measure the quantum of effort put in & the effort saved.
9) We must not endeavor to automate everything. Remember that true benefits can be realized by automating small � small portion of the tests. For instance, immediate problems with the regression test in hand, complex or error prone tests difficult to execute manually.
10) In the beginning, the testing managers must focus on few people may be around 10-15% of the entire team. These must be the opinion leaders, who remain eager for a change & would need lesser convincing for affecting the change.
An expert on R&D, Online Training and Publishing. He is M.Tech. (Honours) and is a part of the STG team since inception.