Exploratory Testing an Indispensable Nonsystematic Software Testing Technique
Going by the versions of ISTQB certified experts like “Advanced Test Managers” & “Technical Test Analysts”, we can say that systematic testing is not enough always! Since faults are sly we have to attack them in unpredictable ways. This is where the nonsystematic testing techniques come in as a valuable supplement to systematic testing techniques.
The great Testing Guru – Cem Kaner drafted the term “Exploratory Testing” being a type of nonsystematic testing techniques more than two decades ago, to elaborate a methodology of performing skilled activities that were quite common in the Silicon Valley. The concept of “Exploratory Testing” eventually evolved quite a lot since then.
In fact all testers perform exploratory testing in some form or the other. Some perform it more intentionally & skillfully while some do it unintentionally.
Sometimes it is worthwhile to search in a structured way and use the results to decide on the future course as they come in. This is the philosophy when people are looking for oil or mines, and it is the philosophy in exploratory testing.
What is Exploratory Testing?
Exploratory testing is testing where the tester actively controls the design of the tests as those tests are performed and uses information gained while testing to design new and better tests.
In other words exploratory testing is simultaneous:
1) Learning
2) Test design
3) Test execution
Exploratory testing is an important supplement to structured testing. As with all the nonsystematic techniques it may be used before the structured test is completely designed or when the structured test has stopped.
It is important that the actions of the exploratory testing activity are properly documented, so that we are able to know at some later date as to what had been done. The idea in exploratory testing is not that it should not be documented, but that it should be documented as we go along.
Exploratory testing is certainly not the baby of novices or inexperienced testers. Extensive testing experience and knowledge of testing techniques and typical failures are indispensable for the performance of an effective exploratory test. It is also an advantage if the tester has some domain knowledge.
The exploratory tester needs to be able to analyze, reason, and make decisions on the fly; and at the same time have a systematic approach and be creative. The tester also needs some degree of independence in relation to the manufacturing of the system – the programmer of a system cannot perform exploratory testing on his or her “own” system.
Perhaps most importantly the exploratory tester must have an inclination towards destruction. Exploratory testing will not work if the tester is “afraid” of getting the system to fail.
What are the different degrees of Exploratory Testing?
Another advocate of exploratory testing & the testing guru James Bach has defined following degrees of exploration.
1) Fully Specified
2) Loosely Specified
3) Somewhat Specified
4) Tasks
5) Roles
6) Free Exploration.
At the end of the hierarchy, we have the totally free exploration. Here the tester simply sits before the system and starts wherever he or she feels like.
A step upper, we find the exploratory testing guided by roles. Here the tester attacks the system under testing assuming a specific user role. This could, for example, be the role of an accountant, a nurse, a secretary, an executive manager, or any other role defined for the system. This provides a starting point and a viewpoint for the testing, which is exploratory within the framework of the role.
While going further up we find the exploratory testing guided by a specific task. In this the testing engineer narrows down the testing framework even more & more by testing within the viewpoint of a specific task defined for a specific role for the system under testing.
On the borderline between exploratory testing and structured testing we have the sporadically specified test. Here the tester has sketched the test beforehand and takes this as the starting point and guideline for the performance of the exploratory testing.
How do we perform Exploratory Testing?
No matter which degree of exploration we use, we have to follow the principles in the general test process. We must plan and monitor; we must specify, execute, and record; and we must check for completion.
In the planning we consider what we are going to do and who is going to do it. We must choose the degree of exploration and describe the appropriate activities. The testing activities should be divided into one-hour sessions. If the sessions are shorter we risk not getting an effective flow in the exploration; if they are longer we get tired and the effectiveness goes down.
It is important to make sure that the tester or testers are protected during the sessions. There should be no phones or other interruptions to disturb the flow of the testing.
The test specification, execution, and recording are done simultaneously during our sessions of exploratory testing. Within the defined boundaries of exploration the tester remains prepared to get distracted � reason being you never know what you may find.
Stock must be taken from time to time to verify that we are on track.
For each session we must do the following:
1) Take extensive notes and attach data files, screen dumps, and/or other documentation as appropriate
2) Produce an overview over findings
3) Reprioritize the remaining activities
When to stop Exploratory Testing?
The exploratory testing can stop when we have fulfilled our purpose.
Full Study Material for ISTQB Advanced Test Manager Exam
An expert on R&D, Online Training and Publishing. He is M.Tech. (Honours) and is a part of the STG team since inception.
Great summary of different aspects of ET.
It was specially good to see the references to James Bach’s different degrees of ET.
I have been working on documenting some of the Exploratory testing techniques and common Test Objectives to support test engineers on ET. Would be great if we could collaborate on this topic bit further. If interested, please reach out to me.
Anyway, great article.