Anatomy of Various Types of Experience-Based Testing Techniques
Experience-based techniques are based on the tester’s experience with testing, development, similar applications, the same application in previous releases, and the domain itself. The tester brings all their knowledge to bear when designing the test cases.
ISTQB certified experts like “Test Analyst” and “Technical Test Analyst” are qualified to carry out experience-based testing because of the following:
1) They greatly understand when and why to use experience-based techniques.
2) They have ability to explain the specification, execution, and reporting of exploratory tests.
3) They have ability to specify tests using software fault attacks that target specific defects.
4) They have ability to analyze a system to determine which specification-based, defect-based, or experience-based techniques should be applied to achieve specified goals.
Experience-based test design techniques also consider defect history, but unlike defect-based test design, these techniques do not necessarily have a systematic way of verifying coverage and may not have any formal coverage criteria.
Because the tester employs his knowledge
gained from experience, this can be a very high-yield technique for error detection. These methods are not as effective at achieving a specific level of test coverage and tend to be light on documentation, making repeatability secondary to accomplishing the goal of finding defects.When executing tests using experience-based techniques, the tester is able to react to events and adjust future tests accordingly. Executing and evaluating the tests are concurrent tasks. In some cases, the tests are actually created at the same time they are executed, making this a dynamic testing approach. In some cases, tests are created in advance, but testing is later adjusted depending on the findings, making this a more structured approach.
Following four major types of experience-based testing are being discussed here.
1) Error guessing
2) Checklist based Testing
3) Exploratory Testing
4) Attack Testing
1) Error Guessing: It doesn�t sound very official, but it works.
Error guessing is commonly used in risk analysis to “guess” where errors are likely to occur and to assign a higher risk to the error-prone areas. Error guessing as a testing technique is employed by the tester to determine the potential errors that might have been introduced during the software development and to devise methods to detect those errors as they manifest into defects and failures.
Error guessing coverage is usually determined based on the types of defects that are being sought. If there is a defect (or bug) taxonomy available, that can be used as the guideline. If taxonomy is not employed, the experience of the tester and the time available for testing usually determine the level of coverage.
2) Checklist-Based Testing: Checklist-based testing is used by experienced testers who are using checklists to guide their testing. The checklist is basically a high-level list, or a reminder list, of areas to be tested. This may include items to be checked, lists of rules, or particular criteria or data conditions to be verified. Checklists are usually developed over time and draw on the experience of the tester as well as on standards, previous trouble-areas, and known usage scenarios. Coverage is determined by the completion of the checklist.
3) Exploratory Testing: Remember that exploratory testing is not ad hoc testing. Exploratory testing occurs when the tester plans, designs, and executes tests concurrently and learns about the product while executing the tests. As testing proceeds, the tester adjusts what will be tested next based on what has been discovered. Exploratory tests are planned and usually guided by a test charter that provides a general description of the goal of the test. The process is interactive and creative, ensuring that the tester’s knowledge is directly and immediately applied to the testing effort. Documentation for exploratory testing is usually lightweight, if it exists at all.
Coverage for exploratory testing can be very difficult to determine. The use of the charter helps to define the tasks and objectives of the testing. The charter is used to specify what is to be tested, what is the goal, and what is considered to be in and out of scope, and sometimes it also indicates what resources will be committed (including time allocated for the test session). If there is a clear charter, coverage can be determined based on adherence or expansion of the charter. In some cases, coverage is also determined based on defect or quality characteristics that have been addressed by the testing.
4) Attack Testing: Software attacks (sometimes called fault attacks) are focused on trying to induce a specific type of failure. When performing attack testing, you should consider all areas of the software and its interaction with its environment as opportunities for failures. Attacks target the user interface, the operating system, interfacing systems, database interfaces, APIs, and any file system interaction. Anytime data is being exchanged, it is potentially vulnerable to a failure and consequently is an excellent target for an attack.
Coverage for attack testing is usually measured by determining if all the potentially vulnerable interfaces have been tested
1) Experience-based testing can have huge benefits, as can be seen from several examples. But it is better to use these techniques wisely.
2) Experience-based testing isn�t an excuse to have poor testing documentation or to jump into testing with no plan.
3) This type of testing is most effective in the hands of experienced testers who will bring their knowledge to bear in the most productive ways possible. Why it is so?
a) Experienced testers know what needs to be documented to assess coverage and ensure some level of repeatability.
b) Experienced testers also know what to target to find the biggest bugs quickly and return a high yield during their testing time.
4) These are not techniques for novices-but they are definitely appropriate and productive techniques for the advanced tester.
An expert on R&D, Online Training and Publishing. He is M.Tech. (Honours) and is a part of the STG team since inception.