of test planning, analysis and design. In turn, this can enable more focused and appropriate testing to be done - rather than having many testers working long hours, running hundreds of tests.
Related to this is the fact that the automation of any process usually results in more predictable and consistent results. Similarly, the use of test tools provides more predictable and consistent results as human failings such as manual keying errors, misunderstandings, incorrect assumptions, forgetfulness, etc., are eliminated.
It also means that any reports or findings tend to be objective rather than subjective. For instance, humans often assume that something that seems reasonable is correct, when in fact it may not be what the system is supposed to do.
The widespread use of databases to hold the data input, processed or captured by the test tool, means that it is generally much easier and quicker to obtain and present accurate test management information, such as test progress, incidents found/fixed, etc.
<<<<<< =================== >>>>>>
Q. 82: What are the risks associated to the use of testing tools?
Most of the risks associated with the use of test tools are concerned with over-optimistic expectations of what the tool can do and a lack of appreciation of the effort required to implement and obtain the benefits that the tool can bring.
For example, consider the production environments of most organizations considering using test tools. They are unlikely to have been designed and built with test tools in mind. Therefore, assuming that you want a test environment to be a copy of production or at least as close to it as possible, you will also have a test environment that is not designed and built with test tools in mind.
Consider the test environments used by vendors to demonstrate their test tools. If you were the vendor would you design the environment to enable you to demonstrate the tool at its best or to demonstrate the shortcomings it may encounter in a typical test environment?
Therefore, unless detailed analysis and evaluation is done, it is likely that test tools will end up as something that seemed a good idea at the time but have been largely a waste of time and money.
After a test tool has been implemented and measurable benefits are being achieved, it is important to put in sufficient effort to maintain the tool, the processes surrounding it and the test environment in which it is used. Otherwise there is a risk that the benefits being obtained will decrease and the tool will become redundant. Additionally, opportunities for improving the way in which the tool is used could also be missed.
For example, the acquisition of various test tools from multiple vendors will require interfaces to be built or configured to import and export data between tools. Otherwise much time may be spent manually cutting and pasting data from one tool to another. If this is not done, then data inconsistencies and version control problems are likely to arise. Similar problems may arise when testing with third-party suppliers or as a result of mergers and acquisitions.
Maintenance effort will also be required to upgrade and re-configure tools so that they remain compliant with new platforms or operating systems.
<<<<<< =================== >>>>>>
Q. 83: What is the purpose of using Test Management Tools?
Test management tools provide support for various activities and tasks throughout the development life cycle. Some of these activities are supported by the provision of interfaces to other more specialist tools like test execution tools. Other test management tools provide fully integrated modules that provide some or all of the services / functions provided by more specialist tools like incident management or requirements management tools.
Following figure shows how a test management tool is the hub of a set of integrated test tools.
Test management tools provide an architecture for creating, storing and editing test procedures. These may be linked or traced to requirements, test conditions and risks. Such test procedures can then be prioritized or grouped together and scheduled so that they are run in the most effective and efficient order. Some test management tools allow dependencies to be recorded so that tests that will fail owing to a known defect can be highlighted and left unexecuted. This allows testers to be redirected to run the highest priority tests available rather than waste their time and the test data they have prepared on tests that are certain to fail.
Tests can be recorded as passed or failed and usually a test management tool provides an interface to an incident management tool so that an incident can be raised if the actual and expected results differ.
Test management tools can provide management information and reports on test procedures passed or failed. The amount of integration with other specialist tools is significant here. For instance, integration with requirements management tools allows reports to be produced on test progress against one or more requirements. Integration with incident management tools allows reports also to include analysis of defects against requirements.
Test management tools generally hold data in a database. This allows a large amount of reports and metrics to be produced. The metrics produced can be used as inputs to:
1) Test and project management to control the current project.
2) Estimates for future projects.
3) Identifying weaknesses or inefficiencies in the development or test process that can be subsequently investigated with the aim of improving them.
Test management information reports should be designed to meet the needs of project managers and other key users. It may be necessary to export data to a spreadsheet in order for it to be manipulated into the format required.
A test management tool can enable reuse of existing testware in future test projects.
<<<<<< =================== >>>>>>
Q. 84: What is the purpose of using Incident Management Tools?
Incident management tools are also known as defect management tools. These are one of the most widely used types of test tool. At a basic level incident management tools are used to perform two critical activities: creation of an incident report; and maintenance of details about the incident as it progresses through the incident life cycle.
The level of detail to be captured about the incident can be varied depending upon the characteristics of the tool itself and the way in which the incident management tool is configured and used by the test organization.
For example, the incident management tool could be configured so that lots of mandatory information is required in order to comply with industry or generic standards such as IEEE 1044. In addition, workflow rules may also be applied to ensure that the agreed incident life cycle is strictly applied, with incidents only able to be assigned to particular teams or users. Alternatively, the tool could be configured to require very limited mandatory information, with most fields being free format.
Incident management tools also use a database to store and manage details of incidents. This allows the incident to be categorized according to the values stored in appropriate fields. Such values will change during the incident life cycle as the incident is analyzed, debugged, fixed and re-tested. It is often possible to view the history of changes made to the incident.
The database structure also enables incidents to be searched and analyzed (using either filters or more complex SQL-type queries). This provides the basis for management information about incidents. Note that as the values held against each incident change, the management information will also change. Therefore users need to be aware of the danger of using outdated reports.
This data can also be used in conjunction with data held in test management tools when planning and estimating for future projects. It can also be analyzed to provide input to test process improvement projects.
Fields in the database structure normally include:
1) Priority (e.g. high, medium, low).
2) Severity (e.g. high, medium, low).
3) Assignee (the person to whom the incident is currently assigned, e.g. a developer for debugging, a tester to perform re-testing).
4) Status in the incident life cycle (e.g. New, Open, Fixed, Reopen, Closed).
Some test management tools include fully integrated incident management tools as part of their core product, whilst other incident management tools can be integrated with test management, requirements management and/or test execution tools. Such integration enables incidents to be input and traced back to test cases and requirements.
<<<<<< =================== >>>>>>
Q. 85: What is the purpose of using Requirements Management Tools?
Requirements management tools are used by business analysts to record, manage and prioritize the requirements of a system. They can also be used to manage changes to requirements - something that can be a significant problem for testers as test cases are designed and executed to establish whether the delivered system meets its requirements. Therefore if requirements change after tests have been written then test cases may also need to change. There is also a potential problem of changes not being communicated to all interested parties, thus testers could be using an old set of requirements whilst new ones are being issued to developers.
The use of a traceability function within a requirements tool (and/or integrated with a test management tool) enables links and references to be made between requirements, functions, test conditions and other testware items. This means that as requirements change, it is easy to identify which other items may need to change.
Some requirements management tools can be integrated with test management tools, whilst some test management tools enable requirements to be input and related to test cases.
Requirements management tools also enable requirements coverage metrics to be calculated easily as traceability enables test cases to be mapped to requirements.
As can be seen, traceability can create a lot of maintenance work, but it does highlight those areas that are undergoing change.
Part - 18 of the Crash Course - ISTQB Foundation Exam