Roles of different experts like “Test Managers”, “Test Analysts” and “Technical Test Analysts” are quite inter-linked to some extent.
The “Test Manager” is obviously responsible for managing the entire testing process. However the roles of “Test Analysts” and “Technical Test Analysts” are equally important since they are instrumental in implementing the testing process. A close understanding among the senior persons like “Test Manager”, “Test Analysts” and “Technical Test Analysts” is extremely important for the success of the project.
Thus before analyzing the roles of the three experts, let us quickly go through the typical steps involved in a generic testing process.
Every test may consist of several individual activities, however there are five generic steps encompassing all such activities. These generic test processes are diagrammatically represented in the following flowchart.
Let us briefly discuss these steps involved in the testing process.
1) Test Planning and Test Control: The “Test Manager” is responsible for following functions.
# Creation of a comprehensive Test Plan � documenting covering most of the information.
# Deciding the approach for testing.
# Planning of resources like equipment, software and personnel.
# Planning of training needs and hiring requirements.
# Setting the strategies.
# Creation of the test schedule.
# Deciding the metrics required for controlling and monitoring the project.
# Risk management � involving identification of risks backed by their mitigation plans.
When the “Test Manager” controls a project, he is expected to control its risks as well. When a risk is identified, the “Test Manager” needs to tackle it using a suitable mitigation plan. He may chose to transfer the risk somewhere else or it may even be ignored also. In order to plan & effectively deal with risk items it is essential that most of the risks be identified at the beginning of the project.
Usually risk management comes under the responsibility of the “Test Manager”. Whereas “Test Analysts” and “Technical Test Analysts” make valuable contribution of providing information helpful in risk identification and making risk mitigation plans as part of the planning process.
2) Test Analysis and Test Design: Involves deep consideration of the details of the testing project. Primarily it needs answering questions like;
# Taking decision on what to test.
# How much effort must be put in.
# What types of testing should be done.
# What type of tools will be required
During the test analysis stage, we may need to review the requirements documents & probably decide to go in for performance testing. This would call for defining performance test cases, procuring suitable performance testing tools and organizing other resources.
During the test analysis stage, we carry out the review of the test basis as well, i.e. the review of documents from which we decide as to what should we test � should we do static testing etc.
The primary objective of carrying out such reviews remains;
# To record different problems to ensure resolution as well as use it as a guideline for process improvement for the future.
# Collection of data needed to write the test specification. The test specification document, if properly named, indicates what is going to be tested.
# Creation of test design specification, including risk analysis tables that will guide a risk-based testing effort.
For an effective risk analysis contribution from different project stakeholders is extremely essential. The people who know the likely areas of failure can predict the technical risk. For example a developer knows that some of the excessively complicated areas of the code are vulnerable. Similarly testers are aware of particular portions of functionality that they find quite difficult to test well.
The “Test Manager” is responsible for coordinating and ensuring the creation of the test design specification & quality risk analysis. The “Test analysts” are responsible to ensure that the testing concerns are adequately taken care of.
3) Test Implementation and Test Execution: Based upon the test design specification document, we create different test cases and test procedures.
Two type of test cases are created:
1) Logical test cases: These are the high-level test descriptions which do not define the data for use in the tests.
2) Concrete test cases: These include the actual data, which is going to be used in the tests. Since spreadsheets containing actual data are referred by many test cases, hence they become concrete only when the instructions get joined with the data in actual practice.
“Test Procedure” is usually included in the test case itself. However according to IEEE-829 “Test Procedure” is a standalone document describing the method to execute a particular test case or a set of test cases.
The “Test Procedure” document typically contain following information:
a) Test procedure specification identifier
b) Purpose describing list of applicable test cases.
c) Special requirements.
d) Procedure steps like – log, setup, start, proceed, measure, shutdown, restart, stop, wrap-up, contingencies etc.
While designing the test cases, possibility of automated test execution is explored. If automation is considered viable, automation scripts are created at this stage of the project. Automation scripts are self-contained test procedures usually known as test scripts.
Next task is execution of the test cases may be manually or by automated means.
4) Evaluation of Exit Criteria and Reporting: It is important to know when we should finish testing. Test completion is decided based on meeting the exit criteria defined during the planning phase.
Usually it is a crucial Ship / Don�t Ship decision point, involving return of the testing information to the project team for the release decision.
“Test Summary Report” is the last test standard document according to IEEE-829 that emerges out of this step. The “Test Summary Report” can be prepared periodically in the form of a test progress report while concluding a particular level of testing � say for example after concluding the integration testing. Common practice of creating “Test Summary Report” is while concluding the testing effort.
The “Test Summary” document typically contain following information:
a) Test summary report identifier
b) Summary of evaluation of the test items
c) Variances
d) Assessment of being comprehensive
e) Summary of results
f) Evaluation of every test item
g) Summary of activities
h) Approvals
5) Test Closure Activities:The test closure activities take place after shipping of the release. These activities usually receive less attention and thus remain under budgeted due to the pressure of the next project already in the queue.
If we want to quickly return to the closing project for any maintenance releases or patches etc, it is essential that the test manager remains firm on the correct & systematic operation of the closure activities.
During test closure stage following activities are carried out;
# Carry out all types of wrap-up reporting.
# Carry out documentation & archiving of the test environments.
# Archiving the test documents and tests data.
# Clearing the decks for the project next in the pipeline.
Technical roles of “Test Analyst” and “Technical Test Analyst”:
In addition to certain specific activities that are handled exclusively by “Test Analyst” or “Technical Test Analyst”, there are certain common activities that are taken care of by either or both these persons.
Activities under the responsibility of “Test Analyst” and “Technical Test Analyst”:
1) To describe the desired features of systems-of-systems
2) To explain the factors influencing the testing of systems-of-systems
3) To describe the desired features of safety-critical systems
4) To describe the testing tasks those are particularly significant for testing safety-critical systems and provide examples of industry-specific standards.
5) To describe the desired features of real-time & embedded systems
6) To explain the criteria that influences the level of test condition development.
7) To explain and provide examples of test oracles and method of using it in test documentation.
8) To describe the conditions that must be in place prior to the test execution, including the test-ware, defect-tracking system, configuration management system and test environment etc.
9) To find out if the test completion criteria have been fulfilled according to the desired norms.
Activities under the responsibility of “Test Analyst”
1) To analyze the requirement specification & breaking it down to a test specification according to IEEE-829 standard, focusing on functional & domain test cases as well as test procedures
2) To explain the stages in the software development lifecycle where functional testing shall be suitable
3) To understand the logic behind test analysis and design being static testing techniques that can be used to find out defects
4) To prioritize the process of test cases creation & subsequent execution according to the risk and create suitable test documentation
5) To identify the activities of risk based testing for domain testing
Activities under the responsibility of “Technical Test Analyst”
1) To explain different stages required in the software development lifecycle needing structure-based testing & nonfunctional testing.
2) To identify the activities of risk based testing for technical testing
Full Study Material for ISTQB Test Analyst & Technical Test Analyst Exam
An expert on R&D, Online Training and Publishing. He is M.Tech. (Honours) and is a part of the STG team since inception.