Analysis of defects found during software testing and action plan to prevent them

Posted in: ISTQB Advanced CTAL Exam Preparation

Analysis of defects found during software testing & action plan to prevent them

ISTQB certified experts like “Technical Test Analysts” perform comprehensive root cause analysis of defects uncovered during software testing. Such root cause analysis leads to the formation of the customized best practices that prevent those defects from recurring in subsequent iterations of software development.

When testers uncover defects during integration, system, and acceptance testing, they assess defect severity and report severe defects in the problem tracking system. Defects should be considered severe if they cause intermittent or long-term system failures, or produce incorrect outputs, or if frequency of occurrence warrants a customized practice to prevent them. Customized best practices are then created based on the root cause analysis of these defects.

The life cycle of a defect detected by a tester during the software testing effort is described in the following figure.

The life cycle of a defect detected by a tester during the software testing

The first step

taken by the “Technical Test Analysts” in the comprehensive analysis is the classification of the reported defects. Such classification depends on the type of products developed by the organization and existing practices.By conducting a comprehensive defect analysis and classification, patterns and trends can be identified and the cause of each defect, or class of defects, can be isolated. As an outcome of defect analysis, following table can be created.

This table contains information about the cause of each defect as well as the best recommended action.

Sr.	Reasons of Defect	Best Action Plan or Policy
1.	Requirement incomplete	Review best practices for Gathering and Organizing Requirements.
2.	Algorithm incorrect	Review design and construction best practices for Defining Application Logic and TDD for the Algorithmically Complex Code.
3.	Variable initialization problems	Create a new Coding Standard Practice and verify it with a static analysis tool.
4.	Metadata not updated	Review the policy for use of the automated build system.
5.	Prohibited function calls used	Create a new Coding Standard Practice and verify it with a static analysis tool.
6.	Shared function modified	Review Coding Standard Practice for cross-code correlations.
7.	Interface type mismatch between modules	Review integration testing test progression criteria.
8.	Code reuse problems	Review use of assertions in Coding Standard Practice.

It is important to understand that the root cause analysis may reveal that some defects should have been prevented by an existing practice or policy that is already used (e.g., the above table shows defects that should have been prevented by an automated build system, already in place). This indicates that the implemented practice or policy needs to be tightened or modified. In other words, for each severe defect that eludes the standard defect prevention practices, the testers, developers, and the architect should try to establish how the defect occurred and how to prevent it.

Classification of different Coding Defects: It is quite similar to the detailed design-level defect classification.

According to categorizations done by software testing experts like Cem Kaner & Burstein, a comprehensive classification of coding defects is as under:

Sr.	Type of Coding Defect	Reasons of Defect
1.	Initialization defects	Due to omitted or incorrect variable initialization statements.
2.	Algorithmic and processing defects	Due to incorrect implementation of algorithmic design, improper progression of steps in processing data, or incorrect coding of arithmetic operations, and Boolean expressions, or inadvertently lost precision.
3.	Control, logic, and sequence defects	Due to incorrect implementation of conditional statements such as “if-then-else,” expressions of “case” statements, or loop boundary problems.
4.	Error handling defects	Due to the failure to properly anticipate invalid inputs to the program, and the omission or incomplete implementation of error handling routines. These types of defects often predispose the system to security vulnerabilities.
5.	Code reuse defects	Due to the failure to account for the reused code constraints in the implementation of its interfaces and in its use.
6.	Data defects	Due to the incorrect implementation of data structures, such as wrong variable types, pointer mismatches, or improper allocation of memory or secondary storage elements.
7.	Data flow defects	Due to improper data sequence. For example, a variable should be initialized first, then used, and then disregarded. If a variable is disregarded before it is used, a data flow defect occurs.
8.	Service, module, and object interface defects	Due to a faulty message format between services, improper parameter type declaration, or incorrect parameter numbers passed between modules or objects.
9.	External hardware and software interface defects	Due to the erroneous coding of interfaces with external hardware components, incorrect implementation of contracts or communication protocols with external services, wrong parameters passed to system calls, or improper access of COTS.
10.	Usability defects	Due to missing, incorrect, or improper implementation of elements in the user interface, mostly due to missing or improper use of the graphical user interface design and its policy.
11.	Code documentation defects	These occur in situations when the code documentation does not reflect code functionality or if it is too cryptic to help in understanding code functionality.
12.	Configuration and version defects	Due to incorrect versioning of source code components, improperly linked libraries, and other parts of the system under the development.