Analysis of defects found during software testing & action plan to prevent them
ISTQB certified experts like “Technical Test Analysts” perform comprehensive root cause analysis of defects uncovered during software testing. Such root cause analysis leads to the formation of the customized best practices that prevent those defects from recurring in subsequent iterations of software development.
When testers uncover defects during integration, system, and acceptance testing, they assess defect severity and report severe defects in the problem tracking system. Defects should be considered severe if they cause intermittent or long-term system failures, or produce incorrect outputs, or if frequency of occurrence warrants a customized practice to prevent them. Customized best practices are then created based on the root cause analysis of these defects.
The life cycle of a defect detected by a tester during the software testing effort is described in the following figure.
taken by the “Technical Test Analysts” in the comprehensive analysis is the classification of the reported defects. Such classification depends on the type of products developed by the organization and existing practices.By conducting a comprehensive defect analysis and classification, patterns and trends can be identified and the cause of each defect, or class of defects, can be isolated. As an outcome of defect analysis, following table can be created.
This table contains information about the cause of each defect as well as the best recommended action.
|Sr.||Reasons of Defect||Best Action Plan or Policy|
|1.||Requirement incomplete||Review best practices for Gathering and Organizing Requirements.|
|2.||Algorithm incorrect||Review design and construction best practices for Defining Application Logic and TDD for the Algorithmically Complex Code.|
|3.||Variable initialization problems||Create a new Coding Standard Practice and verify it with a static analysis tool.|
|4.||Metadata not updated||Review the policy for use of the automated build system.|
|5.||Prohibited function calls used||Create a new Coding Standard Practice and verify it with a static analysis tool.|
|6.||Shared function modified||Review Coding Standard Practice for cross-code correlations.|
|7.||Interface type mismatch between modules||Review integration testing test progression criteria.|
|8.||Code reuse problems||Review use of assertions in Coding Standard Practice.|
It is important to understand that the root cause analysis may reveal that some defects should have been prevented by an existing practice or policy that is already used (e.g., the above table shows defects that should have been prevented by an automated build system, already in place). This indicates that the implemented practice or policy needs to be tightened or modified. In other words, for each severe defect that eludes the standard defect prevention practices, the testers, developers, and the architect should try to establish how the defect occurred and how to prevent it.
Classification of different Coding Defects: It is quite similar to the detailed design-level defect classification.
According to categorizations done by software testing experts like Cem Kaner & Burstein, a comprehensive classification of coding defects is as under:
|Sr.||Type of Coding Defect||Reasons of Defect|
|1.||Initialization defects||Due to omitted or incorrect variable initialization statements.|
|2.||Algorithmic and processing defects||Due to incorrect implementation of algorithmic design, improper progression of steps in processing data, or incorrect coding of arithmetic operations, and Boolean expressions, or inadvertently lost precision.|
|3.||Control, logic, and sequence defects||Due to incorrect implementation of conditional statements such as “if-then-else,” expressions of “case” statements, or loop boundary problems.|
|4.||Error handling defects||Due to the failure to properly anticipate invalid inputs to the program, and the omission or incomplete implementation of error handling routines. These types of defects often predispose the system to security vulnerabilities.|
|5.||Code reuse defects||Due to the failure to account for the reused code constraints in the implementation of its interfaces and in its use.|
|6.||Data defects||Due to the incorrect implementation of data structures, such as wrong variable types, pointer mismatches, or improper allocation of memory or secondary storage elements.|
|7.||Data flow defects||Due to improper data sequence. For example, a variable should be initialized first, then used, and then disregarded. If a variable is disregarded before it is used, a data flow defect occurs.|
|8.||Service, module, and object interface defects||Due to a faulty message format between services, improper parameter type declaration, or incorrect parameter numbers passed between modules or objects.|
|9.||External hardware and software interface defects||Due to the erroneous coding of interfaces with external hardware components, incorrect implementation of contracts or communication protocols with external services, wrong parameters passed to system calls, or improper access of COTS.|
|10.||Usability defects||Due to missing, incorrect, or improper implementation of elements in the user interface, mostly due to missing or improper use of the graphical user interface design and its policy.|
|11.||Code documentation defects||These occur in situations when the code documentation does not reflect code functionality or if it is too cryptic to help in understanding code functionality.|
|12.||Configuration and version defects||Due to incorrect versioning of source code components, improperly linked libraries, and other parts of the system under the development.|