Arial, Helvetica, sans-serif">The customer satisfaction Index is extremely important & information about this index is captured before the delivery of the product as well as after the delivery of the product to the customer.
Periodic customer satisfaction reviews are conducted on regular basis. Standard questionnaires are used to capture & analyze the following information in a structured way.
- Number of system enhancement requests made per year.
- Number of maintenance fix requests made per year.
- User friendliness: Volume of calls made to the customer service center / call center.
- User friendliness: Amount of time required to train a new user.
- Number of product recalls or release of fix.
- Number of production re-runs among in-house information systems groups.
This is normalized
2. Measure of Quantities of Defects Delivered:
- As per function point (or as per the LOC) at the time of product delivery (can be initial 3 months or first year of the operation).
- On the ongoing basis by the level of severity (can be by the year of operation).
- According to the category or cause of the defects e.g. It can be requirements defect, design defect, code defect, documentation / on-line help defect or defect introduced by the fixes itself.
3. Turnaround Time to the Users or Responsiveness:
- Turnaround time for defect fixes according to the level of severity.
- Time for minor vs. major enhancements; actual vs. planned elapsed time.
Product volatility can be measured in terms of a Ratio of number of Maintenance Fixes versus the number of Enhancement Requests made by the customer with an intention to enhance the functionality of the product.
4. Measure of Product Volatility:
Maintenance fixes refer to the repair of the system thereby bringing it back to compliance with the desired specifications.
5. Measure of Defect Ratios:
- Number of defects captured per function point detected after the product delivery.
- Number of defects captured per LOC detected after the product delivery.
- Ratio of number of Pre-delivery defects versus number of annual Post-delivery defects.
- Number of defects per function point of the system modifications.
6. Measure of Efficiency of Defect Removal:Efficiency of defects removal depends upon the number of post-release defects i.e. the defects detected by the customer during field operation). Such defects can be categorized according to the level of their severity.
A good measure of efficiency of defect removal is the ratio of defects detected internally prior to release through inspection and testing reflected as a percentage of all defects. Here the term all defects refer to the defects detected internally plus the number of defects detected externally by the customer during the first year after the delivery of the product.
7. Measure of Complexity of Delivered Product:
- Through McCabe's Cyclomatic complexity counts across the system.
- Through Halstead’s measure.
- Through Card's design complexity measures.
- Through Predicted defects and maintenance costs, based on the complexity measures.
8. Measure of Test Coverage:
- Measure of breadth of functional coverage.
- Measure of percentage of paths, branches or conditions actually tested.
- Measure of percentage by level of criticality i.e. perceived level of risk of paths.
- Measure in terms of ratio of number of detected faults versus the number of predicted faults.
9. Measure of Cost of Defects:
- In terms of business loss accrued due to every single defect which occurs during the operation.
- In terms of costs due to business interruption & costs of work-arounds.
- In terms of cost attributed to the loss of sales and loss of goodwill.
- In terms of litigation costs resulting from defects.
- Annual maintenance cost reflected per function point.
- Annual operating cost reflected per function point.
- In terms of quantifiable damage to the career of the team leader.
10. Measure of Costs related to Quality Activities:
- Costs of conducting reviews, inspections and preventive measures.
- Costs of doing test planning and preparation.
- Costs of test execution, defect tracking, version and change control.
- Costs of performing diagnostics, debugging and fixing.
- Costs of acquiring tools and tool support.
- Costs of maintaining library of various test cases.
- Costs of testing & QA education associated with the product
- Costs of monitoring and oversight by the QA organization
11. Measure of Costs of Re-work:
- Costs of doing re-work. It can be defined as number of rework hours reflected as a percentage of the hours spent in original coding.
- Costs of doing re-work defined as number of reworked LOC reflected as a percentage of the total LOC delivered to the customer.
- Costs of doing re-work defined as number of reworked software components (reflected as a percentage of the total components delivered to the customer.
12. Measure of Reliability:
- In terms of the availability (of the system. This can be measured in terms of a percentage of time a system is actually available for use versus the time the system is expected to be available.
- In terms of Mean Time Between Failure (MTBF).
- In terms of Man Time To Repair (MTTR)
- In terms of Reliability ratio measured as MTBF / MTTR.
- In terms of number of product recalls or number of fix releases.
- In terms of number of production re-runs as a ratio of production runs
Various formulas or Metrics used during Software Testing:
1) Test Coverage: Defined as Number of units (KLOC / FP) tested / total size of the system. (KLOC refers to Kilo Lines of Code)
2) Number of Tests per Unit Size : Defined as Number of Test Cases per KLOC / FP (KLOC refers to Kilo Lines of Code)
3) Acceptance Criteria Tested : Defined as Acceptance Criteria Tested / Total Acceptance Criteria
4) Percentage Testing cost : Defined as Actual Cost of Testing / Total Cost * 100
5) Cost to Find the Defect : Defined as Actual Cost of Testing / Number of Defects Found
6) Achievement of Budget: Defined as Actual Cost Of Testing / Budgeted Cost of Testing
7) Defects Detected during Testing: Defined as Defects Detected during Testing / Total number of System Defects
8) Defects Detected during production : Defined as Defects Detected during Production / System Size
9) Quality of Testing : Defined as No of defects detected during Testing / (No of defects detected during testing + No of Acceptance Defects Detected after the Delivery) * 100
10) System Complaints : Defined as Number of Customer Complaints / Number of Transactions Processed
11) Measure by the Scale of Ten : Defined as Assessment of Testing by assigning rating in terms of scale of 1 to 10
12) Source Code Analysis : Defined as Number of Source Code Statements Changed / Total Number Of Tests.
13) Productivity of Test Planning : Defined as No of Test Cases Designed / Actual Effort for doing Design and Documentation
14) Productivity of Test Execution : Defined as No of Test Cycles Executed / Actual Effort for Testing