software testing genius

All about Metrics Used by Software Testers

Welcome to “Software Testing Genius”. Subscribe to my RSS feed for latest content on Software Testing.

Download Link for your Favorite Presentation is at the End of this Page

Let us firstly understand as to what Metric is. Metric is a mathematical number which points towards some relationship between two variables. However in software terminology Metric Is a measure which quantifies some property of the software or its specifications. Software Metric can be a reflection on the status under software development cycle or some results etc. Since quantitative methods have proved so powerful in the other sciences, computer science practitioners and theoreticians have brought similar approaches to software development as well.

Thus Software Metric is a generic name for the measure of Quality of the Software Product.

Some of the common software metrics are discussed here:

1. Measure of Index of Customer Satisfaction:

Arial, Helvetica, sans-serif">The customer satisfaction Index is extremely important & information about this index is captured before the delivery of the product as well as after the delivery of the product to the customer.

Periodic customer satisfaction reviews are conducted on regular basis. Standard questionnaires are used to capture & analyze the following information in a structured way.

  • Number of system enhancement requests made per year.
  • Number of maintenance fix requests made per year.
  • User friendliness: Volume of calls made to the customer service center / call center.
  • User friendliness: Amount of time required to train a new user.
  • Number of product recalls or release of fix.
  • Number of production re-runs among in-house information systems groups.

2. Measure of Quantities of Defects Delivered:
This is normalized

  • As per function point (or as per the LOC) at the time of product delivery (can be initial 3 months or first year of the operation).
  • On the ongoing basis by the level of severity (can be by the year of operation).
  • According to the category or cause of the defects e.g. It can be requirements defect, design defect, code defect, documentation / on-line help defect or defect introduced by the fixes itself.

3. Turnaround Time to the Users or Responsiveness:

  • Turnaround time for defect fixes according to the level of severity.
  • Time for minor vs. major enhancements; actual vs. planned elapsed time.

4. Measure of Product Volatility:
Product volatility can be measured in terms of a Ratio of number of Maintenance Fixes versus the number of Enhancement Requests made by the customer with an intention to enhance the functionality of the product.

Maintenance fixes refer to the repair of the system thereby bringing it back to compliance with the desired specifications.

5. Measure of Defect Ratios:

  • Number of defects captured per function point detected after the product delivery.
  • Number of defects captured per LOC detected after the product delivery.
  • Ratio of number of Pre-delivery defects versus number of annual Post-delivery defects.
  • Number of defects per function point of the system modifications.

6. Measure of Efficiency of Defect Removal:
Efficiency of defects removal depends upon the number of post-release defects i.e. the defects detected by the customer during field operation). Such defects can be categorized according to the level of their severity.

A good measure of efficiency of defect removal is the ratio of defects detected internally prior to release through inspection and testing reflected as a percentage of all defects. Here the term all defects refer to the defects detected internally plus the number of defects detected externally by the customer during the first year after the delivery of the product.

7. Measure of Complexity of Delivered Product:

  • Through McCabe's Cyclomatic complexity counts across the system.
  • Through Halsteadís measure.
  • Through Card's design complexity measures.
  • Through Predicted defects and maintenance costs, based on the complexity measures.

8. Measure of Test Coverage:

  • Measure of breadth of functional coverage.
  • Measure of percentage of paths, branches or conditions actually tested.
  • Measure of percentage by level of criticality i.e. perceived level of risk of paths.
  • Measure in terms of ratio of number of detected faults versus the number of predicted faults.

9. Measure of Cost of Defects:

  • In terms of business loss accrued due to every single defect which occurs during the operation.
  • In terms of costs due to business interruption & costs of work-arounds.
  • In terms of cost attributed to the loss of sales and loss of goodwill.
  • In terms of litigation costs resulting from defects.
  • Annual maintenance cost reflected per function point.
  • Annual operating cost reflected per function point.
  • In terms of quantifiable damage to the career of the team leader.

10. Measure of Costs related to Quality Activities:

  • Costs of conducting reviews, inspections and preventive measures.
  • Costs of doing test planning and preparation.
  • Costs of test execution, defect tracking, version and change control.
  • Costs of performing diagnostics, debugging and fixing.
  • Costs of acquiring tools and tool support.
  • Costs of maintaining library of various test cases.
  • Costs of testing & QA education associated with the product
  • Costs of monitoring and oversight by the QA organization

11. Measure of Costs of Re-work:

  • Costs of doing re-work. It can be defined as number of rework hours reflected as a percentage of the hours spent in original coding.
  • Costs of doing re-work defined as number of reworked LOC reflected as a percentage of the total LOC delivered to the customer.
  • Costs of doing re-work defined as number of reworked software components (reflected as a percentage of the total components delivered to the customer.

12. Measure of Reliability:

  • In terms of the availability (of the system. This can be measured in terms of a percentage of time a system is actually available for use versus the time the system is expected to be available.
  • In terms of Mean Time Between Failure (MTBF).
  • In terms of Man Time To Repair (MTTR)
  • In terms of Reliability ratio measured as MTBF / MTTR.
  • In terms of number of product recalls or number of fix releases.
  • In terms of number of production re-runs as a ratio of production runs

Various formulas or Metrics used during Software Testing:

1) Test Coverage: Defined as Number of units (KLOC / FP) tested / total size of the system. (KLOC refers to Kilo Lines of Code)

2) Number of Tests per Unit Size : Defined as Number of Test Cases per KLOC / FP (KLOC refers to Kilo Lines of Code)

3) Acceptance Criteria Tested : Defined as Acceptance Criteria Tested / Total Acceptance Criteria

4) Percentage Testing cost : Defined as Actual Cost of Testing / Total Cost * 100

5) Cost to Find the Defect : Defined as Actual Cost of Testing / Number of Defects Found

6) Achievement of Budget: Defined as Actual Cost Of Testing / Budgeted Cost of Testing

7) Defects Detected during Testing: Defined as Defects Detected during Testing / Total number of System Defects

8) Defects Detected during production : Defined as Defects Detected during Production / System Size

9) Quality of Testing : Defined as No of defects detected during Testing / (No of defects detected during testing + No of Acceptance Defects Detected after the Delivery) * 100

10) System Complaints : Defined as Number of Customer Complaints / Number of Transactions Processed

11) Measure by the Scale of Ten : Defined as Assessment of Testing by assigning rating in terms of scale of 1 to 10

12) Source Code Analysis : Defined as Number of Source Code Statements Changed / Total Number Of Tests.

13) Productivity of Test Planning : Defined as No of Test Cases Designed / Actual Effort for doing Design and Documentation

14) Productivity of Test Execution : Defined as No of Test Cycles Executed / Actual Effort for Testing  

Many More Articles on Best Practices in Manual Testing 

Largest Database of Sample Papers - 1000+ Unique Questions for ISTQB Foundation Exam

ISTQB Foundation Exam - Full Crash Course for Download

ISTQB Advanced CTAL Test Analysts Exam - Full Crash Course for Download

ISTQB Advanced CTAL Test Manager Exam - Full Crash Course for Download

Consolidated Study Material - All ISTQB Certification Exams

What Successful Testers say about the Quality of this website

If you want to keep track of further articles on Software Testing,
I suggest you to subscribe my
RSS feed

You can also Subscribe by E-mail
and get All New articles delivered directly to your Inbox.

Quick Navigation of Software Testing Genius

Get your Absolutely Free Copy of Several MS PowerPoint Presentations & E-Books related to ISTQB, HP Load Runner, IBM RFT, HP QTP & QC Certification Exams, prepared by Popular Writers & Trainers, by writing to:

Study Material for Certification Exams on Other Automation Tools:

Download Full Study Material - HP QTP & QC Certification Exams

Practical Roadmap to QTP Certification

Rehearsal of QTP in 1 Hr. -  Interview Questions

Study Material - HP LoadRunner Certification Exams for All Modules

Rehearsal of LoadRunner in 1 Hr. -  Interview Questions

Study Material - IBM RFT Certification Exam

Study Material to prepare for Manual Testing & QA:

Practical Roadmap to CSTE Certification

Consolidated Study Material - Testing & QA

DownLoad Link for Presentation:

Learn HP QuickTest Professional 10.0 (Part-1) (772 Kb)


Comments :


Leave Your Comments: (*) Marked Fields are Mandatory

You can apply basic formatting to the text

Name *
Email Address *
Speak your mind
sex hikayeleri