software testing genius

Reasons of Slow Death of Automated Regression Tests

Welcome to “Software Testing Genius”. Subscribe to my RSS feed for latest content on Software Testing.

Reasons of Slow Death of Automated Regression Tests

The biggest problem facing automated regression tests is decay and premature death. You design regression tests to detect problems in features you've already tested - problems due to programmers' mistakes when fixing bugs or adding new features.

Regression tests die over time due to following reasons:

Change of the user interface or output formats: These changes are the major cause of decay. Because of them, a large set of tests that used to pass will fail even though nothing obvious has been done to affect the features that the tests are aimed at. The changes may be so minor that manual regression testers would barely take notice. Automated tests, however, are sensitive and brittle, unable to differentiate between improvements and bugs.

Designed-in assumptions about the test environment:

size=2 face=Arial> Test suites may break when moved to different machines or when necessary resources are relocated.

Errors in maintenance: Automation engineers repairing tests make mistakes, introducing bugs into the test suites. Regression test suites thus develop regression bugs themselves.

Changing operators: Test suites may require special skills and knowledge to operate and maintain. When a test automation engineer is reassigned, valuable knowledge can walk away. For instance, person - A disables the disk I/O tests temporarily, due to a particular bug in the product, then leaves to take that programming job in some other place. Person A's replacement assumes that the tests are entirely obsolete and moves them to an archive, never to be seen again.

Testers invest in regression automation with the expectation that the investment will pay off in due time. Sadly, they often find that the tests stop working sooner than expected. The tests are out of sync with the product. They demand repair. They're no longer helping find bugs. Testers reasonably respond by updating the tests. But this can be harder than expected. Many automation engineers have found themselves spending too much time diagnosing test failures and repairing decayed tests.

Exercise intelligence when selecting tests to automate, because anything you automate, you'll have to maintain or abandon. Uncontrolled maintenance costs are probably the most common problem that automated regression test efforts face.

Ref: Notes from Lessons Learned in Software Testing - By Great Testing Guru - Cem Kaner

Many More Articles on Test Automation 

Study Material - HP QTP & QC Certification Exams

Study Material - IBM RFT Certification Exam

Study Material - HP LoadRunner Certification Exams for All Modules

Study Material - ISTQB Certification Exam

What Successful Testers say about the Quality of this website

If you want to keep track of further articles on Software Testing,
I suggest you to subscribe my
RSS feed

You can also Subscribe by E-mail
and get All New articles delivered directly to your Inbox.

Get your Absolutely Free Copy of Several MS PowerPoint Presentations & E-Books related to ISTQB, HP Load Runner, IBM RFT, HP QTP & QC Certification Exams, prepared by Popular Writers & Trainers, by writing to:

Full Study Material for ISTQB Certification Exams:

Largest Database of Sample Papers - 1000+ Unique Questions for ISTQB Foundation Exam

ISTQB Foundation Exam - Full Crash Course for Download

ISTQB Advanced CTAL Test Analysts Exam - Full Crash Course for Download

ISTQB Advanced CTAL Test Manager Exam - Full Crash Course for Download

Most Popular Topics in Demand:

Practical Roadmap to QTP Certification

Practical Roadmap to CSTE Certification

Consolidated Study Material - Testing & QA

Rehearsal of QTP in 1 Hr. -  Interview Questions


Comments :


Leave Your Comments: (*) Marked Fields are Mandatory

You can apply basic formatting to the text

Name *
Email Address *
Speak your mind
sex hikayeleri