Difference between Automated Testing and Having Automated Tests
Before drawing a comparison among these two visibly similar but fairly distinct terms let us understand the test automation process in totality.
We know that before executing the test cases, lot of groundwork related to setting up of several test prerequisites need to be done. This type of groundwork is usually termed as pre processing activity that is essentially done before the test execution can begin.
Similarly after the execution of the test cases, certain set of activities related to management of test result artifacts become essential. The management of such activities is termed as post processing.
Automation engineers usually go in for a common mechanism to automate all these pre & post processing activities. Performing these activities manually is extremely time consuming as well as prone to errors. Thus having an automation system for these in place, the new test cases just need to specify these tasks, instead of prescribing full details of their implementation.
We can�t say that we are doing automated testing if the tester has to be involved at regular intervals during a series of tests simply to restore data, etc. We can�t expect to have an unattended overnight testing if some sort of manual intervention is required.
picture describes various tasks necessary to perform a number of test cases and how automated testing is different to having automated tests. Major difference lies in the automation of the pre and post processing activities.
For distinction, the automated processes have been shown in yellow, while manual processes have been shown in blue.
There is a difference in the sequence of the tasks shown in the above picture If tests are run manually, the analysis of results is usually done immediately after the comparison between actual outcome versus expected outcome. The software-testing engineer will spend time analyzing the reasons of such differences, and whether it is the problem with the software or with the test itself. With automated testing, all analysis of differences is postponed until the tests have run.
For example, If a test suite happened to run weekend night, the very next day the software testing engineer would be occupied viewing the results of the failed tests with a view to analyze whether it is the problem with the software, or some other factor had probably affected the automated test. This can consume good amount of time, hence a wise software testing engineer plans these activities well beforehand.
Ref: Effective use of test execution tools by M. Fewster for detailed reading