Automated Oracle Comparators for Testing Web Applications
Author : Sprenkle, Sara; Pollock, Lori; Esquivel, Holly; Hazelwood, Barbara; Ecott, Stacey
Booktitle : 18th IEEE International Symposium on Software Reliability Engineering
Date : Nov 2007
Publisher : IEEE
Pages : 117–126
Description : An earlier version of this work appeared in Tech Report 2007-339.
Keyword(s) : oracles, web applications, automated, testing
Document Type : In Conference Proceedings
Software developers need automated techniques to maintain the correctness of complex, evolving Web applications. While there has been success in automating some of the testing process for this domain, there exists little automated support for verifying that the executed test cases produce expected results. We assist in this tedious task by presenting a suite of automated oracle comparators for testing Web applications. To effectively identify failures, each comparator is specialized to particular characteristics of the possibly nondeterministic Web applications’ output in the form of HTML responses. We also describe combinations of comparators designed to achieve both high precision and recall in failure detection and a tool for helping testers to analyze the output of multiple oracles in detail. We present results from an evaluation of the effectiveness and costs of the oracle comparators. We also provide recommendations to testers on applying effective oracle comparators based on their application’s characteristics.